Client Recognition

This example shows how to recognize images in the viewfinder and overlay it with images.

For a better understanding, here are some terms that will be used in the following and other section of this documentation related to vision-based augmented reality.

Simple Client Tracking Android

In this section we will go through the code of the SimpleClientTrackingActivity, which you can find in the example application under the package com.wikitude.samples.recognition.client. We will discuss general concepts on how to use the Wikitude Native SDK as we go along, please don't skip this section even if you are for example only interested in cloud recognition.

The WikitudeSDK class is structured to be used in a standard Android activity and to make use of the activities life cycle events. We will use interfaces to communicate to the WikitudeSDK which type of rendering method we would like to use and to provide the necessary callbacks for Client- and Cloud-Trackers.

First let us have a look at the declaration of the activity class.

public class SimpleClientTrackingActivity extends Activity implements ClientTrackerEventListener, ExternalRendering {
...
}

We subclass the standard Android activity and implement the interfaces ClientTrackerEventListener and ExternalRendering. Later on when we create the instance of the WikitudeSDK we will pass the this pointer of our activity to the WikitudeSDK constructor and this way indicate our chosen type of rendering. In this example we will use external rendering, for details on how to setup the rendering and the difference between internal and external rendering please read through the section on rendering.

The next step we will take is create an instance of the WikitudeSDK class. This and the StartupConfiguration are the only objects you will need to create by yourself, every other object will be created by factory methods of the WikitudeSDK instance.

IMPORTANT: Do not instantiate any other class out of the Wikitude Native SDK other than the WikitudeSDK and the WikitudeSDKStartupConfiguration.

We are now going through each method of the SimpleClientTrackingActivity class, starting with onCreate. In the onCreate method we instantiate an instance of the WikitudeSDK, and an instance of the WikitudeSDKStartupConfiguration, which will hold our license key. If you do not have a license key yet, read this chapter on how to obtain a free trial key. After that we are ready to propagate the onCreate life cycle event to the Wikitude SDK. It is very important that you do propagate onCreate, onPause and onResume otherwise the Wikitude SDK won't be able to properly manage its resources which will lead to unexpected behavior.

After we called the WikitudeSDK onCreate method the SDK is initialized and we are now able to create a ClientTracker. To do that we get the TrackerManager from the WikitudeSDK instance and call createClientTracker passing the url to the WTC file. Since we are loading an asset on the device we indicate that by starting the url with the string file:///android_asset/ and add the path to the file starting from the asset root directory.

To get notified when the Tracker finished loading, recognized a target and so on we register the Activity which implements ClientTrackerEventListener as a Listener on the newly created Tracker.

@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    _wikitudeSDK = new WikitudeSDK(this);
    WikitudeSDKStartupConfiguration startupConfiguration = new WikitudeSDKStartupConfiguration(WikitudeSDKConstants.WIKITUDE_SDK_KEY);
    _wikitudeSDK.onCreate(getApplicationContext(), startupConfiguration);
    ClientTracker tracker = _wikitudeSDK.getTrackerManager().createClientTrackerFromUrl("file:///android_asset/magazine.wtc");
    tracker.registerTrackerEventListener(this);
}

The next important method to look at is the onRenderExtensionCreated method. Since we decided to use the external rendering functionality by implementing ExternalRendering, the WikitudeSDK provides us with a RenderExtension. The RenderExtension interface exposes the same methods as the standard GLSurfaceView.Renderer. In your custom GLSurfaceView.Renderer the first thing to do in every method is to always call the same method on the provided RenderExtension. To be able to that we pass the RenderExtension to the constructor of our Renderer, create our SurfaceView, initialize a Driver and set our SurfaceView as our content view.

@Override
public void onRenderExtensionCreated(final RenderExtension renderExtension_) {
    _glRenderer = new GLRenderer(renderExtension_);
    _view = new CustomSurfaceView(getApplicationContext(), _glRenderer);
    _driver = new Driver(_view, 30);
    setContentView(_view);
}

Next we will have a look at the ClientTrackerEventListener interface. The onErrorLoading method will be called like the name suggest when the Wikitude SDK wasn't able to load the client tracker. The most likely cause of this to happen would be that either the path to the WTC file was incorrect or the WTC was corrupted. The onTrackerFinishedLoading method will be called once when the tracker was successfully loaded. When the client tracker first recognizes a target it will call onTargetRecognized providing you with the recognized target name. When the client tracker starts tracking this target it will call onTracking continuously until it loses the target and finishes tracking with a call to onTargetLost.

The RecognizedTarget object provided in the onTracking method contains information about the tracked target like the name, the distance to the target and most importantly the matrices which describe where on the camera frame the target was found. Since we need this information to draw on the target we pass the object to our renderer in the 'onTracking' method and remove it when we lose the target in the onTargetLost method.

@Override
public void onErrorLoading(final ClientTracker clientTracker_, final String errorMessage_) {
    Log.v(TAG, "onErrorLoading: " + errorMessage_);
}

@Override
public void onTrackerFinishedLoading(final ClientTracker clientTracker_, final String trackerFilePath_) {

}

@Override
public void onTargetRecognized(final ClientTracker tracker_, final String targetName_) {

}

@Override
public void onTracking(final ClientTracker tracker_, final RecognizedTarget recognizedTarget_) {
    _glRenderer.setCurrentlyRecognizedTarget(recognizedTarget_);
}

@Override
public void onTargetLost(final ClientTracker tracker_, final String targetName_) {
    _glRenderer.setCurrentlyRecognizedTarget(null);
}

Extended Client Tracking Android

Extended tracking is an optional mode you can set for each target separately. In this mode the Wikitude SDK will try to continue to scan the environment of the user even if the original target image is not in view anymore. So the tracking extends beyond the limits of the original target image. The performance of this feature depends on various factors like computing power of the device, background texture and objects.

Based on the previous sample, to enable Extended Tracking for a tracker you need to provide a String array which defines which targets should be extended. In this sample we simply set a wildcard * so that all targets in this tracker are extended.

ClientTracker tracker = _wikitudeSDK.getTrackerManager().createClientTracker("file:///android_asset/magazine.wtc", new String[]{"*"});