Client Recognition
This example shows how to recognize images in the viewfinder and overlay it with images.
For a better understanding, here are some terms that will be used in the following and other section of this documentation related to vision-based augmented reality.
Target: A target image and its associated extracted data that is used by the tracker to recognize an image.
Target collection: An archive storing a collection of targets that can be recognized by the tracker. A target collection can hold up to 1000 targets.
ClientTracker: The tracker analyzes the live camera image and detects the targets stored in its associated target collection. Multiple trackers can be created, however only one tracker can be active for recognition at any given time.
Simple Client Tracking Android
In this section we will go through the code of the SimpleClientTrackingActivity
, which you can find in the example application under the package com.wikitude.samples.recognition.client
. We will discuss general concepts on how to use the Wikitude Native SDK as we go along, please don't skip this section even if you are for example only interested in cloud recognition.
The WikitudeSDK
class is structured to be used in a standard Android activity and to make use of the activities life cycle events. We will use interfaces to communicate to the WikitudeSDK
which type of rendering method we would like to use and to provide the necessary callbacks for Client- and Cloud-Trackers.
First let us have a look at the declaration of the activity class.
public class SimpleClientTrackingActivity extends Activity implements ClientTrackerEventListener, ExternalRendering {
...
}
We subclass the standard Android activity and implement the interfaces ClientTrackerEventListener
and ExternalRendering
. Later on when we create the instance of the WikitudeSDK we will pass the this
pointer of our activity to the WikitudeSDK constructor and this way indicate our chosen type of rendering. In this example we will use external rendering, for details on how to setup the rendering and the difference between internal and external rendering please read through the section on rendering.
The next step we will take is create an instance of the WikitudeSDK
class. This and the StartupConfiguration
are the only objects you will need to create by yourself, every other object will be created by factory methods of the WikitudeSDK
instance.
We are now going through each method of the SimpleClientTrackingActivity
class, starting with onCreate
. In the onCreate
method we instantiate an instance of the WikitudeSDK
, and an instance of the WikitudeSDKStartupConfiguration
, which will hold our license key. If you do not have a license key yet, read this chapter on how to obtain a free trial key. After that we are ready to propagate the onCreate life cycle event to the Wikitude SDK. It is very important that you do propagate onCreate, onPause and onResume otherwise the Wikitude SDK won't be able to properly manage its resources which will lead to unexpected behavior.
After we called the WikitudeSDK onCreate method the SDK is initialized and we are now able to create a ClientTracker
. To do that we get the TrackerManager
from the WikitudeSDK
instance and call createClientTracker
passing the url to the WTC file. Since we are loading an asset on the device we indicate that by starting the url with the string file:///android_asset/ and add the path to the file starting from the asset root directory.
To get notified when the Tracker finished loading, recognized a target and so on we register the Activity which implements ClientTrackerEventListener as a Listener on the newly created Tracker.
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
_wikitudeSDK = new WikitudeSDK(this);
WikitudeSDKStartupConfiguration startupConfiguration = new WikitudeSDKStartupConfiguration(WikitudeSDKConstants.WIKITUDE_SDK_KEY);
_wikitudeSDK.onCreate(getApplicationContext(), startupConfiguration);
ClientTracker tracker = _wikitudeSDK.getTrackerManager().createClientTrackerFromUrl("file:///android_asset/magazine.wtc");
tracker.registerTrackerEventListener(this);
}
The next important method to look at is the onRenderExtensionCreated
method. Since we decided to use the external rendering functionality by implementing ExternalRendering
, the WikitudeSDK
provides us with a RenderExtension. The RenderExtension interface exposes the same methods as the standard GLSurfaceView.Renderer
. In your custom GLSurfaceView.Renderer
the first thing to do in every method is to always call the same method on the provided RenderExtension. To be able to that we pass the RenderExtension to the constructor of our Renderer
, create our SurfaceView
, initialize a Driver
and set our SurfaceView
as our content view.
@Override
public void onRenderExtensionCreated(final RenderExtension renderExtension_) {
_glRenderer = new GLRenderer(renderExtension_);
_view = new CustomSurfaceView(getApplicationContext(), _glRenderer);
_driver = new Driver(_view, 30);
setContentView(_view);
}
Next we will have a look at the ClientTrackerEventListener
interface. The onErrorLoading
method will be called like the name suggest when the Wikitude SDK wasn't able to load the client tracker. The most likely cause of this to happen would be that either the path to the WTC file was incorrect or the WTC was corrupted. The onTrackerFinishedLoading
method will be called once when the tracker was successfully loaded. When the client tracker first recognizes a target it will call onTargetRecognized
providing you with the recognized target name. When the client tracker starts tracking this target it will call onTracking
continuously until it loses the target and finishes tracking with a call to onTargetLost
.
The RecognizedTarget
object provided in the onTracking
method contains information about the tracked target like the name, the distance to the target and most importantly the matrices which describe where on the camera frame the target was found. Since we need this information to draw on the target we pass the object to our renderer in the 'onTracking' method and remove it when we lose the target in the onTargetLost
method.
@Override
public void onErrorLoading(final ClientTracker clientTracker_, final String errorMessage_) {
Log.v(TAG, "onErrorLoading: " + errorMessage_);
}
@Override
public void onTrackerFinishedLoading(final ClientTracker clientTracker_, final String trackerFilePath_) {
}
@Override
public void onTargetRecognized(final ClientTracker tracker_, final String targetName_) {
}
@Override
public void onTracking(final ClientTracker tracker_, final RecognizedTarget recognizedTarget_) {
_glRenderer.setCurrentlyRecognizedTarget(recognizedTarget_);
}
@Override
public void onTargetLost(final ClientTracker tracker_, final String targetName_) {
_glRenderer.setCurrentlyRecognizedTarget(null);
}
Extended Client Tracking Android
Extended tracking is an optional mode you can set for each target separately. In this mode the Wikitude SDK will try to continue to scan the environment of the user even if the original target image is not in view anymore. So the tracking extends beyond the limits of the original target image. The performance of this feature depends on various factors like computing power of the device, background texture and objects.
Based on the previous sample, to enable Extended Tracking for a tracker you need to provide a String array which defines which targets should be extended. In this sample we simply set a wildcard *
so that all targets in this tracker are extended.
ClientTracker tracker = _wikitudeSDK.getTrackerManager().createClientTracker("file:///android_asset/magazine.wtc", new String[]{"*"});