Using Wikitude’s Object Tracking in an Ionic application

Augmented Reality (aka AR) is all about adding virtual objects to our world. The ultimate goal is to add those virtual objects on every real life object. Wikitude makes this process quite simple and we are going to implement it in an Ionic application.

If you are a beginner, you should have a look at the previous tutorials (here and there).

Wikitude Object Tracking Setup

Before diving into the Ionic application, a setup phase is required.
The hardest task is recording the real life object.
The target manager is where the video will be uploaded to create a WTO file. Depending on the end result, the object will be more or less easy to track.

Wikitude's object recognition technology is quite new, some objects are easier to track than others.

Here are the official restrictions:

• The object should not cast any clearly visible shadows
• Reflective surfaces should be avoided
• There should not be any disturbances of other objects in the scene
• The object should not change its original form. For example, scanning a shoe could be a problem since the form of the shoe could change. If the strings for example are tied or not could result in a different object.
• Typical objects that work well are small-scale objects ranging up to 1,5 meters

They made a video showing a good example:

https://vimeo.com/225822978

An in-depth tutorial is available there.

Here is the video that scored me 3 stars on the target manager:

I didn't need a perfect environment, nor scanning the whole object, the front part was good enough for this tutorial.

Once the video is ready, a new project is created in Wikitude's target manager:

In the previous tutorial, the project's type was 'image' because we were working on the image recognition feature. This time we select 'object'.

Once the video is uploaded, we can download the WTO file:

When it comes to virtual objects, we can create our own 3D models by using software like 3DSMax, SketchUp or Blender.
If we want to skip this step, there are other websites like Turbosquid where 3D objects are available for free or at a small price. We will need to focus on FBX files, they are the easiest to convert to the WT3 format (Wikitude's 3D model format).

Next step, downloading Wikitude3DEncoder from the download page for transforming the FBX to a WT3 file.

Once we have our files ready, we can start coding!

The Ionic Application

Let's start with changing the requirement in the home.ts file:

requiredFeatures = ["object_tracking"];

We can have the most elegant and performant code in the world, if the phone doesn't support the feature, it's not going to work.

Here is the assets folder's architecture:

The object-tracking.ts file starts as follow:

declare var AR;

class World {

  init() {
    const targetCollectionResource = 
      new AR.TargetCollectionResource("assets/wto/pinguin-tracker.wto", { });

    const objectTracker = new AR.ObjectTracker(targetCollectionResource, { });

    ...
  }
};

let world = new World();

world.init();

A new World instance is created then initiated. The init method will create a new TargetCollectionResource instance, this object uses the file we acquired from Wikitude's target manager. An ObjectTracker will track every object that belong to this collection.

Creating the 3D model now:

var scale = 0.001;

const drawableModel = new AR.Model("assets/wt3/mug.wt3", {
  scale: {
    x: scale,
    y: scale,
    z: scale
  },
  translate: {
    x: 0.15,
    z: 1
  },
  rotate: {
    x: -90
  }
});

A new Model is created, its scale is quite small because the model itself is quite big so we reduce its size. It's translated on the x and z axis otherwise it will just overlap with the target, finally it's rotated.

We can now mix together the objectTracker and the drawableModel:

const objectTrackable = new AR.ObjectTrackable(objectTracker, "*", {
  drawables: {
    cam: drawableModel
  }
});

Here is the final result:

Conclusion

Wikitude's Object Recognition technology is quite new (July 13th 2017) and will evolve with time. At the moment the object recognition mechanism is restricted, those restrictions will certainly disappear over time. Once we have the WTO file ready, we can work on the virtual world by first importing our FBX models in the Wikitude3DEncoder app to acquire a WT3 file. Those models can finally be placed in the Ionic application.