Build a Basic Surface Tracking Experience

Construction of an AR experience using SLAM

Blippar Documentation Centre

Surface Tracking

Create a basic scene

• Create a basic scene from playground.babylonjs.com and download the source code by clicking the Download button.

• Or you can download the already generated basic scene code from this link.

Download Basic Scene Code

• Unzip the downloaded code and open the index.html file in a code editor of your choice.

This basic scene source code from Babylon.js playground has the code to create the Babylon rendering engine and construct a scene with a sphere and a ground plane model.

• It also has many javascript libraries which are not necessary for this example.

Delete all the script tags and include only the following Babylon.js scripts as shown in below code sample

<!-- Babylon.js -->
<!-- Delete all the other scripts and add only the following scripts -->
<script src="https://cdn.babylonjs.com/babylon.js"></script>
<script src="https://cdn.babylonjs.com/loaders/babylonjs.loaders.min.js"></script>

This document is compatible with Babylon.js version 4.2.0. Also delete these redundant lines from the playground generated code which are not necessary for webar experience.

// This targets the camera to scene origin
camera.setTarget(BABYLON.Vector3.Zero());

// This attaches the camera to the canvas
camera.attachControl(canvas, true);

Add WebAR SDK script

Add the webar sdk script tag with the following attributes:

• webar-mode="surface-tracking"

• auto-init="true"

• autp-start="true"

• rendering-engine="babylonjs"

Setting the webar-mode attribute to "surface-tracking" makes the webar sdk run in surface tracking mode.

As a query parameter in the https url of webar sdk script, paste your license key obtained from Blippar hub which will be in the following form:

?license-key=xxxxxxxx-1111-2222-3333-yyyyyyyyyyyy

• Make sure this webar sdk script is added only after the babylonjs script has been added in the previous section.

• There are other attributes to customise the way the loading progress bar screen appears. Please refer API Customization documentation for more details.

<script src="https://webar-sdk.blippar.com/releases/latest/webar-sdk.min.js?license-key=xxxxxxxx-1111-2222-3333-yyyyyyyyyyyy"
  webar-mode="surface-tracking"
  auto-init="true"
  auto-start="true"
  rendering-engine="babylonjs">
</script>

Create a webar stage mesh

Webar Stage is an empty Babylon mesh object in the scene. It is passed as one of the arguments of WebAR SDK's InitBabylonJS API which will be used by the sdk to place it at the origin of the detected surface. In this example, we will use this webar stage as a parent for the 3D primitive mesh object, GLTF model mesh object and the particle system object so that they appear on the detected surface.

Under the createScene function:

• Delete these lines of playground generated code which creates sphere and ground meshes.

// Our built-in 'sphere' shape.
var sphere = BABYLON.MeshBuilder.CreateSphere("sphere", {diameter: 2, segments: 32}, scene);

// Move the sphere upward 1/2 its height
sphere.position.y = 1;

// Our built-in 'ground' shape.
var ground = BABYLON.MeshBuilder.CreateGround("ground", {width: 6, height: 6}, scene);

• Create an empty mesh in a variable webarStage.

var webarStage = new BABYLON.Mesh("webarStage", scene);

Integrate Enhanced UX with Stage Cursor, Rotation, and Scaling

This step is mandatory to ensure a fully interactive AR experience. The WebAR UX Control is an empty Babylon mesh object in the scene, passed as one of the arguments to the WebAR SDK's InitBabylonJS API.

Integrating this feature enhances the AR experience by adding a stage cursor, allowing users to place the AR model with precision. It also introduces gesture-based interactions such as rotation and scaling, making the scene more interactive and engaging. This enhancement is implemented using a mesh named webar-ux-control.

Under the createScene function, after creation of webar stage:

• Create the webar-ux-control Mesh: Instantiate a new Babylon.js Mesh with the id webar-ux-control. Set this mesh as a child of the webarStage mesh to ensure it appears on the surface.

// Create a 'webar-ux-control' Mesh
var webarUXControl = new BABYLON.Mesh("webar-ux-control", scene);
webarUXControl.setParent(webarStage); // Set webarStage as parent to webarUXControl mesh

// Set webar options to enable stage cursor UX, user gesture rotation, and scaling
webarUXControl.webarOptions = {
    stageCursorUX: true,
    userGestureRotation: true,
    userGestureScale: true
};

• Configure Interaction Options: The webarOptions property of the webarUXControl mesh controls the interaction features. Setting stageCursorUX to true will display a cursor for model placement. userGestureRotation and userGestureScale allow for intuitive user interactions with the model.

Add a Cylinder mesh to webar stage

Under the createScene function, let's create a PBR material for the cylinder which continuously varies its tintColor based on time.

// Create a time varying PBR material for the base cylinder mesh
var pbr = new BABYLON.PBRMaterial("pbr", scene);
pbr.metallic = 0.0;
pbr.roughness = 0;
pbr.subSurface.isRefractionEnabled = true;
pbr.subSurface.indexOfRefraction = 1.5;
pbr.subSurface.tintColor = new BABYLON.Color3(0.5, 0, 0);
var a = 0;
scene.beforeRender = () => {
  a += 0.05;
  pbr.subSurface.tintColor.g = Math.cos(a) * 0.5 + 0.5;
  pbr.subSurface.tintColor.b = pbr.subSurface.tintColor.g;
}

Next, create a babylon cylinder mesh and set webarStage as its parent and assign the pbr material created above. This cylinder mesh appears as the pedestal of the oscar_trophy.glb model imported in the next section.

// Create primitive shape using mesh builder and set webarStage as its parent so that it appears on the surface
var cylinder = BABYLON.MeshBuilder.CreateCylinder("cylinder", {height: 0.5, diameterTop: 1, diameterBottom: 1, tessellation: 128}, scene);
cylinder.material = pbr;
cylinder.position.y = 0.125;
cylinder.position.z = -0.075;
cylinder.setParent(webarStage);

Add a GLB model to webar stage

Under the createScene function, import the GLB file oscar_trophy.glb using Babylon SceneLoader and set webarStage as the parent for all the mesh nodes.

Note:

• In SceneLoader ImportMesh API filter 'Object_2' node alone to exclude the pedestal part of the oscar_trophy.glb model from loading.

• When Babylon imports GLB model files, it creates a __root__ mesh as a parent node for all the meshes. So skip the __root__ mesh in setting the webarStage as its parent, depending on your project's need.

BABYLON.SceneLoader.ImportMesh(['Object_2'], "./models/", "oscar_trophy.glb", scene, function (meshes, particleSystems, skeletons) {
  let xQuat = new BABYLON.Quaternion();
  BABYLON.Quaternion.FromEulerAnglesToRef(Math.PI / 2, 0, 0, xQuat);
  for (mesh of meshes) {
    if (mesh.name !== '__root__') {
      // Move the loaded models to webarStage
      mesh.setParent(webarStage);
      mesh.rotationQuaternion.multiplyInPlace(xQuat);
      scaleByFactor(mesh, 0.0375);
    }
  }
});

Add a Particle System to webar stage

Under startParticles function,

• Create a sphere mesh named sphereSpark and set webarStage (via fountain) as its parent.

• Load spark_particles.json using Babylon ParticleHelper and set sphereSpark as its region

Similarly,

• Create a sphere mesh named sphereSmoke and set webarStage (via fountain) as its parent.

• Load smoke_particles.json using Babylon ParticleHelper and set sphereSmoke as its region

var sparksystem = null;
var smokesystem = null;

var startParticles = function (scene, fountain) {
  let sphereSpark = BABYLON.MeshBuilder.CreateSphere("sphereSpark", {diameter: 0.4, segments: 32}, scene);
  sphereSpark.isVisible = false;
  sphereSpark.setParent(fountain);
  BABYLON.ParticleHelper.ParseFromFileAsync(null, "./models/spark_particles.json", scene, false).then(system => {
    sparksystem = system;
    system.emitter = sphereSpark;
  });

  let sphereSmoke = BABYLON.MeshBuilder.CreateSphere("sphereSmoke", {diameter: 1.9, segments: 32}, scene);
  sphereSmoke.isVisible = false;
  sphereSmoke.setParent(fountain);
  BABYLON.ParticleHelper.ParseFromFileAsync(null, "./models/smoke_particles.json", scene, false).then(system => {
    smokesystem = system;
    system.emitter = sphereSmoke;
  });
};

Under the createScene function, call the startParticles function by passing scene and webarStage as its arguments.

  // Start spark and smoke particles
  startParticles(scene, webarStage);

Initialise WebAR SDK

Under the createScene function, call the Webar SDK's InitBabylonJs API. Once the webar sdk script is loaded, its APIs can be accessed using the WEBARSDK global variable. Please refer to the API Reference documentation for more details.

// Pass babylon canvas, scene, camera, webarStage mesh & webarUXControl(if defined above, an optional, otherwise pass 'null') to WebarSdk to initialize surface tracking
WEBARSDK.InitBabylonJs(canvas, scene, camera, webarStage, webarUXControl);

Deployment

Deploy your files (the HTML file and, if you downloaded the SDK, the blippar folder) to your web server

You can also develop and host locally in Dev Mode, see more info here Develop Locally.

Your website must have https enabled, this is to enable the browser to access your camera.

Navigate to your page, the WebAR SDK should work with any modern browser but is extensively tested with Safari (iOS) and Chrome (android). If all has gone well you will see the following results below. Information about launching the AR experience is available here.

Full Source Code

Full source code of this WebAR SDK BabylonJS Surface Tracking example can be seen in the Github repository.