Build an Advanced Surface Tracking Experience

Advanced UX Control and Customization for Experienced Developers

Introduction

This guide walks you through building an advanced surface tracking experience using the WebAR SDK with custom progress handlers, error management, and enhanced user experience. This builds upon the basic surface tracking guide by adding professional-grade features for production applications. Why Build an Advanced Experience?

• Professional UX: Custom loading screens with real-time progress indicators and comprehensive error handling provide a polished, branded user experience that builds trust and engagement.

• Real-time Monitoring: Track actual SDK loading progress with precise percentage and object count data instead of generic spinners, giving users meaningful feedback about loading status.

• Full Control on Camera Stream: Manage camera permissions, initialization, and error handling manually with external-camera-stream="true", enabling custom permission prompts and better user guidance for camera-related issues.

• Headless Mode: With minimal-ui="true" + on-progress + on-error, the screen goes completely white (no default SDK UI), then opens camera and starts the AR pipeline - giving you complete control to implement your own UI and user flow.

Prerequisites

Complete the basic surface tracking experience first, or have familiarity with basic WebAR SDK integration.

Understanding Custom Handlers

on-progress Attribute

The on-progress attribute allows you to receive real-time loading progress data from the WebAR SDK:

// Your custom progress handler receives this data:
function customProgressHandler(progress) {
    // progress.percentage - Loading completion (0-100)
    // progress.current - Current loaded objects  
    // progress.total - Total objects to load
}

Pass this function to on-progress attribute while loading WebAR SDK script:

<script src="./webar-sdk.min.js" 
    webar-mode="surface-tracking"
    auto-init="true"
    auto-start-tracking="true"
    minimal-ui="true"         
    on-progress="customProgressCallback" <!-- Required for custom progress bar -->>
</script>

Why use on-progress?

• Replace generic "Loading..." with actual progress percentages

• Show users detailed loading status (e.g., "Loading AR engine: 75%")

• Debug loading performance issues

• Provide smooth, professional loading experience

on-error Attribute

The on-error attribute provides comprehensive error information when something goes wrong:

// Your custom error handler receives detailed error data:
function customErrorHandler(error) {
    // error.code - Specific error type (e.g., 'ERR_CAMERA_PERMISSION_DENIED')
    // error.description - Human-readable error message
    // error.instruction - Suggested user action
    // error.browser - Browser information for debugging
    // error.timestamp
}

Error Codes You May Receive:

// Browser Compatibility Errors
'ERR_UNSUPPORTED_BROWSER'     // Browser doesn't support WebAR
'ERR_WEBRTC_NOT_SUPPORTED'    // Browser lacks WebRTC support
'ERR_FACEBOOK_BROWSER'        // Facebook in-app browser detected
'ERR_INSTAGRAM_BROWSER'       // Instagram in-app browser detected

// Orientation Errors
'ERR_LANDSCAPE_MODE'          // Device in unsupported landscape mode

// Permission Errors
'ERR_CAMERA_PERMISSION_DENIED'   // User denied camera access
'ERR_MOTION_PERMISSION_DENIED'   // User denied motion sensor access

// System Errors
'ERR_INITIALIZATION_FAILED'   // SDK failed to initialize
'ERR_TRACKING_FAILED'         // AR tracking failed to start
'ERR_RESOURCE_LOAD_FAILED'    // Failed to load SDK resources

// Network Errors
'ERR_NETWORK_ERROR'          // Network connectivity issues
'ERR_SERVER_ERROR'           // Server-side error

// Generic Errors
'ERR_UNKNOWN'                // Unspecified error

<script src="./webar-sdk.min.js"
    webar-mode="surface-tracking"
    auto-init="true"
    auto-start-tracking="true"
    minimal-ui="true"         
    on-error="customErrorHandler" <!-- Required for error details & customization -->>
</script>

Why use on-error?

• Show user-friendly error messages instead of browser console errors

• Provide specific instructions for different error types

• Track and debug issues in production

• Handle camera, sensor, and browser compatibility problems gracefully

minimal-ui Attribute

Set minimal-ui="true" to disable the SDK's default UI and enable your custom handlers.

<script src="./webar-sdk.min.js" 
    webar-mode="surface-tracking"
    auto-init="true"
    auto-start-tracking="true"
    minimal-ui="true"          <!-- Required for custom UX -->
    on-progress="customProgressCallback"
    on-error="customErrorCallback">
</script>

Why minimal-ui="true" is required:

• Disables SDK's built-in loading screen and error dialogs

• Allows your custom progress and error handlers to take full control

• Prevents conflicts between default UI and your custom UI

• Essential for professional, branded experiences

Process

Step 1: Project Structure Setup

Why this structure? Separating concerns makes your code maintainable and reusable across projects.

your-project/
├── index.html
├── custom-handlers.js    ← Reusable progress/error handling/splash screen
├── loader.js             ← Handles camera stream & sensors, may extend to your main application logic  
├── styles.css            ← Custom UI styling
├── models/
│   └── astronaut.glb
└── webar-sdk.min.js

Step 2: Create Custom Handler Functions

Why create this first? The SDK needs these functions available before it loads, so we define them early. Create custom-handlers.js:

custom-handlers.js

// ============================================
// CUSTOM WEBARSDK HANDLERS
// ============================================
// Load this BEFORE the WebAR SDK script

console.log('🔧 Loading custom WebAR handlers...');

function customProgressHandler(progress) {
    console.log('🔄 WebAR SDK Progress Data:', progress);
    
    // Why check for useful data? SDK may send empty callbacks initially
    const hasUsefulData = progress && (
        (progress.percentage !== undefined && progress.percentage !== null) ||
        (progress.current !== undefined && progress.current !== null) ||
        (progress.total !== undefined && progress.total !== null)
    );
    
    const progressBar = document.getElementById('splashProgressBar');
    const progressText = document.getElementById('splashProgressText');
    const progressDetails = document.getElementById('splashProgressDetails');
    
    if (hasUsefulData && progressBar && progressText) {
        // Use real SDK data when available
        console.log('🎯 Using real SDK progress data');
        stopFallbackProgress(); // Stop simulation
        
        const percentage = progress.percentage || 0;
        const current = progress.current || 0;
        const total = progress.total || 0;
        
        // Update progress bar with real SDK percentage
        progressBar.style.width = percentage + '%';
        progressText.textContent = `Loading SDK: ${Math.round(percentage)}%`;
        
        // Show detailed progress information
        if (progressDetails) {
            if (total > 0) {
                progressDetails.textContent = `Objects: ${current} of ${total} created`;
            } else {
                progressDetails.textContent = `Loading components...`;
            }
        }
        
        // Show raw SDK data for debugging
        const rawDataDiv = document.getElementById('splashRawData');
        if (rawDataDiv) {
            rawDataDiv.textContent = `Real SDK Progress: ${JSON.stringify({
                percentage: percentage,
                current: current,
                total: total
            }, null, 2)}`;
        }
        
    } else {
        // Why fallback? Provides smooth UX even when SDK data isn't available
        console.log('⚠️ No useful SDK progress data - using fallback simulation');
        
        // Start fallback only if it's not already running
        if (!fallbackProgressInterval) {
            startFallbackProgress();
        }
    }
}

function customErrorHandler(error) {
    console.error('🚨 Custom Error Handler:', error);
    
    // Why detailed error handling? Users need clear, actionable information
    let errorTitle = 'WebAR Error';
    let errorMessage = error.description || 'An error occurred';
    let errorInstruction = error.instruction || '';
    
    // Provide specific guidance for common errors
    switch (error.code) {
        case 'ERR_CAMERA_PERMISSION_DENIED':
            errorTitle = 'Camera Permission Required';
            errorMessage = 'AR experiences need camera access to work properly';
            errorInstruction = 'Please refresh and allow camera access when prompted';
            break;
        case 'ERR_WEBRTC_NOT_SUPPORTED':
            errorTitle = 'Browser Not Supported';
            errorMessage = 'Your browser doesn\'t support WebAR features';
            errorInstruction = 'Please use Safari on iOS or Chrome on Android';
            break;
        case 'ERR_LANDSCAPE_MODE':
            errorTitle = 'Please Rotate Device';
            errorMessage = 'This AR experience works best in portrait mode';
            errorInstruction = 'Rotate your device to portrait orientation';
            break;
        default:
            errorTitle = 'WebAR Error';
            errorMessage = error.description || 'Something went wrong';
    }
    
    // Show user-friendly error UI
    showErrorUI(errorTitle, errorMessage, errorInstruction, error.code);
}

// Why global? SDK needs to find these functions by name
window.customProgressCallback = customProgressHandler;
window.customErrorCallback = customErrorHandler;

console.log('✅ Custom WebAR handlers loaded and registered');

Step 3: Enhanced HTML with Advanced Configuration

Why load handlers before SDK? The SDK looks for callback functions immediately when it loads.

<!DOCTYPE html>
<html>
<head>
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no, viewport-fit=cover">
    <title>Advanced WebAR Surface Tracking</title>
    
    <!-- A-Frame (required before WebAR SDK) -->
    <script src="https://aframe.io/releases/1.6.0/aframe.min.js"></script>
    
    <!-- Load custom handlers BEFORE SDK - Why? SDK needs these functions available immediately -->
    <script src="custom-handlers.js"></script>
    
    <!-- WebAR SDK with advanced configuration -->
    <script src="./webar-sdk.min.js" 
        webar-mode="surface-tracking"
        auto-init="true"
        auto-start-tracking="true"
        on-progress="customProgressCallback"    ← Your progress function name
        on-error="customErrorCallback"          ← Your error function name  
        webar-mode="surface-tracking"
        auto-init="true"                        ← Automatically start SDK
        auto-start-tracking="true"              ← Automatically start tracking
        minimal-ui="true"                       ← REQUIRED: Disables default UI
        render-scene-on-desktop="false"         ← Mobile-first development
        external-camera-stream="true">          ← Manual camera control
    </script>
</head>

Critical Configuration Explained:

minimal-ui="true" - REQUIRED for custom UX

• What it does: Disables SDK's default loading screen and error dialogs

• Why required: Allows your custom handlers to control the entire user experience

• Without it: Your custom progress/error handlers won't be used

on-progress="customProgressCallback"

• What it does: Tells SDK to call your customProgressCallback function with loading data

• Why useful: Get real-time loading progress instead of generic spinners

• Data received: percentage, current objects loaded, total objects

on-error="customErrorCallback"

• What it does: Tells SDK to call your customErrorCallback function when errors occur

• Why essential: Handle camera permissions, browser compatibility, and other issues gracefully

• Data received: error code, description, user instructions, browser info

external-camera-stream="true"

• What it does: Lets you manage camera permissions and stream manually

• Why important: Better user experience with custom permission prompts

• Enables: iOS permission handling, custom camera error messages

Step 4: Enhanced Loading Screen / Custom Splash Screen

Why custom loading screen? Professional apps need branded, informative loading experiences.

With minimal-ui="true", the SDK provides no default UI - you must create your own splash screen and manage its lifecycle. Please note that you may use the on-load attribute to override the default SDK splash screen while keeping other SDK UI elements when minimal-ui="false" .

Critical Timing: Load Splash Screen Before SDK loads

Important: Load your custom-handlers.js script BEFORE the WebAR SDK script. With minimal-ui="true", the SDK provides no default UI, so without your custom handlers loaded first, users will see a blank white screen during SDK initialization. Your custom handlers automatically create and inject the splash screen when the SDK starts loading.

<head>
    <!-- A-Frame -->
    <script src="https://aframe.io/releases/1.6.0/aframe.min.js"></script>
    
    <!-- Load custom handlers BEFORE SDK -->
    <script src="custom-handlers.js"></script>
    
    <!-- WebAR SDK with minimal-ui -->
    <script src="./webar-sdk.min.js" 
        webar-mode="surface-tracking"
        auto-init="true"
        minimal-ui="true"                      ← Disable ALL SDK UI
        on-progress="customProgressCallback"   ← Get progress updates, may use the data to update loading screen UI
        >
    </script>
</head>

Why this order matters:

• minimal-ui="true" disables all SDK UI

• SDK immediately calls customProgressCallback when loading starts

• If your handlers aren't loaded yet → white screen

• With handlers loaded first → instant professional splash screen

Your custom-handlers.js contains all the splash screen creation logic, so no additional HTML or CSS setup is required in your main page.

Step 5: Camera and Sensor Management

Why manual camera management? Better control over permissions and error handling.

loader.js

const iOS = /iPad|iPhone|iPod/.test(navigator.userAgent);
let camstream = null;
let sdkloaded = false;

// Why check iOS separately? iOS requires user interaction before camera access
function handleCameraPermissions() {
    if (iOS) {
        showPermissionPrompt('camera');
    } else {
        startCameraStream();
    }
}

function startCameraStream() {
    navigator.mediaDevices.getUserMedia({ 
        video: { 
            facingMode: 'environment',
            advanced: [{ focusMode: 'manual', focusDistance: 0 }]
        } 
    })
    .then(function(stream) {
        camstream = stream;
        const videoElement = document.getElementById('video');
        videoElement.srcObject = stream;
        
        // Why wait for gyro? Tracking needs motion sensors
        handleMotionPermissions();
    })
    .catch(function(error) {
        console.error('Camera access failed:', error);
        customErrorHandler({
            code: 'ERR_CAMERA_PERMISSION_DENIED',
            description: 'Camera access was denied',
            instruction: 'Please refresh and allow camera access'
        });
    });
}

// Why motion permissions? AR tracking requires device orientation data
function handleMotionPermissions() {
    if (typeof DeviceMotionEvent.requestPermission === 'function') {
        DeviceMotionEvent.requestPermission().then(response => {
            if (response === 'granted') {
                initializeTracking();
            } else {
                customErrorHandler({
                    code: 'ERR_MOTION_PERMISSION_DENIED',
                    description: 'Motion sensor access was denied',
                    instruction: 'Motion sensors are required for AR tracking'
                });
            }
        });
    } else {
        // Non-iOS devices
        initializeTracking();
    }
}

function initializeTracking() {
    if (sdkloaded && camstream) {
        WEBARSDK.SetCameraStream(camstream);
    }
}

// SDK callback when ready
WEBARSDK.SetAppReadyCallback(() => {
    sdkloaded = true;
    initializeTracking();
});

// SDK callback when video starts
WEBARSDK.SetVideoStartedCallback(() => {
    // Hide loading screen when AR is ready
    const loadingScreen = document.getElementById('loadingscreen');
    if (loadingScreen) {
        loadingScreen.style.display = 'none';
    }
    
    // Remove original video element (SDK creates its own)
    const videoElement = document.getElementById('video');
    if (videoElement && videoElement.parentElement) {
        videoElement.parentElement.removeChild(videoElement);
    }
});

Step 6: Complete WebAR Scene

<!-- WebAR Scene -->
<div id="aframe-container">
    <a-scene id="mainScene" 
        webar-scene="key: YOUR_LICENSE_KEY_HERE" 
        xr-mode-ui="enabled: false"
        device-orientation-permission-ui="enabled: false"
        loading-screen="enabled: false"
        renderer="antialias: true; colorManagement: true; physicallyCorrectLights: true">
        
        <a-camera webar-camera></a-camera>

        <a-assets>
            <a-asset-item id="astronaut" src="models/astronaut.glb"></a-asset-item>
        </a-assets>

        <!-- Why webar-stage? Represents the tracked surface -->
        <a-entity webar-stage>
            <!-- Why webar-ux-control? Enables user interaction with AR objects -->
            <a-entity webar-ux-control="stageCursorUX: true; userGestureRotation: true; userGestureScale: true">
                <a-entity gltf-model="#astronaut" 
                    position="0 0 0" 
                    rotation="0 0 0" 
                    scale="0.25 0.25 0.25"
                    webar-loadmonitor="elType: glb"
                    class="clickable">
                </a-entity>
            </a-entity>
        </a-entity>
    </a-scene>
</div>

Advanced Features Explained

Real-time Progress Monitoring

The customProgressHandler receives actual SDK loading data including:

• progress.percentage - Loading completion (0-100)

• progress.current - Current loaded objects

• progress.total - Total objects to load

External Camera Stream

Using external-camera-stream="true" gives you full control over camera initialization, allowing for custom permission handling and better user experience.

Comprehensive Error Handling

The error handler provides detailed error information and user-friendly messages for different error scenarios.

Deployment Considerations

1. HTTPS Required: WebAR requires HTTPS for camera access

2. License Key: Replace YOUR_LICENSE_KEY_HERE with your actual license key

3. Asset Optimization: Optimize 3D models for web delivery

4. Browser Testing: Test across Safari (iOS) and Chrome (Android)

Full Source Code

The complete implementation is available in the WebAR SDK repository under /aframe/advanced-surface-tracking/.

Common Issues and Solutions

Progress handler not called?

• Ensure minimal-ui="true" is set

• Check that custom-handlers.js loads before the SDK

• Verify function names match exactly

Error handler not working?

• Confirm on-error attribute points to correct function name

• Check browser console for JavaScript errors

• Ensure error UI elements exist in DOM

Camera permissions failing?

• Verify HTTPS is enabled

• Test iOS/Android permission prompt timing

• Check camera constraints are supported

Key Benefits of This Advanced Implementation

Your advanced surface tracking experience will include:

• Professional loading screen with real-time progress

• Comprehensive error handling and user guidance

• Smooth camera and sensor permission management

• Enhanced debugging capabilities for development

• Modular code architecture for easy maintenance

This advanced implementation provides a professional, production-ready foundation for WebAR applications with comprehensive UX control and error handling.