Build an Advanced Surface Tracking Experience
Advanced UX Control and Customization for Experienced Developers
Introduction
This guide walks you through building an advanced surface tracking experience using the WebAR SDK with custom progress handlers, error management, and enhanced user experience. This builds upon the basic surface tracking guide by adding professional-grade features for production applications. Why Build an Advanced Experience?
Professional UX: Custom loading screens with real-time progress indicators and comprehensive error handling provide a polished, branded user experience that builds trust and engagement.
Real-time Monitoring: Track actual SDK loading progress with precise percentage and object count data instead of generic spinners, giving users meaningful feedback about loading status.
Full Control on Camera Stream: Manage camera permissions, initialization, and error handling manually with
external-camera-stream="true"
, enabling custom permission prompts and better user guidance for camera-related issues.Headless Mode: With
minimal-ui="true"
+on-progress
+on-error
, the screen goes completely white (no default SDK UI), then opens the camera and starts the AR pipeline - giving you complete control to implement your own UI and user flow.
Prerequisites
Complete the basic surface tracking experience first, or have familiarity with basic WebAR SDK integration.
Understanding Custom Handlers
on-progress
Attribute
on-progress
AttributeThe on-progress
attribute allows you to receive real-time loading progress data from the WebAR SDK:
// Your custom progress handler receives this data:
function customProgressHandler(progress) {
// progress.percentage - Loading completion (0-100)
// progress.current - Current loaded objects
// progress.total - Total objects to load
}
Pass this function to on-progress
attribute while loading WebAR SDK script:
<script src="./webar-sdk.min.js"
webar-mode="surface-tracking"
auto-init="true"
auto-start-tracking="true"
minimal-ui="true"
on-progress="customProgressCallback" <!-- Required for custom progress bar -->>
</script>
Why use on-progress?
Replaces generic "Loading..." with actual progress percentages
Shows users detailed loading status (e.g., "Loading AR engine: 75%")
Debugs loading performance issues
Provides smooth, professional loading experience
on-error
Attribute
on-error
AttributeThe on-error
attribute provides comprehensive error information when something goes wrong:
// Your custom error handler receives detailed error data:
function customErrorHandler(error) {
// error.code - Specific error type (e.g., 'ERR_CAMERA_PERMISSION_DENIED')
// error.description - Human-readable error message
// error.instruction - Suggested user action
// error.browser - Browser information for debugging
// error.timestamp
}
Error Codes You May Receive:
// Browser Compatibility Errors
'ERR_UNSUPPORTED_BROWSER' // Browser doesn't support WebAR
'ERR_WEBRTC_NOT_SUPPORTED' // Browser lacks WebRTC support
'ERR_FACEBOOK_BROWSER' // Facebook in-app browser detected
'ERR_INSTAGRAM_BROWSER' // Instagram in-app browser detected
// Orientation Errors
'ERR_LANDSCAPE_MODE' // Device in unsupported landscape mode
// Permission Errors
'ERR_CAMERA_PERMISSION_DENIED' // User denied camera access
'ERR_MOTION_PERMISSION_DENIED' // User denied motion sensor access
// System Errors
'ERR_INITIALIZATION_FAILED' // SDK failed to initialize
'ERR_TRACKING_FAILED' // AR tracking failed to start
'ERR_RESOURCE_LOAD_FAILED' // Failed to load SDK resources
// Network Errors
'ERR_NETWORK_ERROR' // Network connectivity issues
'ERR_SERVER_ERROR' // Server-side error
// Generic Errors
'ERR_UNKNOWN' // Unspecified error
<script src="./webar-sdk.min.js"
webar-mode="surface-tracking"
auto-init="true"
auto-start-tracking="true"
minimal-ui="true"
on-error="customErrorHandler" <!-- Required for error details & customization -->>
</script>
Why use on-error?
Shows user-friendly error messages instead of browser console errors
Provides specific instructions for different error types
Tracks and debug issues in production
Handles camera, sensor, and browser compatibility problems gracefully
minimal-ui
Attribute
minimal-ui
Attribute Set minimal-ui="true"
to disable the SDK's default UI and enable your custom handlers.
<script src="./webar-sdk.min.js"
webar-mode="surface-tracking"
auto-init="true"
auto-start-tracking="true"
minimal-ui="true" <!-- Required for custom UX -->
on-progress="customProgressCallback"
on-error="customErrorCallback">
</script>
Why minimal-ui="true" is required:
Disables SDK's built-in loading screen and error dialogs
Allows your custom progress and error handlers to take full control
Prevents conflicts between default UI and your custom UI
Essential for professional, branded experiences
Process
Step 1: Project Structure Setup
Why this structure? Separating concerns makes your code maintainable and reusable across projects.
your-project/
├── index.html
├── custom-handlers.js ← Reusable progress/error handling/splash screen
├── loader.js ← Handles camera stream & sensors, may extend to your main application logic
├── styles.css ← Custom UI styling
├── models/
│ └── astronaut.glb
└── webar-sdk.min.js
Step 2: Create Custom Handler Functions
Why create this first? The SDK needs these functions available before it loads, so we define them early. Create custom-handlers.js:
// ============================================
// CUSTOM WEBARSDK HANDLERS
// ============================================
// Load this BEFORE the WebAR SDK script
console.log('🔧 Loading custom WebAR handlers...');
function customProgressHandler(progress) {
console.log('🔄 WebAR SDK Progress Data:', progress);
// Why check for useful data? SDK may send empty callbacks initially
const hasUsefulData = progress && (
(progress.percentage !== undefined && progress.percentage !== null) ||
(progress.current !== undefined && progress.current !== null) ||
(progress.total !== undefined && progress.total !== null)
);
const progressBar = document.getElementById('splashProgressBar');
const progressText = document.getElementById('splashProgressText');
const progressDetails = document.getElementById('splashProgressDetails');
if (hasUsefulData && progressBar && progressText) {
// Use real SDK data when available
console.log('🎯 Using real SDK progress data');
stopFallbackProgress(); // Stop simulation
const percentage = progress.percentage || 0;
const current = progress.current || 0;
const total = progress.total || 0;
// Update progress bar with real SDK percentage
progressBar.style.width = percentage + '%';
progressText.textContent = `Loading SDK: ${Math.round(percentage)}%`;
// Show detailed progress information
if (progressDetails) {
if (total > 0) {
progressDetails.textContent = `Objects: ${current} of ${total} created`;
} else {
progressDetails.textContent = `Loading components...`;
}
}
// Show raw SDK data for debugging
const rawDataDiv = document.getElementById('splashRawData');
if (rawDataDiv) {
rawDataDiv.textContent = `Real SDK Progress: ${JSON.stringify({
percentage: percentage,
current: current,
total: total
}, null, 2)}`;
}
} else {
// Why fallback? Provides smooth UX even when SDK data isn't available
console.log('⚠️ No useful SDK progress data - using fallback simulation');
// Start fallback only if it's not already running
if (!fallbackProgressInterval) {
startFallbackProgress();
}
}
}
function customErrorHandler(error) {
console.error('🚨 Custom Error Handler:', error);
// Why detailed error handling? Users need clear, actionable information
let errorTitle = 'WebAR Error';
let errorMessage = error.description || 'An error occurred';
let errorInstruction = error.instruction || '';
// Provide specific guidance for common errors
switch (error.code) {
case 'ERR_CAMERA_PERMISSION_DENIED':
errorTitle = 'Camera Permission Required';
errorMessage = 'AR experiences need camera access to work properly';
errorInstruction = 'Please refresh and allow camera access when prompted';
break;
case 'ERR_WEBRTC_NOT_SUPPORTED':
errorTitle = 'Browser Not Supported';
errorMessage = 'Your browser doesn\'t support WebAR features';
errorInstruction = 'Please use Safari on iOS or Chrome on Android';
break;
case 'ERR_LANDSCAPE_MODE':
errorTitle = 'Please Rotate Device';
errorMessage = 'This AR experience works best in portrait mode';
errorInstruction = 'Rotate your device to portrait orientation';
break;
default:
errorTitle = 'WebAR Error';
errorMessage = error.description || 'Something went wrong';
}
// Show user-friendly error UI
showErrorUI(errorTitle, errorMessage, errorInstruction, error.code);
}
// Why global? SDK needs to find these functions by name
window.customProgressCallback = customProgressHandler;
window.customErrorCallback = customErrorHandler;
console.log('✅ Custom WebAR handlers loaded and registered');
Step 3: Enhanced HTML with Advanced Configuration
Why load handlers before SDK? The SDK looks for callback functions immediately when it loads.
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no, viewport-fit=cover">
<title>Advanced WebAR Surface Tracking</title>
<!-- A-Frame (required before WebAR SDK) -->
<script src="https://aframe.io/releases/1.6.0/aframe.min.js"></script>
<!-- Load custom handlers BEFORE SDK - Why? SDK needs these functions available immediately -->
<script src="custom-handlers.js"></script>
<!-- WebAR SDK with advanced configuration -->
<script src="./webar-sdk.min.js"
webar-mode="surface-tracking"
auto-init="true"
auto-start-tracking="true"
on-progress="customProgressCallback" ← Your progress function name
on-error="customErrorCallback" ← Your error function name
webar-mode="surface-tracking"
auto-init="true" ← Automatically start SDK
auto-start-tracking="true" ← Automatically start tracking
minimal-ui="true" ← REQUIRED: Disables default UI
render-scene-on-desktop="false" ← Mobile-first development
external-camera-stream="true"> ← Manual camera control
</script>
</head>
Critical Configuration Explained:
minimal-ui="true"
- REQUIRED for custom UX
What it does: Disables SDK's default loading screen and error dialogs
Why required: Allows your custom handlers to control the entire user experience
Without it: Your custom progress/error handlers won't be used
on-progress="customProgressCallback"
What it does: Tells SDK to call your customProgressCallback function with loading data
Why useful: Get real-time loading progress instead of generic spinners
Data received: percentage, current objects loaded, total objects
on-error="customErrorCallback"
What it does: Tells SDK to call your customErrorCallback function when errors occur
Why essential: Handle camera permissions, browser compatibility, and other issues gracefully
Data received: error code, description, user instructions, browser info
external-camera-stream="true"
What it does: Lets you manage camera permissions and stream manually
Why important: Better user experience with custom permission prompts
Enables: iOS permission handling, custom camera error messages
Step 4: Enhanced Loading Screen / Custom Splash Screen
Why custom loading screen? Professional apps need branded, informative loading experiences.
With minimal-ui="true"
, the SDK provides no default UI - you must create your own splash screen and manage its lifecycle. Please note that you may use the on-load
attribute to override the default SDK splash screen while keeping other SDK UI elements when minimal-ui="false"
.
Critical Timing: Load Splash Screen Before SDK loads
Important: Load your custom-handlers.js script BEFORE the WebAR SDK script. With minimal-ui="true"
, the SDK provides no default UI, so without your custom handlers loaded first, users will see a blank white screen during SDK initialization. Your custom handlers automatically create and inject the splash screen when the SDK starts loading.
<head>
<!-- A-Frame -->
<script src="https://aframe.io/releases/1.6.0/aframe.min.js"></script>
<!-- Load custom handlers BEFORE SDK -->
<script src="custom-handlers.js"></script>
<!-- WebAR SDK with minimal-ui -->
<script src="./webar-sdk.min.js"
webar-mode="surface-tracking"
auto-init="true"
minimal-ui="true" ← Disable ALL SDK UI
on-progress="customProgressCallback" ← Get progress updates, may use the data to update loading screen UI
>
</script>
</head>
Why this order matters:
minimal-ui="true"
disables all SDK UISDK immediately calls
customProgressCallback
when loading startsIf your handlers aren't loaded yet → white screen
With handlers loaded first → instant professional splash screen
Your custom-handlers.js contains all the splash screen creation logic, so no additional HTML or CSS setup is required in your main page.
Step 5: Camera and Sensor Management
Why manual camera management? Better control over permissions and error handling.
const iOS = /iPad|iPhone|iPod/.test(navigator.userAgent);
let camstream = null;
let sdkloaded = false;
// Why check iOS separately? iOS requires user interaction before camera access
function handleCameraPermissions() {
if (iOS) {
showPermissionPrompt('camera');
} else {
startCameraStream();
}
}
function startCameraStream() {
navigator.mediaDevices.getUserMedia({
video: {
facingMode: 'environment',
advanced: [{ focusMode: 'manual', focusDistance: 0 }]
}
})
.then(function(stream) {
camstream = stream;
const videoElement = document.getElementById('video');
videoElement.srcObject = stream;
// Why wait for gyro? Tracking needs motion sensors
handleMotionPermissions();
})
.catch(function(error) {
console.error('Camera access failed:', error);
customErrorHandler({
code: 'ERR_CAMERA_PERMISSION_DENIED',
description: 'Camera access was denied',
instruction: 'Please refresh and allow camera access'
});
});
}
// Why motion permissions? AR tracking requires device orientation data
function handleMotionPermissions() {
if (typeof DeviceMotionEvent.requestPermission === 'function') {
DeviceMotionEvent.requestPermission().then(response => {
if (response === 'granted') {
initializeTracking();
} else {
customErrorHandler({
code: 'ERR_MOTION_PERMISSION_DENIED',
description: 'Motion sensor access was denied',
instruction: 'Motion sensors are required for AR tracking'
});
}
});
} else {
// Non-iOS devices
initializeTracking();
}
}
function initializeTracking() {
if (sdkloaded && camstream) {
WEBARSDK.SetCameraStream(camstream);
}
}
// SDK callback when ready
WEBARSDK.SetAppReadyCallback(() => {
sdkloaded = true;
initializeTracking();
});
// SDK callback when video starts
WEBARSDK.SetVideoStartedCallback(() => {
// Hide loading screen when AR is ready
const loadingScreen = document.getElementById('loadingscreen');
if (loadingScreen) {
loadingScreen.style.display = 'none';
}
// Remove original video element (SDK creates its own)
const videoElement = document.getElementById('video');
if (videoElement && videoElement.parentElement) {
videoElement.parentElement.removeChild(videoElement);
}
});
Step 6: Complete WebAR Scene
<!-- WebAR Scene -->
<div id="aframe-container">
<a-scene id="mainScene"
webar-scene="key: YOUR_LICENSE_KEY_HERE"
xr-mode-ui="enabled: false"
device-orientation-permission-ui="enabled: false"
loading-screen="enabled: false"
renderer="antialias: true; colorManagement: true; physicallyCorrectLights: true">
<a-camera webar-camera></a-camera>
<a-assets>
<a-asset-item id="astronaut" src="models/astronaut.glb"></a-asset-item>
</a-assets>
<!-- Why webar-stage? Represents the tracked surface -->
<a-entity webar-stage>
<!-- Why webar-ux-control? Enables user interaction with AR objects -->
<a-entity webar-ux-control="stageCursorUX: true; userGestureRotation: true; userGestureScale: true">
<a-entity gltf-model="#astronaut"
position="0 0 0"
rotation="0 0 0"
scale="0.25 0.25 0.25"
webar-loadmonitor="elType: glb"
class="clickable">
</a-entity>
</a-entity>
</a-entity>
</a-scene>
</div>
Advanced Features Explained
Real-time Progress Monitoring
The customProgressHandler
receives actual SDK loading data including:
progress.percentage
- Loading completion (0-100)progress.current
- Current loaded objectsprogress.total
- Total objects to load
External Camera Stream
Using external-camera-stream="true"
gives you full control over camera initialization, allowing for custom permission handling and better user experience.
Comprehensive Error Handling
The error handler provides detailed error information and user-friendly messages for different error scenarios.
Deployment Considerations
HTTPS Required: WebAR requires HTTPS for camera access
License Key: Replace YOUR_LICENSE_KEY_HERE with your actual license key
Asset Optimization: Optimize 3D models for web delivery
Browser Testing: Test across Safari (iOS) and Chrome (Android)
Full Source Code
The complete implementation is available in the WebAR SDK repository under /aframe/advanced-surface-tracking/.
Common Issues and Solutions
Progress handler not called?
Ensure minimal-ui="true" is set
Check that custom-handlers.js loads before the SDK
Verify function names match exactly
Error handler not working?
Confirm
on-error
attribute points to correct function nameCheck browser console for JavaScript errors
Ensure error UI elements exist in DOM
Camera permissions failing?
Verify HTTPS is enabled
Test iOS/Android permission prompt timing
Check camera constraints are supported
Key Benefits of This Advanced Implementation
Your advanced surface tracking experience will include:
Professional loading screen with real-time progress
Comprehensive error handling and user guidance
Smooth camera and sensor permission management
Enhanced debugging capabilities for development
Modular code architecture for easy maintenance
This advanced implementation provides a professional, production-ready foundation for WebAR applications with comprehensive UX control and error handling.
Last updated