Softnami
Author: Hussain Mir Ali

I am interested in web and mobile technologies.

If you any questions or feedback then message me at devtips@softnami.com.

Announcements
Ads

Call predictor: AI based call prediction

Download

2020

Feb
Jan

2019

Dec
Nov
Jul
May
Mar
Jan

2018

Nov
Sep
Jul
Jun
Apr
Feb
Jan

2017

Dec
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Jan

2016

Dec
Oct
Sep
Aug
Jul

Camera Tracking with JavaScript

With a variety of I/O options today's smart devices have a lot of potential to leverage user interaction. The most common peripheral which is embedded in almost all smartphones, laptops and computers is the camera. Developers can use this to build better apps that enable better ways to interact with the webpage.

In this blog I have discussed the implementation of  visual object tracking using camera. I will be using the tracking.js library. The goal of this post is to show how visual object tracking can be implemented for JavaScript based applications. So I will use the WebGL app from my previous blog post to demonstrate this library.

Goal: Use visual tracking to change orientation of the JavaScript 3D object.


Step1:

Download tracking.js through this link https://trackingjs.com/ .  You can either use .zip file or install it using 'bower'. In this project I have used 'bower' to install the library.



Step2:

Add video and canvas elements in index.html file under the root directory. The video element will be responsible for showing the webcam video and the canvas element will show the object co-ordinates. Also include the tracking-min.js file from your earlier bower installation.

<html>
<body>
<script src="assets/three.min.js"></script>
<script src="assets/TrackballControls.js"></script>
<script src="scripts/camera.js"></script>
<script src="scripts/main.js"></script>


<script src="bower_components/tracking/build/tracking-min.js"></script>
<link rel="stylesheet" type="text/css" href="theme.css">
<link rel="stylesheet" type="text/css" href="styles/main.css">
<div class="demo-frame">
<div class="demo-container">
<video id="video" width="300" height="250" preload autoplay loop muted controls></video>
<canvas id="canvas" width="300" height="250"></canvas>
</div>
</div>

</body>
</html>


Add this to main.css.


 #video, #canvas {
margin-left: 100px;
margin-top: 35px;
position: absolute;
z-index: 200000;
}

Step3:


Create camera.js file under the scripts folder.


//global variable for storing x, y position of camera tracking.
track_position = {x: 0, y: 0};

window.onload = function() {
var video = document.getElementById('video');
var canvas = document.getElementById('canvas');
var context = canvas.getContext('2d');
      //Tracking yellow color in the video.
var tracker = new tracking.ColorTracker(['yellow']);
      tracking.track('#video', tracker, { camera: true });
tracker.on('track', function(event) {
context.clearRect(0, 0, canvas.width, canvas.height);
event.data.forEach(function(rect) {

          //Drawing rectangle for representation.
if (rect.color === 'custom') { rect.color = tracker.customColor; } context.strokeStyle = rect.color; context.strokeRect(rect.x, rect.y, rect.width, rect.height); context.font = '11px Helvetica'; context.fillStyle = "#fff"; context.fillText('x: ' + rect.x + 'px', rect.x + rect.width + 5, rect.y + 11); context.fillText('y: ' + rect.y + 'px', rect.x + rect.width + 5, rect.y + 22);
          //Update global position variables.
          track_position.x = rect.x;
track_position.y = rect.y;
});
});
};


This will help us track the position of the yellow object from the user's webcam. The track_position variable is global object that contains the x and y co-ordinates for the yellow object this is available throughout the app.

Step 4:

Update animateMotion() function in main.js file.


            function animateMotion (mesh3,mesh2,mesh4,refx2,refx3,refz3,refz4,refy4,refy2) {
var i =0;
setInterval(function(){
if(i===360)i=0;
mesh3.position.x = refx3*Math.cos(i);
mesh3.position.z = refz3*Math.sin(i);
mesh2.position.x = refx2*Math.cos(i);
mesh2.position.y = refy2*Math.sin(i);
mesh4.position.y = refy4*Math.sin(i);
mesh4.position.z = refz4*Math.cos(i);

//Updating 3D camera's x and y co-ordinates using
// global track_position object.
camera.position.x = track_position.x*10;
camera.position.y = track_position.y*10;
camera.position.z = 700;


i=i+0.04;
render();
},5);
}

The only change made to the original code is that now the 3D camera's x and y positions are being updated according object tracking position in real-time.

Step 5:

Run the app.


root@ubuntu:~# node app.js

Object translation relationship:


Whenever the yellow object in the webcam moves left it updates the global x value. This enables the atom to move left. Also when the yellow object moves down starting from (0,0) the atom moves down. The representation axis is horizontally flipped for the webcam canvas so when the yellow object is physically moving left the webpage shows it moving right.



   
The above image shows how co-ordinates of the yellow object are represented.



Working Demo: https://comicatom-track.herokuapp.com/