ARToolKitPlus

ARToolKitPlus 2.1.1

ARToolKit is a software library that can be used to calculate camera position and orientation relative to physical markers in real time. This enables the easy development of a wide range of Augmented Reality applications. ARToolKitPlus is an extended version of ARToolKit's vision code that adds features, but breaks compatibility due to a new class-based API.

PLEASE NOTE: ARToolKitPlus was developed internally as part of the Handheld AR project and released to public due to many requests. This version of ARToolKit is only targeted to experienced C++ programmers. It does not read camera images or render geometry of any kind (especially no VRML) nor is it an out-of-the box solution. In contrast to the original ARToolKit, this library is not meant for starters in developing AR software. No binary executables are provided and no help in setting up your IDE is given.

ARToolKitPlus' id detection code is inspired by ARTag. A big thanks goes to Mark Fiala for his great publication on this topic. Another big thanks goes to Thomas Pintaric for implementing many of the new features in version 2.0.

ARToolKitPlus is no longer updated since June 2006.

It has been succeeded by Studierstube Tracker.

Main features over ARToolKit

Class-based API (compile-time parametrization using templates)
Up to 4096 id-based markers (no speed penalty due to large number of markers)
New camera pixel-formats (RGB565, Gray)
Variable marker border width
Many speed-ups for low-end devices (fixed-point arithmetic, look-up tables, etc.)
New Pose estimation algorithm gives more stable tracking (less jitter).
Implementation of "Robust Planar Pose" by G. Schweighofer and A. Pinz
Loads MATLAB camera calibration toolbox files
Automatic thresholding (see description below)
Tools: pattern-file to TGA converter, pattern mirroring, off-line camera calibration, id-marker generator

Known Issues

RPP not optimized for fixed-point - this means it will be extremely slow on PDAs

Download

ARToolKitPlus is released under the GPL (as ARToolKit is). If you have any comments or request, please send them to the ARToolKit mailing list. ARToolKitPlus version 2.1.1 can be downloaded from here. If you have problems with the lastest version, then the previous latest stables 2.0.2 and 1.3.1 is still available.

News
2007-02-06: Finally an ARToolKitPlus Publication We finally published a paper about ARToolKitPlus at the Computer Vision Winter Workshop 2007. The paper outlines how ARToolKitPlus goes beyond ARToolKit and presents detailed benchmarks of ARToolKitPlus in different scenarios on various handheld devices ranging from low to high end.

2006-07-26: ARToolKitPlus on the Gizmondo Starting with version 2.1.1 we support ARToolKitPlus on the Gizmondo. The new download package includes a project file for VS2005 to build a lib that can be linked to your Gizmondo game or application. Click here to check out our AR Gizmondo demo!
2006-05-02: New fixed-point implementation We recently implemented a new new fixed-point implementation for ARToolKitPlus which is not only faster that the Intel GPP based version, but now allows running ARToolKitPlus at high performance on non XScale powered mobile devices. As a first test we ran ARToolKitPlus on a regular smartphone (i-mate SP5) where it runs at an overall performance (retrieving video, tracking, rendering) of ~12 frames per second. Click on the image on the right side for an enlarged picture. More information on this project can be found here. Due this new speedup, ARToolKitPlus can now track up to 200 images per second (5.0ms per image) on a regular PDA (Dell Axim X50v). These updates are available since version 2.1.

Platform Support

Project files for Embedded Visual C++ 4.0, Visual Studio 6.0, Visual Studio .NET 2003 and Visual Studio 2005 as well as makefiles for Linux are included.

Performance

ARToolKitPlus includes many performance improvements over the original ARToolKit.

The initial port of ARToolKit to the PDA (just resolving memory issues) ran at roughly 1-2 fps. Our optimizations concentrated primarily on improving performance on embedded devices such as PocketPC PDAs. Applying all our optimizations and using the Intel XScale compiler, we achieve 200 image analyses per second using the test camera image (in camera-native RGB565 format) shown below on a Dell Axim X30 (timed while performing 2000 times with this image). The same scene is analyzed 73 times per second at 640x480 on that PDA.

Image with one id-based marker (id 31)

List of changes over ARToolKit

Class-based API
We wrapped ARToolKit into a C++ class (called Tracker) and added some new (derived) classes for single (class TrackerSingleMarker) and multi-marker tracking (class TrackerMultiMarker). This eases development, improves memory management and allows using more than one ARToolKit instance in one process.
Low-level configuration via C++ Templates
ARToolKit can be highly configured using preprocessor commands (#define ...). This makes is inflexible when switching between these configurations frequently. To improve this, we switched several of these preprocessor definitions (marker-size, pixel-format) to template parameters.
Variable border width
Border width can be changed to use more area of the marker for pattern instead of the black border. This increases detection of very small markers.
Support for 8-bit greyscale Images
On embedded platforms such as PDAs or mobile phones memory access is very expensive. Therefore low bitrate images improve overall speed on these platforms.
Support for 16-bit RGB565 Images
On embedded platforms such as PDAs cameras return image data as RGB565. Native support (using a lookup table for greyscale conversion) reduces the need for image conversion.
Partial fixed-point implementation for PocketPC
For the PocketPC platform we reimplemented the most important functions with fixed-point which resulted in an enormous speedup on this platform.
Thomas Pintaric's pointer index optimization
This optimization gives ~10% speedup on PCs (less on PDAs) by storing a pointer index calculation in a temporary variable.
Lens undistortion lookup-table
We added two more modes for lens undistortion: no undistortion (only useful for undistorted camera images) and undistortion using a lookup-table (built on first access using the standard undistortion functions). This gives an enormous speedup on PDAs which are slow in doing floating-point operations as heavily performed by the undistortion algorithm. The LUT method introduces not inaccuracies.
Simple Id-encoded markers
We added the possibility to switch to id-encoded marker detection instead of the built-in template matching. This allows using up to 512 different markers without training and without speed penalty and gives a speedup even if just one marker is used.

Id-encoded marker detection is generally faster than using template (image) markers.
Id-based markers encode a 9-bit number into a 6x6 pattern (see image below). The 9 bits of the number are repeated four times to fill up the 36 bits that can be stored in this pixel array. To improve robustness (and allow marker numbers such as 0 and 511) all pixels are scrambled with an XOR mask. In order to use id-encoded markers the pattern size has to be set to 6x6, 12x12 or 18x18. In the first case each pixel is directly extracted. In the other two cases a subsampling is performed which should improve detection quality in most cases.
More information about on how id-based markers work can be found in the source code file src/core/arBitFieldPattern.cxx.
All markers can be downloaded as separate images from here.
A bitmap with all 512 id-encoded markers (border width: 0.250) can be found here.
Single-precision floating point support
Embedded platforms usually don't have hardware support for floating-point math. Therefore smaller (and simpler) numeric data results in speedups.
Support for special case when AR_PATT_SIZE_X and AR_PATT_SIZE_Y are same as AR_PATT_SAMPLE_NUM
For this case some divides and multiplies were removed in arGetPatt()
Improved pattern detection for half-res mode
Changed arGetPatt() to unproject the pattern always at full-res.
Automatic thresholding
ARToolKitPlus can do dynamic thresholding by looking at the marker content (pattern) and taking the average between the darkest and brightest pixels. If no marker is found the threshold value is randomized. This feature requires almost no additional processing power. Watch the video in the Media/Press section to see the dynamic thresholding in action. The video starts with normal light, then the light is turned down to an absolute minimum. The 2nd part of the video shows how ARToolKitPlus finds the correct threshold value although the marker was not visible during light change. The tracking stays stable all the time. Call activateAutoThreshold(true) to enable this feature.

ARToolKitPlus doing automatic thresholding
Installable custom memory de/allocator
On some platforms it is preferable to handle memory management manually. ARToolKitPlus provides a simple mechanism to take over the complete memory management by overloading a single class.
Vignetting (radial luminance falloff) compensation
Some cameras have a radial falloff in luminance in the image, as can be seen in the first image below which was taken with a Spectec SD camera. When applying a threshold value of 150 (thresholding is the first thing ARToolKit does...) the result looks like in the second image below. In this case markers near the corners will no longer be detected. After activating the radial luminance falloff compensation, the thresholded image looks like in the third image which allows tracking marker in the whole image.

Source camera image

Thresholded image without border compensation

Thresholded image with border compensation
BCH Id-encoded markers
Since version 2.0 ARToolKitPlus supports BCH id-encoded markers in addition to the "simple id-encoded markers" that were introduced in version 1.0. This new marker system is even more robust than the previous one, since it uses an advanced CRC algorithm to restore damaged marker images instead of a simple voting as the "simple-id" version does. Consequently we increased the number of available markers to 4096 when using BCH markers.
A bitmap with all 4096 BCH id-encoded markers (border width: 0.125) can be found here.
Improved camera calibration model (MATLAB camera calibration toolbox support)

As of version 2.0, ARToolKitPlus is compatible with the camera calibration
model used by Jean-Yves Bouguet's Camera Calibration Toolbox for MATLAB. An improved version of the toolbox (highly recommended) that includes automated corner/calibration object detection is available
from the Graphics Media Lab at Moscow State University.

The ASCII-file containing the camera calibration parameters should look as follows:
[line1]: ARToolKitPlus_CamCal_Rev02
[line2]: xsize ysize cc_x cc_y fc_x fc_y kc1 kc2 kc3 kc3 kc5 kc6 iter

xsize, ysize: calibrated frame dimensions (does not have to match the frame dimensions at runtime)
cc_x, cc_y: principal point location (in pixels)
fc_x, fc_y: focal length (in pixels)
kc1..kc6: radial/tangential distortion coefficients (kc6 currently not in use)
iter: number of iterations for distortion compensation

It is strongly recommended to enable undistortion lookup tables when using the improved calibration model.
A detailed description of the calibration parameters is available here.
Implementation of the "Robust Planar Pose" (RPP) algorithm
The RPP algorithm gives a more stable tracking (less jitter) than ARToolKit's pose estimation algorithm.

The robust pose estimator algorithm has been provided by G. Schweighofer and A. Pinz (Inst.of l.Measurement and Measurement Signal Processing, Graz University of Technology). Details about the algorithm are given in a Technical Report: TR-EMT-2005-01, available here. Thanks go to Thomas Pintaric for implementing the C++ version of this algorithm.