Structured Light For 3d Scanning

I learned a little about structured light from the 2008 House of Cards Radiohead video and more since Kyle McDonald created an open source google code implementation- but only very recently really gotten into how it works.

Simulated Structured Light

I had trouble getting good scans initially and I thought it would be good to create a perfectly controlled setup that didn't require a projector or camera- instead generate scenes from software, projecting textures onto 3d objects.

One of the generated input phase images:


Output from the google code ThreePhase processing app:



The obvious improvement is to add in an obj loader to this program, instead of just generating a semi-random blobby shape.

Real World Scans

I was hoping for a lot more but managed to get a few scans of people during a New Year's Eve party:

From 2009.12.31_nextyear

From 2009.12.31_nextyear

From 2009.12.31_nextyear

Those were all made using modified versions of Kyle McDonald's slDecode and slCapture programs. I'll post those somewhere eventually.


Matlab Script

This script is currently very very slow, but it's good to be able to debug in matlab and have easy access to fft and other functions.

Wrapped phase:


Unwrapped phase:


Note the vertical lines where phase propagates vertically in glitchy ways- some filtering (and even slower processing time) ought to be able to clean that during the flood fill.

Next

I hope to go mainly in the direction of high resolution high fidelity scans, as opposed to high frame-rate low sample-time scans, though I think I can access several high frame rate cameras for use in the latter. There is also lots of room for improvements (especially at the phase unwrapping stage) that would benefit either one of those.

Bundler - the Photosynth core algorithms GPLed

[update- the output of bundler is less misaligned looking than this, I was incorrectly displaying the results here and in the video]

Bundler (http://phototour.cs.washington.edu/bundler) takes photographs and can create 3D point clouds and camera positions derived from them similar to what Photosynth does- this is called structure from motion. It's hard to believe this has been out as long as the publically available Photosynth but I haven't heard about it- it seems to be in stealth mode.


Bundler - GPLed Photosynth - Car from binarymillenium on Vimeo.

From that video it is apparent that highly textured flat surfaces do best. The car is reflective and dull grey and so generates few correspondences, but the hubcaps, license plate, parking strip lines, and grass and trees work well. I wonder if this could be combined with a space carving technique to get a better car out of it.

It's a lot rougher around the edges lacking the Microsoft Live Labs contribution, a few sets I've tried have crashed with messages like "RunBundler.sh: line 60: 2404 Segmentation fault (core dumped) $MATCHKEYS list_keys.txt matches.init.txt" or sometimes individual images throw it with "This application has requested the Runtime to terminate it..." but it appears to plow through (until it reaches that former error).

Images without good EXIF data trip it up, the other day I was trying to search flickr and find only images that have EXIF data and allow full view, but am not successful so far. Some strings supposed limit search results by focal length, which seems like would limit results only to EXIF, but that wasn't the case.

Bundler outputs ply files, which can be read in Meshlab with the modification that these two lines be added to ply header:

element face 0
property list uchar int vertex_index

Without this Meshlab will give an error about there being no faces, and give up.

Also I have some Processing software that is a little less user friendly but doesn't require the editing:

http://code.google.com/p/binarymillenium/source/browse/trunk/processing/bundler/

Bundler can't handle filenames with spaces right now, I think I can fix this myself without too much work, it's mostly a matter of making sure names are passed everywhere with quotes around them.

Multi-megapixel files load up sift significantly until it crashes after taking a couple of gigabytes of memory (and probably not able to get more from windows):

...
[Found in EXIF tags]
[CCD width = 5.720mm]
[Resolution = 3072 x 2304]
[Focal length (pixels) = 3114.965
[Found 18 good images]
[- Extracting keypoints -]

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.


Resizing them to 1600x1200 worked without crashing and took only a few hundred megabytes of memory per image, so more megapixels may work as well.

The most intriguing feature is the incremental option, I haven't tested it yet but it promises to be able to take new images and incorporate them into existing bundles. Unfortunately each new image has a matching time proportional to the number of previous images- maybe it would be possible to incrementally remove images also, or remove found points that are in regions that already have high point densities?