binarymillenium

Lunar DTM100 to Blender displacment map

2011-11-23T20:03:00.001-08:00

This post is an aggregration of multiple threads created on google plus and additional findings, the scattered nature of those threads made it impossible to find all the information in one place hence this post. It's also a work in progress with some blank spots- help is welcome!

(I've mostly moved to google plus which is great for mini-blogging (as opposed to the micro blogging of twitter and full size regular blogger blogging) and has very good engagement once you find good people to put in your circles. I expect greater plus/blogger integration in the future probably starting with comments becoming plusified.)

From LROC Lunar Map

Lunar Elevation Data

Source DTM files are in the IMG files:

http://lroc.sese.asu.edu/data/LRO-L-LROC-5-RDR-V1.0/LROLRC_2001/DATA/SDP/WAC_DTM/

The highest resolution maps are in the 100M.IMG files, which means 100 meters/pixel (which seems large for an object so close as the moon- why don't we have 1 meter per pixel, or 0.1 meter yet?).

I haven't written a script for handling the files that don't completely cover the lunar globe, and will probably use other tools to get that right. (grass gis http://grass.fbk.eu/gdp/index.php?)

TBD detail on file format.

Generating 32-bit elevation tifs with Python

http://code.google.com/p/binarymillenium/source/browse/trunk/python/lroc_to_vtx.py

Current 32-bit tifs:
https://docs.google.com/open?id=0BwP06t4a405kNDFhNjE3NTUtNTc2YS00NjFlLWExNTgtOTBkZjVmM2M0MGM2

https://docs.google.com/open?id=0BwP06t4a405kMzA0MGMzY2ItZGJmZC00NDI2LWE5NmYtYjM1ZTI5MmQ1OWZh

https://docs.google.com/open?id=0BwP06t4a405kNzM3ODIyZjktOWM3MC00NDJlLWI3ZGYtNGIwMWRmMWJjYTUz

Conversion to viewable

Not a lot of programs can display those 32-bit tifs correctly, and Blender didn't like them. Get a version of imagemagick with hdri enabled (./configure --enable-hdri when building it from source) so they can be converted to friendlier formats.

Making an easier to view jpg from the 32-bit tifs:

convert -define quantum:scale=255.0 -normalize moon.tif moon_fromtif.jpg

And the following is the result:

From LROC Lunar Map

The jpeg is usable in Blender but doesn't hold up well with a lot of zooming- the 255 levels of elevation possible in a jpeg produce stair step artifacts:

From LROC Lunar Map

Conversion to blender usable

So Blender could use a 16-bit format like openexr for 255x smoother gradations, use this conversion:

convert -define quantum:scale=255.0 moon.tif moon.exr

The quantum scale there seems like it ought to be 65535.0 but the 255.0 works, and imagemagick identify -verbose shows that the range of values is 65535.0.

From LROC Lunar Map

UV Sphere Projection Polar Problems

The UV spheres generated by blender have the problem of having triangles instead of quads around the poles. The spherical projection will produce distortion at the poles, smoothing the sphere prior to uv unwrap helps minimize it. I wonder if there is something problematic about making the pole polygons quads because it would involve multiple polygon points and edges right on top of each other.

You can see the problem areas in the uv image below- the nice quad projections become distorted triangles at the top and bottom.

The result is these pinched areas:

Lunar Visual Mosaics

The moon isn't perfectly grey, there are many interesting light and dark features. I haven't located a good texture generated from LROC or Clementine data (LROC would be ideal since it would probably guarantee all visual features line up with elevation features).

http://wms.lroc.asu.edu/lroc/wac_mosaic

http://ser.sese.asu.edu/MOON/clem_color.html

There are some random ones to be found on the web but I haven't tried them yet.

Future

Grass gis http://grass.fbk.eu/gdp

gdal - has python bindings (ubuntu intall python-gdal gdal-bin) http://www.gdal.org/
Turn python generated tiff images into geotiffs

osgearth - uses geotiff output from gdal to produce lod/paged terrain databases viewable in OpenSceneGraph osgviewer.

Use 100M data (the 256P IMG files). Parse dtm within python to do this? Minimum is extracting width x height.

Sources

Original discussions that originated this post:

http://feeds.laughingsquid.com/~r/laughingsquid/~3/mGUeRmGUfu0/

https://plus.google.com/u/0/107180534974005900062/posts/713nRXGEhn7

https://plus.google.com/u/0/116599331662269985445/posts/5GJw1T8Tu27

https://plus.google.com/u/0/103190342755104432973/posts/1krmFVvnh7d

Google Docs Storage

2011-01-30T15:49:00.000-08:00

I've been playing with google docs storage for about a month. The user interface is inferior to Amazon S3 + S3Fox in every way, but for 1/5th the cost I'm willing to put up with it (though it's only 1/5th the cost if I fill up all the storage for the given price tier, I think they expect most users not to use a large fraction). AVIs and JPEGs can be dragged-and-dropped in quantity, but NEF and .ini files have to use the 'select more pictures' dialog which can only handle 10-15 files at a time (otherwise a strange character appears instead of a list of all the files). Upload speeds seem good (a megabyte every 3 seconds or so), better than I remember S3 being last time I tried it.

There are some linux filesystem programs that can mount my entire google online storage (including blog posts), but only allow uploading of the google docs formats and not any file at all like can be done with the upload dialog. Hopefully this changes.

5/8/2011 update

Folder upload is now possible (no more shift selecting the contents of a folder and having to cut and paste folder names), and it looks like subfolders are uploaded properly. But sometimes a file fails to upload properly, and using the file method it was possible to see which file failed and re-queue for upload. Now after I've uploaded a folder with 90 items and it took 20 minutes, there is an error message saying that one file failed to upload, but no way to know which one. Retry fails repeatedly. Some reports of same, though I haven't seen the other bugs mentioned.

OpenGameArt

2010-12-14T20:46:00.000-08:00

You can't help but want to make a 2d overhead or isometric game after looking at some of the art available on OpenGameArt- I'll settle for a little procedural terrain generation:

http://www.openprocessing.org/visuals/?visualID=16704 (click run if it asks about running old java)

The OGA site could use more contributors, as well as better organization to promote the best work- and of course it could be popularized by making use of the content there. I think I'll use this and some variations at the next VJ event I do- I've used recorded video from games before, but even better to generate some game-like imagery live.

Another useful feature would be for terrain tilesets to have a standard connectivity definition file- an ascii file that says which tiles connect best to which neighbors in which direction. Current that's hardcoded into the processing sketch.

Gephex 0.4.3b built on 64-bit Ubuntu 10.10

2010-12-07T18:38:00.000-08:00

I haven't found a better graph-based VJ tool than Gephex to use in Linux or Windows (but haven't been looking much either), and it's not trivial to get it running on a modern Ubuntu system- but I've worked through all the compiler messages (with the big exception of leaving out ffmpeg) and have an archive of the results available for download.

Go here for details and a link to the download:

http://code.google.com/p/binarymillenium/wiki/Gephex64BitBuild

The reconfiguring may screw it up a little, otherwise let it install to my choice of directories and then move the bin and lib etc. files as appropriate.

I'm hoping there is a new and well-supported tool out there that can take the place of gephex- movid is not intended for VJ work but may be retrofittable.

First Steps in Bullet physics

2010-08-23T19:30:00.000-07:00

I've played with ODE for several small projects, but recently discovered Bullet physics while looking through ROS (Robot Operating System) documentation. The difference in rigid body simulation between the two is not that obvious, the Bullet rigid body demos seem very impressive and perhaps speedier than ODE. But the soft body physics have no equivalent in ODE, and seem very worthy of investigation- and equally intriguing are the decomposition capabilities, where objects can be shattered into smaller pieces with a function call (it may be more difficult than that, I haven't looked at the code yet).

There is a good forum, not a ton of documentation or second-party dispersed know-how in the form of tutorials especially for the soft body physic, but the demo code is great- and beyond that it comes down to experimentation.

Soft bodies from 3d files

A long time ago I made my own soft body physics simulation code where I would load an .obj file and replace all vertex edges with spring-dampers, and also create a grid of internal springs to give the body volume. It worked okay in the best conditions but would frequently explode in others. Now in bullet there is a toolchain to take a 3d object and turn it volumetric, and then simulate much more robustly than in my amateur effort (though explosions still can occur).

The bunny.inl and cube.inl files in Demos/SoftDemo provide the first clues, they have auto-generated comments like '# Generated by tetgen -YY bunny.smesh'.

Installation of tools

Tetgen is in the Ubuntu repositories, but tetview has to be downloaded as a binary. libg2c isn't in Ubuntu 10.04 lucid lynx, I had to download prebuilt version here http://www.fluvial.ch/d/libg2c.tgz , and also libstdc++.so.5 from http://packages.debian.org/lenny/i386/libstdc++5/download (extract with dpkg-deb file.deb .).

Wings3D is a good program for generating 3d meshes, and is in Ubuntu repositories.

The stl it exports in binary and doesn't work for tetgen ("wrong number of vertices"), use meshlab (also in ubuntu repos) to save it as an stl but uncheck binary. Objects with holes in them didn't work right, tetview kind of locks up on them. Concave areas don't seem quite right either.

I used 'tetgen -p file.stl' and it outputs file.1.ele, file.1.face, file.1.node, and file.1.smesh. 'tetview file.1' will view the output. In the bunny.inl file there is a getElements() and getNodes() function, the data there corresponds to the lists in file.1.ele and file.1.node. Cut and paste the lists into the functions, then use a text editor to put the quotes and line breaks in as seen in bunny.inl.

On trying to use the inl in SoftDemo I get this error

./AppSoftBodyDemo 
*** glibc detected *** ./AppSoftBodyDemo: malloc(): memory corruption: 0x090f5c38 ***
======= Backtrace: =========
/lib/tls/i686/cmov/libc.so.6(+0x6b591)[0x75f591]
/lib/tls/i686/cmov/libc.so.6(+0x6e395)[0x762395]
/lib/tls/i686/cmov/libc.so.6(__libc_malloc+0x5c)[0x763f9c]
/usr/lib/nvidia-current/libGL.so.1(+0x33e80)[0x2fae80]

Building the debug version helps out (cmake -DCMAKE_BUILD_TYPE=debug), but I haven't been able to figure it out yet.

Here is the backtrace:


Program received signal SIGABRT, Aborted.
0x0012d422 in __kernel_vsyscall ()
(gdb) bt
#0  0x0012d422 in __kernel_vsyscall ()
#1  0x003ff651 in raise () from /lib/tls/i686/cmov/libc.so.6
#2  0x00402a82 in abort () from /lib/tls/i686/cmov/libc.so.6
#3  0x0043649d in ?? () from /lib/tls/i686/cmov/libc.so.6
#4  0x00440591 in ?? () from /lib/tls/i686/cmov/libc.so.6
#5  0x00443395 in ?? () from /lib/tls/i686/cmov/libc.so.6
#6  0x00444f9c in malloc () from /lib/tls/i686/cmov/libc.so.6
#7  0x08157d51 in btAllocDefault (size=1195)
    at /home/bm/other/bullet-2.77/src/LinearMath/btAlignedAllocator.cpp:24
#8  0x08157e6f in btAlignedAllocInternal (size=1176, alignment=16)
    at /home/bm/other/bullet-2.77/src/LinearMath/btAlignedAllocator.cpp:170
#9  0x080b4803 in btCollisionObject::operator new (sizeInBytes=1176)
    at /home/bm/other/bullet-2.77/src/BulletCollision/CollisionDispatch/btCollisionObject.h:115
#10 0x080e9e85 in btSoftBodyHelpers::CreateFromTetGenData (worldInfo=..., 
    ele=0x81707e4 "48  4  0\n    1      26    28    16    12\n    2      28    21    16    12\n    3      25    19     2    20\n    4      28    27    10    14\n    5      17     3    20     1\n    6      20    24    18    19"..., 
    face=0x0, node=0x81707bb "# Generated by tetgen -YY test.1.smesh \n", 
    bfacelinks=false, btetralinks=true, bfacesfromtetras=true)
    at /home/bm/other/bullet-2.77/src/BulletSoftBody/btSoftBodyHelpers.cpp:957
#11 0x080b0bc5 in Init_TetraCube (pdemo=0x8309c60)
---Type  to continue, or q  to quit---
    at /home/bm/other/bullet-2.77/Demos/SoftDemo/SoftDemo.cpp:1315
#12 0x080b122c in SoftDemo::clientResetScene (this=0x8309c60)
    at /home/bm/other/bullet-2.77/Demos/SoftDemo/SoftDemo.cpp:1451
#13 0x080b383f in SoftDemo::initPhysics (this=0x8309c60)
    at /home/bm/other/bullet-2.77/Demos/SoftDemo/SoftDemo.cpp:1849
#14 0x080a5b90 in main (argc=1, argv=0xbffff384)
    at /home/bm/other/bullet-2.77/Demos/SoftDemo/main.cpp:28

I put the question to the forum here but no responses- not many people are that adept at the soft body physics yet.

Something functional

But I've been able to create a trivial example from scratch that works:


static const char* getNodes() { return(
"8  3  0  0\n"
"   0    1   1   1\n"
"   1    1   1  -1\n"
"   2    1  -1  -1\n"
"   3    1  -1   1\n"
"   4   -1  -1   1\n"
"   5   -1   1   1\n"
"   6   -1   1  -1\n"
"   7   -1  -1  -1\n"
"#  \n"); }

static const char* getElements() { return(
"4  4  0\n"
"    0      0    1    3    5\n"
"    1      4    7    5    3\n"
"    2      6    5    1    7\n"
"    3      2    1    7    3\n"
"#  \n"); }

Maybe I should try to corrupt the above into provoking the same kind of crash?

Update- Solution Found

tetgen by default is creating 1-based vertex indices, while Bullet is expecting 0-based indices- the -z flag will make tetgen output zero-based. So no more crashing.

Bullet Soft Body Physics from binarymillenium on Vimeo.

Bundler on 64-bit Ubuntu 9.10

2010-04-08T07:45:00.000-07:00

It builds fine, but seg faults when run:


...
[KeyMatchFull] Matching to image 312
[KeyMatchFull] Matching took 71.610s
[KeyMatchFull] Matching to image 313
[KeyMatchFull] Matching took 69.720s
mkdir: cannot create directory `bundle': File exists
[- Running Bundler -]
/home/binarymillenium/bundler-v0.3-source/RunBundler.sh: line 93: 22589 Segmentation fault      $BUNDLER list.txt --options_file options.txt › bundle/out
[- Done -]


Program received signal SIGSEGV, Segmentation fault.
0x0000000000522650 in dscal_ ()
(gdb) bt
#0  0x0000000000522650 in dscal_ ()
#1  0x0000000000519d16 in dsytf2_ ()
#2  0x00000000004e9644 in dsytrf_ ()
#3  0x00000000004a6d99 in sba_symat_invert_BK ()
#4  0x00000000004a05ec in sba_motstr_levmar_x ()
#5  0x000000000049d058 in sba_motstr_levmar ()
#6  0x000000000049ae8e in run_sfm ()
#7  0x0000000000415e06 in BundlerApp::RunSFM(int, int, int, bool, camera_params_t*, v3_t*, int*, v3_t*, std::vector‹std::vector‹std::pair‹int, int›, std::allocator‹std::pair‹int, int› › ›, std::allocator‹std::vector‹std::pair‹int, int›, std::allocator‹std::pair‹int, int› › › › ›&, double, double*, double*, double*, double*, bool) ()
#8  0x000000000042a16e in BundlerApp::BundleAdjustFast() ()
#9  0x0000000000405c33 in BundlerApp::OnInit() ()
#10 0x0000000000406f77 in main ()

I'll update this post with results of trying out the 32-bit binaries, and also trying to build it on 64-bit with 32-bit compilation.

There needs to be better buzz blogger integration

2010-02-14T09:01:00.000-08:00

The generic rss import gadget sort of works, but the rss headlines of buzz are really useless, only saying that there is a new buzz from username- they should have the first sentence at least of a buzz item:

Xbox Project Natal - Estimated Specs

2010-01-08T18:25:00.001-08:00

This video went up yesterday, was pulled down and then restored later for some reason (not before other youtube users posted their own copies and gathered many more views than the official channel)

The video states 30Hz, and there is an interesting shot starting at two minutes.

From the above screenshots it's possible to count the number of pixels horizontally and vertically, and also most of the depth bins (there may be more range beyond the wall behind the person but a minimum count is better than nothing).

So there at least 18 depth bins, counted at 2:01-2:04 in the video. Each is maybe 2 or3 inches spaced.

The resolution looks like 64x64.

Structured Light For 3d Scanning

2010-01-04T06:17:00.000-08:00

I learned a little about structured light from the 2008 House of Cards Radiohead video and more since Kyle McDonald created an open source google code implementation- but only very recently really gotten into how it works.

Simulated Structured Light

I had trouble getting good scans initially and I thought it would be good to create a perfectly controlled setup that didn't require a projector or camera- instead generate scenes from software, projecting textures onto 3d objects.

One of the generated input phase images:

Output from the google code ThreePhase processing app:

The obvious improvement is to add in an obj loader to this program, instead of just generating a semi-random blobby shape.

Real World Scans

I was hoping for a lot more but managed to get a few scans of people during a New Year's Eve party:

From 2009.12.31_nextyear

Those were all made using modified versions of Kyle McDonald's slDecode and slCapture programs. I'll post those somewhere eventually.

Matlab Script

This script is currently very very slow, but it's good to be able to debug in matlab and have easy access to fft and other functions.

Wrapped phase:

Unwrapped phase:

Note the vertical lines where phase propagates vertically in glitchy ways- some filtering (and even slower processing time) ought to be able to clean that during the flood fill.

Next

I hope to go mainly in the direction of high resolution high fidelity scans, as opposed to high frame-rate low sample-time scans, though I think I can access several high frame rate cameras for use in the latter. There is also lots of room for improvements (especially at the phase unwrapping stage) that would benefit either one of those.

Natal competitor: Optricam?

2009-11-21T13:13:00.000-08:00

For a while I was worried that after Microsoft acquired 3DV Systems and their depth sensing ZCam, which might have been available at the beginning of this year, they were going to release the Natal exclusively for the Xbox and the possibilities for non-MS sanctioned applications would be severely hampered. And then to have to suffer delays for software I don't care about when the hardware may be ready now. But there may be similar products from other vendors due to arrive soon.

I learned about the PMDTec Camcube a few months ago. It is a lot less expensive than a Swissranger, but there are no U.S. distributors and nothing on the web from customers that are using it.

But this last week there was this press-release about a Belgian company Optrima (Israel, Germany, Switzerland, and now Belgium- all the U.S. flash lidar vendors seemed to have missed the boat on low cost continuous-wave IR led range finding systems to instead focus on extremely expensive aerospace and high end mobile robotics applications) teaming with TI and a body motion capture software vendor (Softkinetic) to produce Natal/Zcam-like results in the same application area- games and general computer UI. There is mention of Beagleboard support in the press-release, so having say Ubuntu be able to communicate is very likely.

Hopefully TI will really get behind it and all the economies of scale that MS can bring to bear can be matched, and it will only cost around $100.

Also, I'm seeing more and more red clothing and jumpsuits- possibly with specially IR reflective component, which makes me suspicious about the limitations of these sensors (also using them with windows letting in direct sunlight is probably out of the question).

Using a canon camera to view jpegs

2009-11-02T20:40:00.000-08:00

I tried using my Canon camera as a picture viewer, putting some downloaded jpegs on it- the pictures didn't show up at all when I tried to review them, only the pictures I had taken with the camera. Renaming the pictures to have camera names like DSC_0025.jpg made the camera show a question mark icon for the picture at least.

A little searching later I discovered a tool called paint.net, which saves jpegs in the proper format so the Canon will like them. The other trick is to re-size the canvas the picture is on so all dimensions are multiples of 8.

paint.net itself is somewhat interesting, quicker to load than Gimp but not so much easier or more intuitive or like Deluxe Paint that I'll use it for anything else.

Instructions for rendering with Processing on Amazon EC2

2009-09-12T18:54:00.000-07:00

There are detailed instructions elsewhere on how to get started with EC2 in general, here are the high level things to do for my headless rendering project:

Get a unix command line environment that has python and ssh, I use cygwin under Windows, other times I dual boot into Ubuntu.

Get an Amazon EC2 account, create a ~/username.pem file, and make environmental variables for the keys (follow boto instructions).
Make sure pem permission are set to 700.

Edit ssh_config so that StrictHostChecking is set to no, otherwise ssh sessions started by the scripts will ask if it's okay to connect to every created instance- I could probably automate that response though.

Make sure there are no carriage returns (\r) in the pem file in Linux.

Get Elasticfox, put your credentials in.

Get boto

Get trajectorset

Create a security group called http that at least allows your ip to access a webserver of an ec2 instance that uses it.

At this point it should be possible to run ec2start.py, visit the ip address of the head node and watch the results come in. The ec2start script launches a few instances, one head node that will create noise seeds to send to the worker nodes via sqs, and then wait for the workers to process the seeds and send sqs messages back. The head node then copies the results files and renders the graphics, copying the latest results to folder that can be seen by index.html for web display.

My code is mainly for demonstration, so the key things I did that will help with alternate applications follow:

Custom AMI

You can use the AMI I created with the id 'ami-2bfd1d42', I used one of the Alestic Ubuntu amis and added Java, Xvfb, Boto, and a webserver like lighttpd (I forget if Xvfb was already installed or not).

Headless rendering

The EC2 instance lack graphics contexts at first, and trying to run a graphical application like an exported Processing project will not work (TBD did I ever try that?). Xvfb creates a virtual frame buffer that Processing can render to after running these commands:

Xvfb :2
export DISPLAY=:2

Launching processes and detaching from them

I use python subprocess.Popen frequently to execute commands on the instances like this:


cmd = "Xvfb :2" 
whole_cmd = "ssh -i ~/lucasw.pem root@" + dns_name + " \"" + cmd + "\""   
proc = subprocess.Popen(whole_cmd, shell=True, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
(stdout,stderr) = proc.communicate()

The problem is when one wants to run something and close the connection, and leave it running - like Xvfb above, it needs to run and stay running. One method is to leave the ssh connection open, but there is a limit of about 20 ssh sessions.

The trick is to use nohup:

cmd = "nohup Xvfb :2"

Don't put extra quotes around the command to execute, which brings me to the next topic.

Quote escaping

There are a few bash commands that require parts to be in quotes- but in python the bash command is already is in quotes, and python will not understand the inner set of quotes unless they are escaped with the backslash:
cmd = "echo \"blah\" > temp.txt";

Then at other times an additional level of quote escaping is required:
cmd = "echo \\\"blah\\\" > temp.txt";
(I do this when I pass all of the cmd variable to be executed by ssh, and ssh wants it in quotes)

One backslash escapes on level of quoting, three escapes two levels? It's because the escaping backslash itself needs to be escaped. This gets confusing fast, and some experimentation with python in interactive mode is required to get it right.

Config file driven

It's not currently, not as much as at it needs to be, which makes it very brittle- to change plots requires making about three different edits, when a source config file should specify it for all.

Computing Cloud Rendering with Processing and Amazon EC2

2009-08-27T22:26:00.000-07:00

This project is my first experiment with using Amazon EC2 for cloud rendering. The source code is all there and I'll post detailed instructions on how to use it later, but here is a speeded up video of the output:

Computing Cloud Rendering from binarymillenium on Vimeo.

It looks kind of cool, but not too exciting- but there's potential for better things.

What I've done is launched several compute instances on EC2, where worker nodes create individual lines seen in the plots, and then pass data back to a head node, which creates the plots, puts them on a web page for real-time feedback, and stores all the frames for retrieval at the end of the run.

The plots are aggregations of all the results, blue is the presence of any line, and white is a high density of lines, and greenish tinge signifies the line was from a recently aggregated set. It's interesting because the more lines are aggregated, the less the plot changes, so it becomes increasingly boring.

All the plotting and data generation is done using java applications exported from Processing. 3D graphics are also possible, and something like this earlier video could be ported to the scripts I've made. There is no graphics card accessible on the EC2 machines, but virtual frame buffer software like Xvfb and software rendering (either Processing's P3D or software opengl) make it possible to trick the application into thinking there is.

It's not distributed rendering since all the rendering is on one computer, but I think I need to distribute the rendering in order to speed it up.

There is potential for more dynamic applications, involving user interaction through webpages, or simulations that interact with the results of previous simulations, and communicate with other nodes to alter what they are doing.

Save Image As And Close Tab Firefox Addon

2009-08-23T17:30:00.000-07:00

I haven't made a firefox addon before, but I thought I'd try something simple: combine the context menu "Save Image As..." with closing the current tab. My contribution consists of putting these two lines together:


gContextMenu.saveImage();
gBrowser.removeCurrentTab();

To start out with I used the Firefox/Thunderbird Extension Wizard. Initially I didn't select the 'Create context menu item' and that may have caused problems with gContextMenu not being defined - it was either that or the fact I was trying to embed the commands into the firefoxOverlay.xul file as embedded javascript instead of putting it in the overlay.js file.

I found the first function by looking through the firefox source code first for the menuitem name of the function "Save Image As", and from there finding saveImage. The removeCurrentTab function was harder to find, but this addon provided source code that showed it: Stephen Clavering's CTC.

Tutorial pages I initially found about extension development were helpful, but I didn't see anything that talks about mozilla fundamentals- probably I need to find a book about it.

This addon goes well with the Menu Editor and Download Sort.

There is code in the real Save Image As for determining whether an image is being selected or not (my addon shows up regardless) I should add in next, and there should be logic that prevents the close action if the save as was canceled (less sure how to do that).

Quick jmatio in Processing example

2009-08-05T09:32:00.000-07:00

1. Download jmatio from mathworks file exchange
2. unzip and put contents in folder called jmatio
3. rename lib dir to library
3. rename library/jamtio.jar to library/jmatio.jar
4. create a mat file in the sketch data dir called veh_x.mat which contains an array called veh_x
5. Run the following code:


import com.jmatio.io.*;
import com.jmatio.types.*;

  MatFileReader mfr = null;
  try {
    mfr = new MatFileReader(sketchPath + "/data/veh_x.mat" );
  } catch (IOException e) {
   e.printStackTrace();
   exit(); 
  }
  
if (mfr != null) {
  double[][] data = ((MLDouble)mfr.getMLArray( "veh_x" )).getArray();
  
  println(data.length +" " + data[0].length + " "  + data[0][0]);
  
}

TBD use getContents instead of requiring the mat file name and array name be the same.

Building Bundler v0.3 on Ubuntu

2009-07-03T14:59:00.000-07:00

'The Office Box' requested help with running Bundler on linux ( specifically Ubuntu 9- I have a virtualbox vm of Ubuntu 8.04 fully updated to today, but I'll try this out on Ubuntu 9 soon) so I went through the process myself.

The binary version depends on libgfortran.so.3, which I couldn't find with aptitude, so I tried the building from source- it turned out to be not that hard. The is no 'configure' for bundler 0.3 to search for dependencies that aren't installed, so I built incrementally and installed packages as I ran into build failures. I might be missing a few I already had installed for other purposes, but do a sudo aptitude install on the following:


build-essentials
gfortran-4.2  
zlib1g-dev
libjpeg-dev

A missing gfortran produces the cryptic 'error trying to exec 'f951': execvp: No such file or directory)' message.

These might be necessary I'm not sure:


lapack3
libminpack1
f2c

After that run the provided makefile, add the bundler bin folder to your LD_LIBRARY_PATH, and then go into the examples/kermit directory and run ../../RunBundler.sh to see there are good ply files in the bundle directory. Bundler is a lot slower than Photosynth for big jobs, I haven't tried the intel math libs though.

The full output from a successful kermit RunBundler run looks like this:


Using directory '.'
0
Image list is list_tmp.txt
[Extracting exif tags from image ./kermit000.jpg]
  [Focal length = 5.400mm]
[Couldn't find CCD width for camera Canon Canon PowerShot A10]
[Found in EXIF tags]
  [CCD width = 5.230mm]
  [Resolution = 640 x 480]
  [Focal length (pixels) = 660.803
[Extracting exif tags from image ./kermit001.jpg]
  [Focal length = 5.400mm]
[Couldn't find CCD width for camera Canon Canon PowerShot A10]
[Found in EXIF tags]
  [CCD width = 5.230mm]
  [Resolution = 640 x 480]
  [Focal length (pixels) = 660.803
[Extracting exif tags from image ./kermit002.jpg]
  [Focal length = 5.400mm]
[Couldn't find CCD width for camera Canon Canon PowerShot A10]
[Found in EXIF tags]
  [CCD width = 5.230mm]
  [Resolution = 640 x 480]
  [Focal length (pixels) = 660.803
[Extracting exif tags from image ./kermit003.jpg]
  [Focal length = 5.400mm]
[Couldn't find CCD width for camera Canon Canon PowerShot A10]
[Found in EXIF tags]
  [CCD width = 5.230mm]
  [Resolution = 640 x 480]
  [Focal length (pixels) = 660.803
[Extracting exif tags from image ./kermit004.jpg]
  [Focal length = 5.400mm]
[Couldn't find CCD width for camera Canon Canon PowerShot A10]
[Found in EXIF tags]
  [CCD width = 5.230mm]
  [Resolution = 640 x 480]
  [Focal length (pixels) = 660.803
[Extracting exif tags from image ./kermit005.jpg]
  [Focal length = 5.400mm]
[Couldn't find CCD width for camera Canon Canon PowerShot A10]
[Found in EXIF tags]
  [CCD width = 5.230mm]
  [Resolution = 640 x 480]
  [Focal length (pixels) = 660.803
[Extracting exif tags from image ./kermit006.jpg]
  [Focal length = 5.400mm]
[Couldn't find CCD width for camera Canon Canon PowerShot A10]
[Found in EXIF tags]
  [CCD width = 5.230mm]
  [Resolution = 640 x 480]
  [Focal length (pixels) = 660.803
[Extracting exif tags from image ./kermit007.jpg]
  [Focal length = 5.400mm]
[Couldn't find CCD width for camera Canon Canon PowerShot A10]
[Found in EXIF tags]
  [CCD width = 5.230mm]
  [Resolution = 640 x 480]
  [Focal length (pixels) = 660.803
[Extracting exif tags from image ./kermit008.jpg]
  [Focal length = 5.400mm]
[Couldn't find CCD width for camera Canon Canon PowerShot A10]
[Found in EXIF tags]
  [CCD width = 5.230mm]
  [Resolution = 640 x 480]
  [Focal length (pixels) = 660.803
[Extracting exif tags from image ./kermit009.jpg]
  [Focal length = 5.400mm]
[Couldn't find CCD width for camera Canon Canon PowerShot A10]
[Found in EXIF tags]
  [CCD width = 5.230mm]
  [Resolution = 640 x 480]
  [Focal length (pixels) = 660.803
[Extracting exif tags from image ./kermit010.jpg]
  [Focal length = 5.400mm]
[Couldn't find CCD width for camera Canon Canon PowerShot A10]
[Found in EXIF tags]
  [CCD width = 5.230mm]
  [Resolution = 640 x 480]
  [Focal length (pixels) = 660.803
[Found 11 good images]
[- Extracting keypoints -]
Finding keypoints...
1245 keypoints found.
Finding keypoints...
1305 keypoints found.
Finding keypoints...
1235 keypoints found.
Finding keypoints...
1220 keypoints found.
Finding keypoints...
1104 keypoints found.
Finding keypoints...
1159 keypoints found.
Finding keypoints...
949 keypoints found.
Finding keypoints...
1108 keypoints found.
Finding keypoints...
1273 keypoints found.
Finding keypoints...
1160 keypoints found.
Finding keypoints...
1122 keypoints found.
[- Matching keypoints (this can take a while) -]
../../bin/KeyMatchFull list_keys.txt matches.init.txt
[KeyMatchFull] Reading keys took 1.020s
[KeyMatchFull] Matching to image 0
[KeyMatchFull] Matching took 0.010s
[KeyMatchFull] Matching to image 1
[KeyMatchFull] Matching took 0.170s
[KeyMatchFull] Matching to image 2
[KeyMatchFull] Matching took 0.380s
[KeyMatchFull] Matching to image 3
[KeyMatchFull] Matching took 0.560s
[KeyMatchFull] Matching to image 4
[KeyMatchFull] Matching took 0.740s
[KeyMatchFull] Matching to image 5
[KeyMatchFull] Matching took 0.960s
[KeyMatchFull] Matching to image 6
[KeyMatchFull] Matching took 1.060s
[KeyMatchFull] Matching to image 7
[KeyMatchFull] Matching took 1.210s
[KeyMatchFull] Matching to image 8
[KeyMatchFull] Matching took 1.410s
[KeyMatchFull] Matching to image 9
[KeyMatchFull] Matching took 1.600s
[KeyMatchFull] Matching to image 10
[KeyMatchFull] Matching took 1.760s
[- Running Bundler -]
[- Done -]

Xbox Project Natal

2009-06-16T05:08:00.000-07:00

A little less than a year ago I remember stumbling across the Zcam from 3DV Systems, the company promised two orders of magnitude decreases in the cost of flash array lidar through mass production- the trick is to market it as a device anyone can use, not just as a robotics or general automation tool. The company promised to be in the market by the end of 2008, and after emails went unanswered I assumed it was vaporware.

The closest competitor would be the Mesa Imaging SwissRanger, which I think goes for $5000-$10000. Beyond that there are very expensive products from Advanced Scientific Concepts or Ball Aerospace that are in the hundreds of thousands of dollars range at least. ASC made a deal with iRobot that might bring the price down through economies of scale, though they probably aren't going to put it on the Roomba anytime soon. More likely the Packbot which already costs $170K, why not round that up to half a million?

In late 2008 to early 2009 rumors surfaced that Microsoft was going to buy 3DV Systems, and now we have the official announcements about Natal. And of course no mention of 3DV Systems (which hasn't updated their webpage in over a year) or even how it measures the phase shift or time of flight of light pulses in a sensor array to produce depth images. Given enough processing power, the right software, and good lighting, it would be possible to do everything seen in the Natal videos with a single camera. The next step up would be stereo vision to get depth images- it's possible that's what Natal is, but it seems like they would have mentioned that since that technology is so conventional.

But that won't stop me from speculating:

Natal is probably a 0.5-2 megapixel webcam combined with a flash lidar with a resolution of 64x64 or 128x128 pixels, and maybe a few dozen levels of depth bins.

The low resolution means there is a ton of software operating on the video image and the depth information to derive the skeletal structure for full body motion capture. All that processing means the speed and precision is going to be somewhat low- it would be great to buy one of these and be able to record body movements for use in 3D animation software, machinima, independent games, or full body 3D chat (there's no easy way to do intersections or collisions with other people in an intuitive way so don't get too excited), but I doubt it will capture a lot of nuance.

The lidar might be continuous wave (CW) like the SwissRanger. This has an interesting property where beyond the maximum range of the sensor, objects appear closer again- if the range was 10 feet, an object 12 feet away is indistinguishable from one 2 feet away, or 22 feet away.

Beyond that, hopefully MS sees the potential for this beyond an Xbox peripheral. It would be criminal not to be able to plug this into a PC, and have at least Windows drivers, an SDK + DirectX support. The next most obvious thing would be to use it to promote MS Robotics Studio, and offer a module for that software to use the Natal. If it just has a USB connection then it could be placed on a moderately small mobile robot, and software could use the depth maps for collision avoidance and with some processing power be able to computer 3D or 2D grid maps (maybe like this) and figure out when it has returned to the same location.

The next step is to make a portable camera that takes a high megapixel normal image along with a depth image. Even with the low resolution and limited range (or range that rolls over), the depth information could be passed on to photosynth to reduce the amount of pictures needed to make a good synth. MS doesn't make cameras, but why not license the technology to Nikon or Canon? Once in dedicated cameras, it's on to cell phone integration...

The one downside is that the worst application seems to be as a gaming device, which is bad because I'd like it to be very successful in order to inspire competing products and later generations of the same technology. It is certainly not going to have the precision of a Wii MotionPlus, and maybe not even a standard Wii controller (granted that it can do some interesting things that a Wii controller can't).

But even if it isn't a huge success, it should be possible to get a device from the first generation, and it's only a matter of time before someone hacks it and produces Linux drivers, right?

Weaponizers

2009-05-02T12:39:00.000-07:00

Rusty of Seattle's HazardFactory, known for power tool drag racing and other events usually involving conflagrations of some sort, is now turning cars into remote control tools of destruction:

Some choice quotes on Bre's blog.

Discovery channel listing

The RC is probably great fun and well suited for television but I'd like to see (or better yet work on) an autonomous version of this, or incorporate more autonomous elements into it, though the cars would probably be an order or three of magnitude more expensive. Which is why you should tune in and make sure the ratings are high so they can afford to destroy $5K lidars or gimbal mounts in future iterations...

OpenCV example, and why does Google do so poorly?

2009-04-06T06:05:00.000-07:00

Take searching for cvGetSpatialMoment:
http://www.google.com/search?hl=en&q=cvGetSpatialMoment&btnG=Google+Search&aq=f&oq=

All the top results are nearly useless, just code that doesn't help much if you don't know what cvGetSpatialMoment does.

The "CV Reference Manual" that comes with an install of OpenCV probably should come up first (the local html files of course aren't google searchable), or any real text explanation or tutorial of the function. So scrolling down further there are some odd but useful sites like http://www.ieeta.pt/~jmadeira/OpenCV/OpenCVdocs/ref/opencvref_cv.htm. I guess the official Willow Garage docs here haven't been linked to enough.

The official OpenCV book on Google is highly searchable, some pages are restricted but many are not.

Through all that frustration I did manage to learn a lot of basics to load an image and process a portion of the image to look for a certain color, and then find the center of the region that has that color.

IplImage* image = cvLoadImage( base_filename, CV_LOAD_IMAGE_COLOR );

split it into two halves for separate processing

IplImage* image_left = cvCreateImage( cvSize( image->width/2, image->height), IPL_DEPTH_8U, 3 ); cvSetImageROI( image, cvRect( 0, 0, image->width/2, image->height ) ); cvCopy( image, image_left );

convert it to hsv color space

IplImage* image_left_hsv = cvCreateImage( cvSize(image_left->width, image_left->height), IPL_DEPTH_8U, 3 ); cvCvtColor(image_left,image_left_hsv,CV_BGR2HSV);

get only the hue component using the COI '[color] Channel Of Interest' function

IplImage* image_left_hue = cvCreateImage( cvSize(image_left->width, image_left->height), IPL_DEPTH_8U, 1 ); cvSetImageCOI( image_left_hsv, 1); cvCopy(image_left_hsv, image_left_hue);

find only the parts of an image within a certain hue range

cvInRangeS(image_left_hue, cvScalarAll(huemin), cvScalarAll(huemax), image_msk);

erode it down to get rid of noise

cvErode(image_msk,image_msk,NULL, 3);

and then find the centers of mass of the found regions

CvMoments moments; cvMoments(image_msk, &moments, 1); double m00, m10, m01; m00 = cvGetSpatialMoment(&moments, 0,0); m10 = cvGetSpatialMoment(&moments, 1,0); m01 = cvGetSpatialMoment(&moments, 0,1); // TBD check that m00 != 0 float center_x = m10/m00; float center_y = m01/m00;

Copy the single channel mask back into a three channel rgb image

IplImage* image_rgb = cvCreateImage( cvSize(image_msk->width, image_msk->height), IPL_DEPTH_8U, 3 ); cvSetImageCOI( image_rgb, 2); cvCopy(image_msk,image_rgb); cvSetImageCOI( image_rgb, 0);

and draw circles on a temp image where the centers of mass are

cvCircle(image_rgb,cvPoint(int(center_x),int(center_y)), 10, CV_RGB(200,50,50),3);

All the work of setting channels of interest and regions of interest was new to me. I could have operated on images in place rather than creating many new ones, taking up more memory (and I would need to remember to free the memory created by all of them), but for debugging it's nice to keep around the intermediate steps.

mewantee example

2009-03-29T09:42:00.001-07:00

I've made enough fixes to mewantee to open it open and allow most of it to be viewed without logging in, and creating a user no longer requires activation.

There isn't much on there right now, but I have a good example: There's a project called crossephex I was working on a few months ago, and I'll probably start on it again soon. It's supposed to be a vj/visuals generating tool for processing similar to gephex. I need a bunch of basic graphics to use as primitives to mix with each other to create interesting effects, so on mewantee I have this request, which asks for help from other people generating those graphics. Each one shouldn't take more than a few minutes to make, of course I could do it myself but I think it's a good example of what the site might be good for.

mewantee!

2009-03-25T19:02:00.000-07:00

I created a website called mewantee using google appengine. It's closed to the public right now, but I need some users to try it out and tell me if they run into any problems using it normally, or any feedback at all. If you login with a gmail account (google handles the login, I won't know anything except your email address, and even that will be hidden from other users), I'll be sent a notification email and I can then activate your account.

What is it about? Mainly I'd like it to incentivize the creation of creative commons and open source content and it uses a sort of economic model to do it. Even if it is too strange or the kind of users needed to make it work don't show up, it was a good exercise to learn python and appengine.

Something else to figure out- I have mewantee.com pointing to mewanteee.appspot.com, is there any way to make it stay mewantee.com to everyone else like they way this blog is really on blogspot.com but is seen as binarymillenium.com.

Gephex 0.4.3 updated for Ubuntu 8.10

2009-02-21T13:35:00.000-08:00

Since there hasn't been a better version of Gephex since 0.4.3 (though I haven't tried compiling the repository recently, last time was not successful), I've downloaded the source and hacked it until it built on Ubuntu 8.10 updated to today:

http://binarymillenium.googlecode.com/files/gephex-0.4.3updated.tgz

I haven't tested it all the way, especially the video input modules, but it probably works.

Most of the changes have to do with updates to gcc, where it treats classname::method in cpp files as errors, and some files needed to include stdlib.h or string.h that didn't before. Also some structure definition in libavcodec had to be messed with- the static declaration removed.

nasm, qt3 in the form libqt3-headers, and libxv-dev had to be installed (and other non-standard things for 8.10 that I already had installed for other purposes). For qt3, flags for the include, bin, and lib dir needed to be passed to configure.

I had to run configure in the ffmpeg library and disable mmx with the --disable-mmx flag, putting that flag in the top-level makefile didn't work. My configuration specific makefiles are in the tarball so you would definitely have to rerun configure to override them.

Next I'll be creating a new custom gephex module for my ARToolkit multimarker UI project.

----

Update

I've tested this build more extensively, and have discovered that the Ubuntu visual effects that are on by default cause the gephex output window to flicker. To disable them go to System | Preferences | Appearance | Visual Effects and select none. It's possible I need to build gephex with OpenGL support and these options will co-exist better.

Also, my screencap frei0r module I've depended on extensively in the past updates extremely slowly on the laptop I'm using currently, it may be an ATI thing (I originally developed it on an Nvidia system).

Marker Tracking as Visualization Interface

2009-02-18T07:17:00.000-08:00

My idea is that I would be able to do an ARToolkit based visualization performance by using a clear table with markers I can slide, rotate, add and remove, and all those movement could correspond to events on screen. Unlike other AR videos the source video wouldn't be incorporated into the output necessarily, the markers provide an almost infinitely expressive set of UI knobs and sliders.

So far I have this:

AR User Interface from binarymillenium on Vimeo.

The lighting is difficult, the markers need to be white and black pixels but the plexiglass tends to produce reflections. Also if the light source itself is visible a marker will not be able to be right on top of it. I need a completely black backdrop under the plexiglass so there are no reflections that will obscure the markers, and also more numerous and softer diffuse lights.

One way to solve the reflection problem is to have the camera looking down at a table, though it's a little harder to get the camera up high enough, and I didn't want my hands or body to obscure the markers- the clear table idea is more elegant and self-contained.

The frame rate isn't very high, I need to work on making it all more real-time and responsive. It may have to be that one computer is capturing video and finding marker positions and sending them to another computer completely free to visualize it. Also more interpolation and position prediction could smooth things out, and cover up gaps if a marker isn't recognized in a frame, but that could produce more lag.

Bundler - the Photosynth core algorithms GPLed

2009-01-29T06:37:00.000-08:00

[update- the output of bundler is less misaligned looking than this, I was incorrectly displaying the results here and in the video]

Bundler (http://phototour.cs.washington.edu/bundler) takes photographs and can create 3D point clouds and camera positions derived from them similar to what Photosynth does- this is called structure from motion. It's hard to believe this has been out as long as the publically available Photosynth but I haven't heard about it- it seems to be in stealth mode.

Bundler - GPLed Photosynth - Car from binarymillenium on Vimeo.

From that video it is apparent that highly textured flat surfaces do best. The car is reflective and dull grey and so generates few correspondences, but the hubcaps, license plate, parking strip lines, and grass and trees work well. I wonder if this could be combined with a space carving technique to get a better car out of it.

It's a lot rougher around the edges lacking the Microsoft Live Labs contribution, a few sets I've tried have crashed with messages like "RunBundler.sh: line 60: 2404 Segmentation fault (core dumped) $MATCHKEYS list_keys.txt matches.init.txt" or sometimes individual images throw it with "This application has requested the Runtime to terminate it..." but it appears to plow through (until it reaches that former error).

Images without good EXIF data trip it up, the other day I was trying to search flickr and find only images that have EXIF data and allow full view, but am not successful so far. Some strings supposed limit search results by focal length, which seems like would limit results only to EXIF, but that wasn't the case.

Bundler outputs ply files, which can be read in Meshlab with the modification that these two lines be added to ply header:

element face 0
property list uchar int vertex_index

Without this Meshlab will give an error about there being no faces, and give up.

Also I have some Processing software that is a little less user friendly but doesn't require the editing:

http://code.google.com/p/binarymillenium/source/browse/trunk/processing/bundler/

Bundler can't handle filenames with spaces right now, I think I can fix this myself without too much work, it's mostly a matter of making sure names are passed everywhere with quotes around them.

Multi-megapixel files load up sift significantly until it crashes after taking a couple of gigabytes of memory (and probably not able to get more from windows):


...
[Found in EXIF tags]
  [CCD width = 5.720mm]
  [Resolution = 3072 x 2304]
  [Focal length (pixels) = 3114.965
[Found 18 good images]
[- Extracting keypoints -]

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.

Resizing them to 1600x1200 worked without crashing and took only a few hundred megabytes of memory per image, so more megapixels may work as well.

The most intriguing feature is the incremental option, I haven't tested it yet but it promises to be able to take new images and incorporate them into existing bundles. Unfortunately each new image has a matching time proportional to the number of previous images- maybe it would be possible to incrementally remove images also, or remove found points that are in regions that already have high point densities?

Nested blocks in Django duplication problems

2009-01-24T23:20:00.000-08:00

The official instructions and cursory google searches didn't turn up a good explanation, but I've figured it out for myself. I was confused about nesting blocks, sometimes getting no output or getting duplicate output.

In this example the base has a single level of nesting with two sub-blocks.

base.html:


{% block outer %}

{% block inner1 %}
this is inner1
{% endblock inner1 %}


{% block inner2 %}
this is inner2
{% endblock inner2 %}

{% endblock outer %}

This file duplicate the original block structure but adds a comment:
some.html:


{% extends base.html %}

{% block outer %}
{{ block.super }}

new stuff

{% endblock outer %}

The output would be

this is inner1
this is inner 2
new stuff

Moving the 'new stuff' line before the the block.super would swap the order of the output statements. There is no way to interject the new comment inbetween inner1 and inner2 without creating a new block that sits inbetween them in the parent base.html file.

Don't try to do this (which is what I thought to do initially):


{% extends base.html %}

{% block outer %}
{{ block.super }}

new stuff

{% block inner2 %}
new inner2
{% endblock inner2 %}

{% endblock outer %}

It will result in duplication like this:

this is inner1
new inner2
new stuff
new inner2

Instead, the extending file that wants to alter any parent block does it in a non-nested way, don't redefine an inherited block while inside of another inherited block:


{% extends base.html %}

{% block outer %}
{{ block.super }}

new stuff

{% endblock outer %}

{% block inner2 %}
new inner2
{% endblock inner2 %}

And now the output will be without duplication.

this is inner1
new inner2
new stuff

block.super needs to be in there or the redefinition of inner2 won't be applied to anything.

Laser Scanning

2009-01-04T09:13:00.000-08:00

The idea is to project laser lines onto a flat surface, image them, and then put objects in front of the surface and compute the displacement made by the object.

Here is the flat base with a line on it:

Here is the line at the same position with objects intersecting:

Finding depth involves figuring out what the 2d projection of the normal line that is perpendicular to the wall at any point along the laser line. I'm working on this but it's also possible to guess an average line for low precision demonstration. The software looks for all points where it believes the laser is shining, and then computes the intersection of the normal line with the original object free laser line, and gets depth.

I had about 8 different images from laser lines, here are the results from two:

The yellow lines are the projected normals from the base line to the found laser line on the backpack and broom. There are some spurious results, and also on the dark woven backpack material the laser was not always reflected strongly enough to register.

The source code is here:

http://code.google.com/p/binarymillenium/source/browse/trunk/processing/laserscan

Multicamera Balloon Imagery

2008-12-09T20:30:00.000-08:00

AHAB Tether Test from binarymillenium on Vimeo.

I originally was reminded of the unused source images because another team member recently posted some pictures from their camera, and then I made this, and then Bre posted about it, so it's getting a lot of positive feedback.

This video would have been made earlier but I had assumed that the cameras were screwy and firing at different times and image sequences would not line up at all- turns out they did, they just had wildly different start points.

Also I finished this school project that was kind of a simple occupancy grid inspired thing, this video shows parts of it:

2.5D from binarymillenium on Vimeo.

I might revisit some of this and get the registration code working (and working a lot faster), instead of cheating and using the known camera position and attitude.

Depth buffer to 3d coordinates?

2008-11-21T08:34:00.000-08:00

I'm having trouble transforming screen coordinates back to 3d, which this post describes- can anyone help me?

---
Update - I've got it figured out now, I should have been using gluUnProject:



FloatBuffer fb;

fb = BufferUtil.newFloatBuffer(width*height);

gl.glReadPixels(0, 0, width, height, GL.GL_DEPTH_COMPONENT, GL.GL_FLOAT, fb); 
fb.rewind();

int viewport[] = new int[4]; 
double[] proj=new double[16];
double[] model=new double[16];
gl.glGetIntegerv(GL.GL_VIEWPORT, viewport, 0);
gl.glGetDoublev(GL.GL_PROJECTION_MATRIX,proj,0);
gl.glGetDoublev(GL.GL_MODELVIEW_MATRIX,model,0);

...
for(int i...
for (int j...
...
glu.gluUnProject(i,height-j,rawd, model,0,proj,0,viewport,0,pos,0); 
float d = (float)-pos[2];

After all that depth d will be linear and in proper world coordinates.

Artoolkit + rangefinder

2008-11-20T07:02:00.000-08:00

Since my relatively inexpensive purely visual depth map approach wasn't that successful, I've tried it out using a rangefinder instead of a visible laser. This means I can point the video camera straight at the marker (which is attached to the rangefinder), and it can point at anything provided I don't tilt it so the camera can't see the marker/fiducial.

This is the result:

Artoolkit with a Rangefinder from binarymillenium on Vimeo.

The following plots show the tracked attitude of the rangefinder as measured by ARToolkit:

My left to right bottom to top scanning approach is very apparent.

And here is the tracked attitude (as a 3-component vector) plus the range vs. time:

You can see how cyclical it is, as I scan the floor in front of me the range doesn't change much until I reach one end and tilt the tripod up a little, and then later on I start to capture the two wheels of the car.

University of Washington BioRobotics Lab

2008-11-09T08:16:00.000-08:00

I took a tour of the UW BioRobotics Lab, where an old professor of mine works on telerobotics with haptic interfaces.

This is a surgery robot called 'The Raven'. It's mostly camouflaged due to the large amounts of detail and contrast in the robot itself and in the background. The DV camera is going to be replaced by a pair of HD cameras that will provide stereo vision.

Multiple motors pull on cables seen in a later photo that control the manipulator end of the arm.

Blake Hannaford shows the arms that will replace the manually positioned arms seen in the previous photos.

2008-10-14T20:35:00.001-07:00

Hey lil feller. I'm just going to pick you up with this stick and you can hang on, take you back to the garden. I didn't mean to wake you up from your hibernation or whatever.

Artoolkit rangefinding continued

2008-10-07T21:10:00.000-07:00

I've discovered the arParamObserv2Ideal() function to correct for image distortion, and I think it has improved things. But my main problem is figuring out how to properly project the line from the origin through the laser dot in marker/fiducial space. I have a message out on the mailing list but it is not that active.

The results of my crude fillgaps processing app are shown below, using the somewhat sparse points from above.

The above results look pretty good- the points along the edge of the wall and floor are the furthest from the camera so appear black, and the floor and wall toward the top and bottom of the image are closer and get brighter.

My main problem is getting live feedback of where I've gotten points. With a live view that showed all found depth points it would be easier to achieve uniform coverage, rather than going off of memory. My problem there is that to use artoolkit I have to detect the markers, then shrink the image down and draw dots over it- not too hard sounding but the first time I tried it got all messed up.

Depth Maps with ARToolkit and a Laser Pointer

2008-10-05T21:43:00.000-07:00

Flying home from a recent trip to the east coast, I tried to figure out what the most inexpensive method for approximating scanning lidar would be. This is my answer:

ARToolKit assisted laser rangefinding from binarymillenium on Vimeo.

It's not that inexpensive, since I'm using a high resolution network camera similar to an Elphel- but it's possible I could replace it with a good consumer still camera with a linux supported usb interface for getting live images.

In the above screen shot the line projected from the found fiducial is shown, and the target where the found red laser dot is- they ought to cross each other but I need to learn more about transforming coordinates in and out of the camera space in ARToolkit to improve upon it.

This picture shows that the left side of the screen is 'further' away than the wall on the right, but that is not quite right- it is definitely further away from the fiducial, so I may be making an error.

source code

Intel Research Seattle - Open House

2008-10-05T09:59:00.000-07:00

From 2008.10.01 Intel Research Seattle

Robotic Arm

From 2008.10.01 Intel Research Seattle

This robot had a camera and em field sensors in its hand, and could detect the presence of an object within grasping distant. Some objects it had been trained to recognize, and others it did not but would attempt to grab anyway. Voice synthesis provided additional feedback- most humorously when it accidently (?) dropped something it said 'oops'. Also motor feedback sensed when the arm was pushed on, and the arm would give way- making it much safer than an arm that would blindly power through any obstacle.

From 2008.10.01 Intel Research Seattle

It's insane, this application level taint

From 2008.10.01 Intel Research Seattle

I don't really know what this is about...

Directional phased array wireless networking

From 2008.10.01 Intel Research Seattle

The phased array part is pretty cool, but the application wasn't that compelling: Using two directional antennas a select zone can be provided with wireless access and other zones not overlapped by the two excluded. Maybe if it's more power efficient that way?

From 2008.10.01 Intel Research Seattle

This antenna also had a motorized base, so that comparisons between physically rotating the antenna and rotating the field pattern could be made.

Haptic squeeze

From 2008.10.01 Intel Research Seattle

This squeeze thing has a motor in it to resist pressure, but was broken at the time I saw it. The presenter said it wasn't really intended to simulate handling of real objects in virtual space like other haptic interfaces might, but be used more abstractly as a interface to anything.

From 2008.10.01 Intel Research Seattle

RFID Accelerometer

From 2008.10.01 Intel Research Seattle

This was one of two rfid accelerometers- powered entirely from the rfid antenna, the device sends back accelerometer data to rotate a planet on a computer screen. The range was very limited, and the update rate about 10 Hz. The second device could be charged within range of the field, then be taken out of range and moved around, then brought back to the antenna and download a time history (only 2 Hz now) of measurements taken. The canonical application is putting the device in a shipped package and then reading what it experienced upon receipt.

Wireless Resonant Energy

From 2008.10.01 Intel Research Seattle

This has had plenty of coverage elsewhere but is very cool to see in person. Currently moving the receiver end more than an inch forward or back or rotating it causes the light buld to dim and go out.

Scratch Interface

From 2008.10.01 Intel Research Seattle

A simple interface where placing a microphone on a surface and then tapping on the surface in distinct ways can be used to control a device. Also a very simple demo of using opencv face tracking to reorient a displayed video to correct for distortion seen when viewing a flat screen from an angle.

Look around the building

From 2008.10.01 Intel Research Seattle

I found myself walking in a circle and not quite intuitively feeling I had completed a circuit when I actually had.

Also see another article about this, and Intel's flickr photos, and a a video.

initialization discards qualifiers from pointer target type

2008-09-26T21:08:00.000-07:00

A const may be needed, the following produces the error:


 PixelPacket *p = AcquireImagePixels(image,0,y,image->columns,1,&image->exception);

while this fixes it:


 const PixelPacket *p = AcquireImagePixels(image,0,y,image->columns,1,&image->exception);

In other news hopefully soon I'll have an ARToolkit app for reading in jpegs using ImageMagick, and also that app will have some other more exciting attributes.

Increased Dynamic Range For Depth Maps, and Collages in Picasa 3

2008-09-08T22:24:00.000-07:00

360 Vision + 3rd Person Composite from binarymillenium on Vimeo.

After I compressed the above video into a WMV I was dissatisfied with how little depth detail there is in the 360 degree vision part - it's the top strip. I initially liked the cleaner single shade look, but now I realize the utility of using a range of colors for depth fields (or IR/thermal imaging also)- it increases the number of colors to represent different depths beyond 256 to a larger number. Earlier I tried using one color channel for higher order bits and another for lower order bits (so the depth could be computed like red*256+green) for a total of 256*256 depth levels (or even 256^3 or 256^4 using alpha), but visually it's a mess.

But visual integrity can be maintained while multiplying that 256 levels by five or a bit more with additional work.

Taking six colors, three of them are pure red, green, blue, and inbetween there is (255,255,0) for yellow and the other two pure combinations of two channels. Between each subsequent set there can be 256 interpolated values, and in the end a color bar like the follow is generated with 1280 different values:

The bottom color bar shows the differences between adjacent values- if the difference was none then it would be black in spots, so my interpolation is verified.

Applying this to the lidar data, I produced a series of images with a processing project:

After making all the images I tried out Picasa 3 to produce a collage- the straightforward grid makes the most sense here. Picasa 3 crashed a few times in the collage editor before I was able to get this exported.

Photosynth Export Process Tutorial

2008-08-31T07:50:00.000-07:00

It looks like I have unofficial recognition/support for my export process, but I get the feeling it's still too user unfriendly:

http://getsatisfaction.com/livelabs/topics/pointcloud_exporter

What to do

Get Wireshark http://www.wireshark.org/

Allow it to install the special software to intercept packets.

Start Wireshark. Put


http.request

into the filter field.

Quit any unnecessary network activity like playing youtube videos- this will dump in a lot of data to wireshark that will making finding the bin files harder.

*** Update ***
Some users have found the bin files stored locally in %temp%/photosynther, which makes finding them much easier than using Wireshark, but the for me on Vista the directory exists for one user but not others- but the bin files have to be stored locally somewhere right?
***

Open the photosynth site in a browser. Find a synth with a good point cloud, it will probably be one with several hundred photos and a synthiness of > 70%. There are some synths that are 100% synthy but have point clouds that are flat billboards rather than cool 3D features- you don't want those. Press p or hold ctrl to see the underlying point cloud.

*** Update ***
Use the direct 3d viewer option to view the synth, otherwise you won't get the synth files. (thanks losap)
***

Start a capture in Wireshark - the upper left butter and then click the proper interface (experiment if necessary).

Hit reload on the browser window showing the synth. Wireshark should then start show ing what files are being sent to your computer. Stop the capture once the browser has finished reloading. There may be a couple screen fulls but near the bottom should be a few listings of bin files.

Select one of the lines that shows a bin file request, and right-click and hit Copy | Summary (text). Then in a new browser window paste that into the address field. Delete the parts before and after /d8/348345348.../points_0_0.bin. Look back in Wireshark to discover what http address to use prior the that- it should be http://mslabs-nnn.vo.llnwd.net, but where nnn is any three digit number. TBD- is there a way to cut and paste the fully formed url less manually?

If done correctly hit return and make the browser load the file- a dialog will pop up, save it to disk. If there were many points bin files increment the 0 in the file name and get them all. If you have cygwin a bash script works well:


for i in `seq 0 23`
do
wget http://someurl/points_0_$i.bin
done

Python

Install python. If you have cygwin installed the cygwin python with setup.exe, otherwise http://www.python.org/download/ and download the windows installer version.
*** UPDATE *** It appears the 2.5.2 windows python doesn't work correctly, which I'll look into- the best solution is to use Linux or Cygwin with the python that can be installed with Linux ***

Currently the script http://binarymillenium.googlecode.com/svn/trunk/processing/psynth/bin_to_csv.py works like this from the command line:


python bin_to_csv.py somefile.bin > output.csv

But I think the '>' will only work with cygwin and not the windows command prompt. I'll update the script to optionally take a second argument that is the output file.

If there are multiple points bin files it's easy to do another bash loop to process them all in a row, otherwise manually do the command above and create n different csvs for n bin files, and then cut and paste the contents of each into one complete csv file.

The output will be file with a long listing of numbers, each one looks like this:


-4.17390823, -1.38746762, 0.832364499, 24, 21, 16
-4.07660007, -1.83771312, 1.971277475, 17, 14, 9
-4.13320493, -2.56310105, 2.301105737, 10, 6, 0
-2.97198987, -1.44950056, 0.194522276, 15, 12, 8
-2.96658635, -1.45545017, 0.181564241, 15, 13, 10
-4.20609378, -2.08472299, 1.701148629, 25, 22, 18

The first three numbers are the xyz coordinates of a point, and the last three is the red, green, and blue components of the color. In order to get a convention 0-255 number for each color channel red and blue would have to be multiplied by 8, and green by 4. The python script could be easily changed to do that, or even convert the color channels to 0.0-1.0 floating point numbers.

Point Clouds - What Next?
The processing files here can use the point clouds:
http://binarymillenium.googlecode.com/svn/trunk/processing/psynth/

Also programs like Meshlab can use them with some modification- I haven't experimented with it much but I'll look into that and make a post about it.

Color Correction

2008-08-28T20:59:00.001-07:00

I have the colors figured out now: I was forgetting to byteswap the two color bytes, and after that the rgb elements line up nicely. And it's 5:6:5 bits per color channel rather than 4 as I thought previously, thanks to Marvin who commented below.

The sphinx above looks right, but earlier the boxer shown below looked so wrong I colored it falsely to make the video:

The Boxer - Photosynth Export from binarymillenium on Vimeo.

But I've fixed the boxer now:

The python script is updated with this code:


  bin.byteswap()
  red =  (bin[0] >> 11) & 0x1f
  green = (bin[0] >> 5)  & 0x3f
  blue =  (bin[0] >> 0)  & 0x1f

Exporting Point Clouds From Photosynth

2008-08-27T10:28:00.000-07:00

Since my last post about photosynth I've revisited the site and discovered that the pictures can be toggled off with the 'p' key, and the viewing experience is much improved given there is a good point cloud underneath. But what use is a point cloud inside a browser window if it can't be exported to be manipulated into random videos that could look like all the lidar videos I've made, or turned into 3D meshes and used in Maya or any other program?

Supposedly export will be added in the future, but I'm impatient like one of the posters on that thread so I've gone forward and figured out my own export method without any deep hacking that might violate the terms of use.

Using one of those programs to intercept 3D api calls might work, though maybe not with DirectX or however the photosynth browser window is working. What I found with Wireshark is that http requests for a series of points_m_n.bin files are made. The m is the group number, if the photosynth is 100% synthy then there will only be one group labeled 0. The n splits up the point cloud into smaller files, for a small synth there could just be points_0_0.bin.

Inside each bin file is raw binary data. There is a variable length header which I have no idea how to interpret, sometimes it is 15 bytes long and sometimes hundreds or thousands of bytes long (though it seems to be shorter in smaller synths).

But after the header there is a regular set of position and color values each 14 bytes long. The first 3 sets of 4 bytes are the xyz position in floating point values. In python I had to do a byteswap on those bytes (presumably from network order) to get them to be read in right with the readfile command.

The last 2 bytes is the color of the point. It's only 4-bits per color channel, which is strange. The first four bits I don't know about, the last three sets of 4 bits are red, blue, and green. Why not 8-bits per channel, does the photosynth process not produce that level of precision because it is only loosely matching the color of corresponding points in photos? Anyway as the picture above shows I'm doing the color wrong- if I have a pure red or green synth it looks right, but maybe a different color model than standard rgb is at work.

I tried making a photosynth of photos that were masked to be blue only- and zero synthiness resulted - is it ignoring blue because it doesn't want to synth up the sky in photos?

Anyway here is the python script for interpreting the bin files.

The sceneviewer (taken from the Radiohead sceneviewer) in that source dir works well for displaying them also.

Anyway to repeat this for any synth wireshark needs to figure out where the bin files are served from (filter with http.request), and then they can be downloaded in firefox or with wget or curl, and then my script can be run on them, and processing can view them. The TOC doesn't clearly specify how the point clouds are covered so redistribution of point clouds, especially those not from your own synths or someone who didn't CC license it, may not be kosher.

More python pcap with pcapy

2008-08-24T00:50:00.000-07:00

After running into pcap files several hundreds of megabytes in size that caused wireshark to crash when loaded, I returned to trying to make python work with the source pcap file:


import pcapy

vel = pcapy.open_offline('unit 46 sample capture velodyne area.pcap')

vel
Reader object at 0xb7e1b308

pkt = vel.next

pkt
built-in method next of Reader object at 0xb7e1b308

What is a Reader object, and a built-in method of it? Why are the addresses the same?

try


pkt = vel.next()

type(vel)
type 'tuple'

vel[1]


'\xff\xff\xff\xff\xff\xff\x00\x00\x00\x00\x00\x00\x08\x00E\x00\x04\xd2
\x00\x01\x00\x00\x80\x11\xad\x9f\xc0\xa8\x03+\xc0\xa8\x03\xff\x01\xbb
\t@\x04\xbe\x00\x00\xff\xddz\x13\xee
...
\x00\x00\x10\xd63Md\xc2\xff\x00\x00\x0fp\x07v25b'

So that's just the problem I was running into before: '\xff' is an ascii representation of binary data that's really 0xff. But I'm given it in ascii- I can't index into this and get a specific 8-bit 0-255 binary value, I get a '\'. Do I have to write something that goes through this and reinterpets the ascii-ized hex back into real hex?

Also I note the ff dd that marks the beginning of the data frame is there but not at the beginning- so there are other parts of the packet here I need to get rid of. Is this where I need Impacket?

import impacket
from impacket.ImpactDecoder import EthDecoder


decoder = EthDecoder()
b = decoder.decode(a)
Traceback (most recent call last):
File "stdin", line 1, in ?
File "/var/lib/python-support/python2.4/impacket/ImpactDecoder.py", line 38, in decode
  e = ImpactPacket.Ethernet(aBuffer)
File "/var/lib/python-support/python2.4/impacket/ImpactPacket.py", line 340, in __init__
  self.load_header(aBuffer)
File "/var/lib/python-support/python2.4/impacket/ImpactPacket.py", line 255, in load_header
  self.set_bytes_from_string(aBuffer)
File "/var/lib/python-support/python2.4/impacket/ImpactPacket.py", line 59, in set_bytes_from_string
  self.__bytes = array.array('B', data)
TypeError: an integer is required

oops


b = decoder.decode(a[1])
print b
Ether: 0:0:0:0:0:0 -> ff:ff:ff:ff:ff:ff
IP 192.168.3.43 -> 192.168.3.255
UDP 443 -> 2368

ffdd 7a13 ee09 3cbe 093f 1811 2b1f 1020    ..z.....?..+..
0a0a 63ac 0848 d708 53ea 085b 0000 1bc0    ..c..H..S..[....
0a35 0c09 425b 0936 000d 2e44 0d2f 120b    .5..B[.6...D./..
4f5e 0b30 200f 3c4e 0e50 c30b 46c7 0b4d    O^.0 .

The decode makes a nice human readable text of the packet, but not what I want.

Here is a different tack- by looking in the Impacket.py source I found how to do this which converts that annoying ascii back to real bytes, which is the only real issue:


mybytes = array.array('B', vel[1])

mybytes is of size 1248, so there appear to be 42 extra bytes of unwanted ethernet wrapper there- why not just index into mbytes like mybytes[42:] and dump that to a binary file?

I don't know about the dumping to binary file (print mybytes prints it in ascii, not binary)- but I could easily pass that array straight into the velodyne parsing code- and this skips the intermediate file step, saving time and precious room on my always nearly full laptop hd.

So here is the final result, which produce good CSVs I was able to load with my 'velosphere' Processing project to create 360 degree panoramas from the lidar data:

Next I need a way to write pngs from python, and I could eliminate the CSVs & Processing step.

Photosynth

2008-08-23T06:31:00.000-07:00

When I first saw the original demo I was really impressed, but now that is been released I feel like it hasn't advanced enough since that demo to really be useful. I tried a few random synths when the server was having problems, it looks like it isn't being hammered any longer so I ought to try it again soon when I'm using a compatible OS.

Overall it's confused and muddled to use and look at- like a broken quicktime VR.

Photosynth seems to work best in terms of interface and experience when it is simply a panoramic viewer of stitched together images- where all the images are taken from a point of buildings or scenery around the viewer. It's easy to click left or right to rotate left or right and have the view intuitively change. But we've had photostitching software that produces smooth panoramas that look better than this for years, so there's nothing to offer here.

When viewing more complicated synths, the UI really breaks down. I don't understand why when I click and drag the mouse the view rotates to where I'd like, but then it snaps back to where it used to be when I let go of the button. It's very hard to move naturally through 3D space- I think the main problem is that the program is too photo-centric: it always wants to feature a single photograph prominently rather than a more synthetic view. Why can't I pull back to view all the photos, or at least a jumble of outlines of all the photos?

It seems like there is an interesting 3D point cloud of points found to be common to multiple photos underlying the synth but it can't be viewed on it's own (much less downloaded...), there are always photos obscuring it. The photograph prominence is constantly causing nearby photos to become blurry or transparent in visually disruptive ways.

Finally, it seems like the natural end-point of technology like this is to generate 3D textured models of a location, with viewing of the source photos as a feature but not the most prominent mode. Can this be done with photosynth-like technology or is all the aspects I don't like a way of covering up that it can't actually do that? Maybe it can produce 3D models but they all come out horribly distorted (so then provide a UI to manually undistort them).

Hopefully they will improve on this, or another well-backed site will deliver fully on the promise shown here.

Makeavi

2008-08-20T08:48:00.000-07:00

Discovered a neat windows (and vista) tool for turning image sequences into videos: http://makeavi.sourceforge.net/

1280x720 in the 'Microsoft Video 1' format worked well, though 57 MB of pngs turned into 135 MB of video. 'Uncompressed' didn't produce a video just a small 23kb file. 'Intel IYUV' sort of produced a video but not correctly. 'Cinepak' only output a single frame. 'VP60 Simple profile' and 'VP61 Advanced Profile' with the default settings worked, and actually produces video smaller than the source images, though quicktime player didn't like those files. Vimeo seems to think VP61 is okay:

More Velodyne Lidar - overhead view from binarymillenium on Vimeo.

This new video is similar to the animated gifs I was producing earlier, but using a new set of data. Vimeo seems to be acting up this morning, I got 75% through an upload of the entire file (the above is just a subset) and it locked up. I may try to produce a shorter test video to see if it works.

I have around 10 gigs of lidar data from Velodyne, and of course no way to host it.

My process for taking pcap files and exporting the raw data has run into a hitch- wireshark crashes when trying to 'follow udp stream' for pcap files larger than a couple hundred megabytes. Maybe there is another tool that can do the conversion to raw?

Phase Correlation for Lidar Image Alignment

2008-08-14T21:42:00.000-07:00

After going through the manual alignment of the point cloud data noted in the last post, I've gotten the translation step of automatic alignment using phase correlation.

Phase correlation requires the use of a 2d fft, which I couldn't find in Processing (would have to bring in a java lib for that somehow?). Instead I used Octave, which has the fft2, inverse ifft2 function, and many other convenient math functions. The Matlab/Octave file is here.

The fundamental phase correlation code is this:


a = imread('one.png')
b = imread('two.png')

af = fft2(a);
bf = fft2(b);

% cross power
cp = af.*conj(bf) ./ abs(af.*conj(bf));

icp = (ifft2(cp));

mmax = max(max(icp));
[sx,sy,v] = find(mmax == icp);

And sx and sy are the translation to apply to b to make it line up with image a. An additional check is to make sure the largest value in icp is above some threshold- if it is lower than the threshold then there is no good translation to align the data.

I'm a little suspicious that my input pngs are too regular and easy, every frame seems a constant displacement from the former as the vehicle with the lidar was moving at a constant velocity.

Rotation is mostly working in isolation but I need to revisit the proper method to make simultaneous rotation and translation work- there was a paper somewhere I need to dig up.

Point Cloud Alignment

2008-08-11T08:37:00.001-07:00

I took about 1/5th of the png images generated from the Velodyne point cloud data and manually aligned them in Gimp. It's easy to shift-select a bunch of images and open them as individual layers in Gimp, but there are no multiple layer selection capabilities: it's not possible for instance select all the layers and change their transparency.

In all of the images there are a few features, mostly beyond the edges of the road, that can be aligned with the earlier and later images. The closer together the images are in time the easier this is, but I was skipping every 4 images to take the 5th in order to speed up the process- also a gimp image with 300 layers is difficult to handle.

The later portions of the data are all purely translational, only at the very beginning are rotations and translations needed.

I think an automatic process for alignment won't be that hard, but the inherent inconsistency in frame to frame image will make for a lot of error. Translations correspond to phase shifts in the frequency domain, and I think rotations are almost as simple- and there isn't any scaling to account for.

Dorkbot Cacophony

2008-08-07T06:21:00.000-07:00

The theme is catastrophic cacophony and the challenge is to build the loudest, craziest, flashiest automated "musical" instrument ever. You will have to be able to step back and let your creation play by itself, even if it destroys itself in the process.

This event was a lot of fun, I helped build a sort of guitar that gets strummed by a motor. With a little more time I think we could have made it less repetitive by add another motor to move the first one back and forth or up and down.

Other teams had things that sounded like higher frequency wind-chimes, or just pure noise.

Dorkbot Cacophony from binarymillenium on Vimeo.

Animated gif of height map

2008-08-03T17:23:00.000-07:00

animated gif of height map

Source code is here:

http://code.google.com/p/binarymillenium/source/browse/#svn/trunk/processing/velodyne

One interesting thing I discovered is that animated gifs with an alpha channel don't just let the back ground of the web page show through, they also don't clear the last frame of the gif- which was confusing for this gif before I blackened the background with ImageMagick:


for i in *png; do convert $i -background black -flatten +matte flat_$i; done
convert flat*png velodyne_hgt.gif

Velodyne Lidar Sample Data

2008-08-03T16:39:00.000-07:00

Applying the db.xml calibration file

As with the pcap parsing, I originally thought I'd look into Python xml parsing. I'm sure if I was really interested in parsing xml I would have tried some of them out, but the interface I was hoping to find would look like this


import pyxmlthing
a = pyxmlthing.load("some.xml")
 some_array= a.item('9').rotCorrection

And I would have an array of all the rotCorrections of all the items of type 9. Instead I found a several xml parsers that required tons of code to get to a point where I'm still not sure if I could get at the rotCorrections or not. Which may be why I don't care for xml, flat-files are it.

So I just used vim to strip all the xml out and leave me with a nice text file that looks like this


0,     -3.8,     -7.0046468,     20,     21.560343,     -2.5999999
    1,     -1.5,     -6.7674689,     26,     21.516994,     2.5999999
    2,     5,     0.44408101,     28,     20.617426,     -2.5999999
    3,     6.8000002,     0.78093398,     32,     20.574717,     2.5999999

Where the first column is the laser index (0-63), the next might be the rotCorrection and so on.

To apply the calibration data, it's very important that the indexing derived from the raw data is correct- reading the 0xDDEE vs. the 0xDDFF (or similar) that designates upper or lower laser block is important.

The velodyne manual doesn't have a good diagram that shows the xyz axes and what is positive or negative direction for both the original lidar angle and the correction angles and offsets, so some experimentation is necessary there. The manual did mention it in this case the rotCorrection had to be subtracted from the base rotation angle. The vertical and horizontal offset are pretty minor for my visualization but important for accurate measurements obviously.

Processing viewer

Using Aaron Koblin's House of Cards SceneViewer as a starting point, I wrote code to take the point cloud data and display it:

Monterey Full from binarymillenium on Vimeo.

The data was split into files each containing a million data points and spanning a second in time. With some testing I found the lidar was spinning at about 10 Hz (of the possible 5, 10, or 15 Hz), so I would bite off 1/10 of the file and display that in the current frame, then the next 1/10th for the next frame, and then load the next file after 10 frames. A more consistent approach would be to split the data into a new text file for each frame.

Next Steps

Right now I'm running a large job to process each point-cloud frame into png height-map files as I did for the Radiohead data. It doesn't work as well with the 360 degrees of heights- some distances just have to be cut off and ignored to keep the resolutions low (although with the png compression having large empty spaces doesn't really take up any additional room). A lot of detail is lost on the nearby objects compared to a lot of empty space between distant objects.

So either using that data or going back to processing the raw point cloud, I'd like to track features frame-to-frame and derive the vehicle motion from them. I suspect this will be very difficult. Once I have the vehicle motion, I could create per-frame transformations that could create one massive point cloud or height map where still objects are in their proper places and other vehicles probably become blurs.

After that, if I can get a dataset from Velodyne or another source where a moving ground lidar moves in a circle or otherwise intersects its own path somewhere, then the proof that the algorithm works will be if in that big point cloud that point of intersection actually lines up. (though I suspect again that more advanced logic is need to re-align the data after the logic determines that it has encounter features that it has scene before).

Velodyne Lidar- Monterey Data

2008-07-30T20:22:00.000-07:00

There's something wrong with how I'm viewing the data, the streaks indicate every revolution of the lidar results in an angular drift- maybe I'm re-adding a rotational offset and allowing it to accumulate?

No, it turned out to be a combination of things- applying the wrong calibration data (essentially flopping the lower and upper laser blocks). Now it's starting to look pretty good, though there is a lot of blurriness because the environment is changing over time- that's a car pulling out into an intersection.

Velodyne Lidar - Monterey from binarymillenium on Vimeo.

Velodyne Lidar Sample Data: Getting a .pcap into Python

2008-07-29T22:00:00.000-07:00

Velodyne has provided me with this sample data from their HDL-64E lidar.

Instead of data exported from their viewer software, it's a pcap captured with Wireshark or a another network capture tool that uses the standard pcap format.

Initially I was trying to extract the lidar packets with libpcap and python, using pcapy or similar, and went through a lot of trouble getting the pcap library to build in the cygwin environment. Python and libpcap were able to load the data from the pcap, but rendered the binary into a long string with escape codes like '\xff'.

I then discovered that Wireshark has a function called 'follow udp stream'- right-click on the data part of a packet in Wireshark and then export as 'raw'. The exported data doesn't preserve division between packets any longer, but since each is of a consistent length (1206 bytes) it's easy to parse.

Python does work well for loading binary data from file:


import array

f = open('data.raw')  # the data exported from wireshark in the raw format

bin = array.array('B') # setup an array of typecode unsigned byte

bin.fromfile(f, 1206) # each packet has 1206 bytes of data

The contents of bin now have 1206 bytes of data that looks like this:


>>> bin
array('B', [255, 221, 33, 39, 5, 9, 67, 178, 8, 116, 160, 13, 126, 222, 13, 63, 217, 8, 162, 204, ...

The 'B' element doesn't prevent indexing into the real data with bin[0] and getting 255 and so on.

The first two bytes indicate whether the data is from the upper or lower array, and the 33 39 is interpreted to mean that the reading was taken when the sensor head was rotated to (39*255+33)/100 or 99.78 degrees.

For 32 times after those first two bytes there are pairs of distance bytes followed by single intensity bytes, and then it starts over for a total of 12 times... and there are 6 more bytes with information that is unnecessary now. See here for what I currently have for parsing the raw file, later I will get into add the per laser calibration data.

More HoC: Preprocessing into pngs

2008-07-26T07:03:00.000-07:00

house of cards height
house of cards intensity

Radiohead - House of Cards 2

2008-07-21T07:46:00.000-07:00

Radiohead - House of Cards from binarymillenium on Vimeo.

This rendered slowly, less than 1 fps. One possible speedup would be to pre-process the csv data into binary heightmap files, rather than loading and processing each frame.

Processing code is here:
http://code.google.com/p/binarymillenium/source/browse/trunk/processing/hoc/hoc.pde

It would be nice if they uploaded the data from the woman singing, and the party scene.

Radiohead - House of Cards

2008-07-15T22:20:00.000-07:00

I've been playing with the data for a couple of hours in Processing. The main problem is that the points are not consistent across animation frames, so it is necessary to produce a new set of points in a regular grid that then can be tessellated easily. By the end of the week I ought to have a video up in the official group on youtube and in higher quality on vimeo.

openprocessing

2008-06-23T21:48:00.000-07:00

This site is good for showcasing Processing projects, but is really rough around the corners- it lacks almost every feature of sites for sharing photos or art or anything except for leaving comments, and tagging. The 'viewed nn times' increments when the page is reloaded, which is like a hit counter straight out of 1997. There's no real good way for good graphs to float to the top and be seen more except for an unknown process by which editors select certain submissions for exhibition. There are a lot of applets that don't work but aren't hidden or demoted from view.

It's not like there are no other options for hosting java applets for free- google code works pretty good, and by putting google analytics javascript into the html page it's easy to track the traffic to the page.

But there do seem to be a number of people that visit the site and look at my submissions, but I think I have more unique views from using the exhibition rollup on the official processing.org site.

thingamajiggr, and 2 new Processing effects

2008-06-14T07:25:00.000-07:00

I'm beginning to incorporate more processing effects into my sets, so I developed two for thingamajiggr. They are less friendly to live alteration than gephex graphs, but if the bulk of the effect is done it's easy to live-code simple keyboard controls or change parameters.

I need a way to get video or screen captures into Processing, on Linux this is not easy- if it doesn't already exist I'm thinking of sending image data from gephex to Processing through an internal loopback path.

Audio input used to work for me but now it doesn't.

First effect: multicolored perlin lines. When shown live the colors dovetailed with led lighting of an artwork in the main theater room:

(click on the image to see the java applet)

The second effect is a standard gas/fluid simulation that uses springs. I spent a lot of time trying to get more complicated flow fields working, but I ended up moderately simple:

(click on the picture to see and play with the java applet)

Things that went wrong or I need to fix:

Holding my laptop on my lap is not very ideal when my laptop has an rgb output without locking screw holes. The connector fell out a few times. The other lesson is to always have good surface so the laptop can be left still.

Not being able to eliminate window manager titles and borders around processing output. I should find a more minimalist window manager.

Optical Flow in Processing

2008-06-08T15:55:00.000-07:00

The standard Lucas-Kanade method as described in the wiki page entry on it here. One key thing not mentioned there (at least until I edit the page) is to use the Sobel operator to get partial derivatives in the x and y direction.

link to applet

Glancing at the OpenCV implementation I notice there is a check for certain kinds of matrices that aren't invertible by the primary method, I may need to be doing that for more robust operation.

Hosting java with google code

2008-06-01T16:56:00.000-07:00

I used the export function of Processing to create a jar of an a-star search/a* search demonstration, and added the files into my google code subversion repository. Then it's necessary to set the proper mime-types:


svn propset svn:mime-type application/java-archive astar.jar
svn propset svn:mime-type text/html index.html

Here's an example:
astar

I don't think I can embed the jar directly here, I get this error:


load: class astar.class not found.
java.lang.ClassNotFoundException: astar.class
 at sun.applet.AppletClassLoader.findClass(AppletClassLoader.java:168)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
 at sun.applet.AppletClassLoader.loadClass(AppletClassLoader.java:119)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
 at sun.applet.AppletClassLoader.loadCode(AppletClassLoader.java:599)
 at sun.applet.AppletPanel.createApplet(AppletPanel.java:721)
 at sun.plugin.AppletViewer.createApplet(AppletViewer.java:1781)
 at sun.applet.AppletPanel.runLoader(AppletPanel.java:650)
 at sun.applet.AppletPanel.run(AppletPanel.java:324)
 at java.lang.Thread.run(Thread.java:595)
Caused by: java.io.IOException: open HTTP connection failed.
 at sun.applet.AppletClassLoader.getBytes(AppletClassLoader.java:271)
 at sun.applet.AppletClassLoader.access$100(AppletClassLoader.java:44)
 at sun.applet.AppletClassLoader$1.run(AppletClassLoader.java:158)
 at java.security.AccessController.doPrivileged(Native Method)
 at sun.applet.AppletClassLoader.findClass(AppletClassLoader.java:155)
 ... 9 more

It seems like certain colormodes don't work when exporting from processing (135)- use the integer 0-255 colormode.

For a jnlp (like created with the gwt kit):

K-means clustering, hough transform

2008-05-29T21:56:00.000-07:00

The hough transform is typically used to search for lines in images, though can be generalized to other forms, and for example is used to find the edge of roads for mobile robotics applications.

I'm not sure if I implemented it quite correctly- what for example should a solid white input image look like? In any case it can make some pretty but somewhat bland curvy colored lines, below the picture on the right is the source and the left is the hough transform:

description and link to processing pde file

The k-means clustering algorithm is very simple but very interesting in it's results- an image is split into clusters and then the clusters take on the average color of the source pixels, and the cluster centers are updated in each step until convergence.

K-Means Clustering Frei0r from binarymillenium on Vimeo.

thingamajiggr

2008-05-29T21:27:00.000-07:00

Hopefully this will go as well as the Dorkbot show from December- I have some new effects now and ought to get some more done in time for it.

Senzala Capoeira at the University District Street Fair

2008-05-18T13:36:00.000-07:00

I thought I saw this group before, sure enough I have a bunch of pictures of them from a year ago at the Georgetown Ol' Skool Carnival:

Toorcon Seattle 2008- HBL Afterparty

2008-04-22T19:30:00.000-07:00

Not much to say about this one, I didn't have much time to prepare... next show I'll develop some new effects.

Retarded Glide Bomb

2008-02-10T12:59:00.000-08:00

It's kind of retarded.

Finally Ubuntu Microphone in

2008-01-07T22:03:00.000-08:00

I've been slow to update my Ubuntu 6.10, mainly because the nvidia drivers need to be reconfigured (envy rerun basically), and minor other things might break. The microphone input didn't work initially, but only the other day did I try to fix it- apparently the alsa drivers didn't fully support the hda-intel MCP51 soundcard I had.

So the non-full update thing was to download the latest alsa driver, lib, utils from http://www.alsa-project.org/main/index.php/Main_Page and configure-make-install them. driver had to be configured with --with-cards=hda-intel.

After that:
/etc/modprobe.d/alsa-base
options snd-hda-intel model=laptop

After restarting, the gnome sound recorder and audacity recorder worked fine, but gephex depends on there being a /dev/dsp0. I only have a /dev/dsp, and creating a symlink with ln -s /dev/dsp /dev/dsp0 works well but doesn't persist after a restart. So edit /etc/udev/rules.d/60-symlinks.rules and add the line:
KERNEL=="dsp", SYMLINK+="dsp0"

Next Year

2008-01-02T19:50:00.000-08:00

http://flickr.com/photos/released/2159331315/

I and a few others from Hackerbot put a bunch of work into getting the visualization running, and it worked mostly and was decent, but I didn't put much work into providing variation on the effects, I just let them go as is for hours. But the live video input was broken, mplayer on a pvrusb2 would work for a few minutes and then lock up so hard a kill -9 wouldn't even kill it. And the audio would work after it was futzed with but then stop upon trying it into a new graph.

http://www.flickr.com/photos/foist/2158862880/
http://www.flickr.com/photos/foist/2158071625/

Harrier from binarymillenium on Vimeo.

Gephex graph examples

2007-12-29T07:50:00.000-08:00

I used to just keep my graphs to myself and only show videos of results occasionally, but from now on I'll be backing up all the good ones online suitable for distribution under the GPL.

Here's a couple:

http://code.google.com/p/binarymillenium/wiki/GephexGraphs

This one is cool:

galaxy from binarymillenium on Vimeo.

This is okay:

BW Plasm demo from binarymillenium on Vimeo.

Dorkbot show

2007-12-16T09:30:00.000-08:00

This went pretty well, it was low-key- the tent I was in was something of a place to relax although some of the DJs played more danceable music, and half a dozen or so people might be dancing at any time. I showed off Gephex to anyone who was curious, and would leave effects running unattended for long lengths of time. Not nearly the same level attention or effort from me as my last show, but good enough for the venue.

I feel like I've done everything I want to do in Gephex, I have to start using it differently or try different software, but there isn't anything free I've found that and as powerful that is suitable for live editing of effects.

Things that would help:
Make the joystick input support all buttons on the logitech controller- currently only two buttons and one analog joystick are supported, I'm sure adding the rest is trivial if the module is recompiled. More inputs to make effects more dynamic would allow more fundamentally different effects.

But what really needs to be done that is outside the scope of what I can do is allowing patches to be selected and reduced to subsystems that can be better organized and combined with other subsystems. There's a limit to how complex a graph can become otherwise that contributes to the 'sameyness' of everything I make with it. I think they are working on this and there may be a beta version.

Better random number generation- perlin noise specifically plus more generic IIR number filter block. I think if I connect a square wave and a random number generator to a flip-flop I can improve the randomness by only infrequently sampling the random number. The feedback option works as a very poor filter and is inadequate.

Radio effect in Audacity

2007-12-16T09:18:00.000-08:00

So recently I needed to figure out how to take some dialogue recorded in person and make it sound like it was being heard over the radio. There's no built in effect for that, but on a bulletin board someone suggested notch filtering 1000-3500 Hz as radio's do the same thing- that gets you in the ball park but it misses the distortion and noise and crackle of a radio. For that I used the 'satan maximizer' effect that came in the large set of ladspa plugins that work in Audacity. With a lot more tuning and research on the type of radio I wanted to imitate I could probably do better, but short of that it at least communicates to the viewer the intention of the effect without being perfect.

Dorkbot - Opening Night Party

2007-11-21T07:21:00.000-08:00

This will be my second non Open Lab show, this year or ever. I need to get to work making a bunch of effects, hopefully mostly new ones that don't all look the same...

Also, a new video from the October Open Lab:

Open Lab - October 2007 from binarymillenium on Vimeo.

Converting vob to avis

2007-10-30T22:27:00.000-07:00

So I have a few vobs from a dvdrecorder that was recording a live set of video from an Open Lab. The vobs must be a little screwy- any editing program I try to load them into thinks they're only 18 seconds long. VLC knows how to play them in their entirety, but I really only want to get a few good moments out of it all.

I think transcode is up to the task, with command like:
transcode -i vts_01_1.vob -y dv -o openlab1.avi

But the video comes out screwy (while the audio is fine).

I google for the answer and got this gem of a thread as the first hit:
http://linux.derkeiler.com/Newsgroups/comp.os.linux.misc/2004-02/2245.html
where one poster repeatedly ask for details beyond 'go look on google' or 'read the manpage'. I think anyone who responds with 'go look on google', when that thread then for some stupid reason uselessly becomes the number one search result on google, deserves to be summarily shot. I'm only half kidding. Anyway the original poster responds with queries for details (like a command line) but repeatedly gets the same useless answers, but more vehemently.

I haven't figured it out yet, but I'll post an actual command line when I do.

Connecting to a Server in GWT

2007-10-26T06:32:00.000-07:00

A while back I started playing with GWT and created a small game GWTTiles. Fine example but there's no client-server interaction, so I tried that out a few days ago. The DynaTables example seemed way too complicated so I looked on the web, and got fairly far but couldn't get the server to work- but then finally a comment in a tutorial pointed out I needed to compile the java server myself, GWT wasn't going to compile it for me- though once placed in the proper directory it would happily run the .class file automatically.

So I didn't even have the java jdk installed, that was necessary to get javac:

javac -cp "$APPDIR/bin:$APPDIR/src:/home/bm/other/gwt-linux-1.4.60/gwt-user.jar:/home/bm/other/gwt-linux-1.4.60/gwt-dev-linux.jar" src/com/binarymillenium/gwt/server/serviceImpl.java

Then the class file had to be placed in bin/com/binarymillenium/gwt/server, and running the -shell script will properly execute it and my client app can get data from the server (just a text string for now, I need to figure out serializing next).

This is the error I used to get:

"com.google.gwt.user.client.rpc.InvocationException: Unable to find/load mapped servlet class 'com.binarymillenium.gwt.server.serviceImpl'"

And in the development shell window:

[ERROR] Unable to instantiate 'com.binarymillenium.gwt.server.serviceImpl'
java.lang.ClassNotFoundException: com.binarymillenium.gwt.server.serviceImpl
...

The Kingdom

2007-09-21T19:56:00.001-07:00

Generating Video

2007-09-01T10:42:00.000-07:00

There have been a lot of Arduino video generation projects covered by the Make blog recently, I did my own a couple of weeks ago:

Arduino Video - NTSC from binarymillenium and Vimeo.

Since then I've experimented with upping the resolution by a lot, though memory becomes an issue quickly. Ideally I could have a whole set of different visuals to generate with an arduino and few knobs and buttons to control them with for a cheap VJ device.

Color is out of the question with an Arduino without hardware components to generate much higher frequencies than can be made in software with the standard 20MHz clock.

So next I'm thinking of trying color video generation with an FPGA. B&W generation on an FPGA seems trivial compared to doing it in software, and many grades of gray could be generated with PWM rather than having to add more resistors and use more digital outputs.

But can a 3.58MHz color signal be put on top of that successfully?

I have a Digilent Spartan 3 dev-kit that's a few years old, it has a 50 MHz oscillator. I could generate 50/14 = 3.571 Mhz clock from that, each period would be 14 cycles long. So that means only 13 different phase shifts from that would be possible, and I'm not sure if a full 360 of phase shifts is allowed- it might only be 1/2 or 1/4 of that. And then there's the problem of PWMing those to produce different amplitudes- for one half period there are 7 cycles. A normal one would have PWM like 1010101, then a 1100110 (or maybe 1100011), and then 1101101 or 1110111 or something else but it's not clear any filtering I would do or the tv input does would actually create a readable 3.571 MHz signal out of that. It's worth a shot anyhow.

There are a few color video FPGA projects out there, one on opencores, I'll look into those if my efforts aren't that successful.

Once I can generate a test pattern, I'll want to synthesize a cpu and program it with a game or VJ stuff to generate interesting moving visuals. I don't think I can make a microblaze with the free Xilinx Webpack tools, but maybe something like OpenFire or PicoBlaze?

WinTV USB with Ubuntu 6.04

2007-06-29T18:52:00.000-07:00

I bought a usb TV tuner/s-video/rca input device for my laptop, so that I can get video from DV video cameras and any other DV source straight into Gephex. Installation in Linux wasn't too hard:

First get usbvision:

cvs -z3 -d:pserver:anonymous@usbvision.cvs.sourceforge.net:/cvsroot/usbvision co -P usbvision

make; sudo make install

Tv time doesn't work, but zapping does:

tvtime:
"Your capture card driver, USBVision USB Video, is not providing
enough buffers for tvtime to process the video."

zapping:
works!

Gephex doesn't know how to change the channels or settings, so the trick is to change things in zapping and then quit and the settings will persist in gephex.

Opticlash 2

2007-06-14T19:25:00.000-07:00

This was an interesting event, I didn't go to the first one in 2005 and I had my doubts about the format going in- and I still have my doubts, but overall it was a success in terms of promoting VJing- and also VJ Scobot won, and I think he did do the best VJing there.

There were three sets of screens at the front of the room, the center had camera views of the two competing VJs and the outer ones had the video they were outputting. This seemed less than ideal because of difficulty in viewing both simultaneously- you could either watch one or the other except from the most distant points or oblique angles.

It's possible the VJs at some points were glimpsing what the other guy was doing and responded in some way, but I think mainly they were just concentrating on their own stuff- which is unfortunate, because the most crowd-pleasing aspect of a competition is any kind of interaction and drama that can be generated between the contestants. It would be great to have one contestant go for a minute or two and the the other goes, trying to outdo the other, maybe playing a similar sort of clip, or mocking them somehow, or anything like that- and they would go back and forth a few times.

Opticlash 2 from binarymillenium on Vimeo

I didn't actually see the last set, since the show was going on a bit longer than advertised- the 15 minute sets for the later rounds should have been at most 10 minutes on schedule or not.

The judges didn't add a lot besides the rote rating judgements they offered, the MC initially wanted some kind of vocal rationale or something out of the judges but they were microphone shy.

Here's a writeup from one of the judges, and he posted some video (Pixelflip vs. ?) on dailymotion.

VJing Toorcon

2007-06-11T19:02:00.000-07:00

Post-mortem thingy:

My setup was to use gephex on a Linux laptop and have a video camera and dvd player as video sources. Switching between them would have been nicer with a switch box, I was just reconnecting cables while having a graph loaded on gephex that wasn't dependent on external video input.

I had a couple of newly burned DVDs, one was lots of video game imagery. Biohazard Battle on the Sega Genesis worked decently, but old Atari games with flatter coloring and graphics could have been more interesting. Unfortunately my Atari 2600 didn't work after I pulled it out of storage. One of those $20 battery powered video game joysticks would work well for sourcing the imagery live, but I'd need someone else to play (I recall somebody brought one to Open Lab a few months ago but didn't try it out).

The other dvd had videos of a computer animation contest, one where the executable creating the animation had to be less than 32K. I had to run the dvd player output through a anything-to-anything box from Canopus with firewire in/out on it.

Four projectors were going with cloned video, and I had an LCD monitor to see what I was doing. None of the projection surfaces in the main room were that ideal but they worked. The better surfaces were out in the hall but I couldn't see them.

Gephex (or underlying Linux graphics software) is finicky about driving full-screen, sometimes there would be a sliver of the desktop underneath it. Playing around with it usually got rid of it. I spent a long time going through all my graphs and deleting ones that didn't work and adjusting output settings for ones to keep. It would be nice if each individual graph didn't have it's own output settings.

Kino plus my screengrab frei0r module was my way of getting 1394 video into gephex. I could probably write my own 1394 gephex/frei0r input module using source from Kino but I haven't gotten around to it yet.

Ideally there would be a way to get video into linux so that it is seen as a webcam- a video-to-usb device sounds ideal- I could get video from a video camera or a dvd player without using firewire at all. Supposedly there is one from X10 with linux drivers, and another from Hauppage, but neither is sold on Amazon or other sites that feel respectable enough to purchase from.

Toorcon 2007 from binarymillenium on Vimeo

Another option would be to run Gephex on the windows side of my laptop, but unfortunately it came with Vista preinstalled, and gephex doesn't run so well there.

I would have liked to be able to record the whole show or portions of it. Another RGB-to-video plus another video camera with video input recording (like my old Canon DV camera) would have worked, but added a lot of wires. Also I didn't have a spare VGA splitter output to drive it.

Europe Has A Mission

2007-05-28T19:00:00.000-07:00

There was an add for this movie in The Stranger. My first thought, naturally, was WTF? Then I wondered if it was 'Command & Conquer: Generals: The Movie', and then I looked it up on IMDB (it wasn't there) and then on google and found out why: it doesn't exist. It's a joke/commentary/guerilla-art/etc.

Amazon S3 for media backup

2007-05-01T07:21:00.000-07:00

I've been concerned about backing up my videos and pictures for as long as I've been taking them, but until a week ago I never bothered to think much about a commercial service. I would periodically burn DVDs, multiple DVDs of the same data- worried that the DVDs would degrade after a few years. I'd also make hard drive backups and then store the hard drives and some of the dvds offsite.

DVDs are poor for backup as soon as a platter is full of them- it takes a lot of time to find specific files later on. Hard drives are better, but expensive. Recently I had two hard drives fail in a row- they still sort of work, I can recover data off them, but they don't work as boot drives and were at times making clunking noises.

Another feature I've been in need of is a way of transferring videos to people, or hosting high quality videos. Youtube/vimeo/google-video/etc. is low quality and I don't necessarily want to give the videos to the entire world.

I'd heard about Amazon's S3 service a while back, but it doesn't have a friendly user interface by default- and I wasn't sure about paying for third party software like jungledrive, and I didn't want to pay another service like that something-else-drive. I've bought things from Amazon before, they have my credit card information already, and I like their pricing scheme. Other services want you to pay an excess- $10 a month for 100 GB, great deal if you fill up that 100 GB immediately, but poor if you only use 10 GB at first.

Then I found S3 firefox plugin. I can't vouch for the security of this plugin, but it seems to work and the media I'm backing up isn't that security critical. The plugin allows simple uploading and downloading and access control setting of files.

Over a cable modem, it took me about 3-days of uploading to get 10 GB of pictures onto S3. This is slow, but once they're uploaded the rate at which I take new pictures is much lower than backing up several years of pictures all at once.

At they're current pricing this 10 GB will cost $1 .50 a month, and I anticipate they'll lower the price per GB as storage prices fall. In the next week I'll probably have 20 GB up, so $3 a month or $36 a year. If I was just storing pictures this would start to compare poorly with flickr or other photo sites, which charge a flat yearly rate for 'unlimited' storage- but I like the raw interface for uploading and the truly unlimited feeling of S3, and I also don't plan on having too many GB of just pictures.

Video is the real hard-drive killer, and unfortunately S3 isn't going to be a good solution for raw off the camera DV or HDV or other kinds of video. The files are too big: S3 file limit size is 5 GB also, though I can chop up some of the bigger video files, it takes too long to transfer, and it will cost too much. I could easily have a terabyte of video pretty soon. I suppose it's not all worth backing up, maybe if I keep my S3 total under 50 -100 GB I won't mind the cost too much.

For now, I'm just backing up editing compressed video- wmv files under 100 MB. I might progress to backing up less heavily compressed files later. It's easy to share the files with others, set them to global read-only and email people the link- or if the third party has an Amazon account and gets Firefox + the S3 plugin going you can specifically allow them and only them to have read or even write access.

911 Media Open Lab - April 15

2007-04-21T14:45:00.000-07:00

911 Media Arts Center Open Lab - 2007.04.15 on Vimeo

I bought a new laptop somewhat recently, and am currently dual-booting between Vista & Ubuntu. Most of my custom software was only set up to run in Ubuntu, but I forgot to figure out getting the S-Video output to work when I brought it to Open Lab- despite all the other ease-of-use advances of Ubuntu something as simple as configuring an external output is still a pain. So instead I went back into Vista and just messed around with Wings3D, while others manipulated that source video with some hardware video mixers.

The slower bits are edited out, and overall I like the most of the scenes in there- even if they don't match up to the (also live generated) music, I think there's a few underlying ideas that could be developed into more interesting clips:
-Flying over alien landscapes, manipulating them, and a kind of 80s CG flat shading look
-Simple shapes that generate fractals

Wings isn't really meant for live performance. With a little more work I could set up a lot more keyboard shortcuts so the context menus don't show up as much. A more intensive effort would be to make models in Wings, export them to objs or something and a have a custom app running on the external monitor that can be triggered to load the model.

Bones Animation - Re-Acting

2007-04-09T19:26:00.000-07:00

Bones - Re-Acting from binarymillenium on Vimeo.

I think I could have done a better job with this video, edited it a little more heavy, but I don't like to get to bogged down with it. I sort of think as these as visual notes to myself, I can refer back to them and recreate the effect I captured in a bigger and more meaningful work.

The main thing making editing more difficult was that I was using image sequences in Premiere- my computer isn't fastest enough to actual play back unrendered image sequences (and I was too lazy to render it), so it was hard to get the edits and feel right.

The source imagery is from a code.google project called 'bones' http://code.google.com/p/binarymillenium/wiki/Bones. It's a very simplistic bones animation implementation, using osg::nodes and with randomly generated hierarchy and animation. Every vertex in the object loaded for a bone has a weight that mixes (using quaternion slerp) the positions of the parent osg::node and the child. The weights are automatically generated based on the distance from the vertex from the root of the object where it joins with the parent.

VJ/Visualization Blogs

2007-03-18T09:29:00.000-07:00

I was randomly searching for something the other day and came across a lot of interesting blogs:

Create Digital Motion
This one has a commercial look, and covers a lot of gear too expensive for the casual reader, and in general focuses on gear and other people's work. But it show me Processing. Processing has a horrible name for googleability, which used to be addressed by calling it Proce55ing. Idiotically they've assumed that because the google search for just processing by itself points to the proper website, they can stop using the distinctive 55 in the name. But what happens when you try combined searches for processing and some other search term? You get tons of site that use processing as a common English word.

flight404
Personal blog covering projects by the author of the site. He's done some really cool things with processing and also griffin powermate knobs.

Processing Blogs
Haven't really read this that much, but it looks interesting.

VLOBLIVE
It's a clunky word for 'Very Low Budget Live video for Events'. It's not that applicable to much that I do, and their definition of 'very low budget' is still a lot of money to me. They use the other terms like IMAG, for image magnification, which very mundanely just refers to show a big closeup of a performer on a stage on a big screen. The blog is pretty practical, not about cool art but just has a lot of helpful stuff.

There was another site linked to from Create Digital Motion but I don't have a link to it in front of me.

I've been playing with Processing, and it is very fast and easy to use. The support for microphone and video input doesn't seem as robust as in gephex, but the flexibility for making effects and taking data from other sources seems high. It's wierd, because a lot of the stuff to do is the sort of thing I'd do already in C++ and OSG rather than Java here- but I can understand the appeal to people who don't want to spend hours compiling the latest OSG sources, and there's few button and mouse clicks saved by just typing in code and hitting run. But I feel like I should learn it anyway even though its 3D capabilities are a step back from OSG, if nothing else because there's a higher likelihood of being able to collaborate with other people into processing.

Vimeo

2007-03-17T13:07:00.000-07:00

It's easy to spread yourself too thin by trying to maintain an online presence on a lot of similar community sites for hosting content. They all have different designs and by virtue of their architecture or just the vibe the creators and the initial group of users create they seem to be good for different things. But I'm trying out vimeo because of their focus on user created content- there's a lot of artists in there, and it sort of has a flickr feel to it- and the video quality is pretty good, the UI is nice that it disappears when not in use (though it obscures video when in use). I'm just hosting files identical to ones I have on myspace, google video, and youtube, but whatever site feels the most productive I'll probably eventually use to the exclusion of the rest. When someone starts a free 640x480 or HD hosting site I'll probably re-evaluate...

Youtube is the most popular, but video quality is garbage and most of the content is garbage. I imagine the user base is on the more youthful side.

Google video is better quality, though has no real community feel- it's just a faceless generic place to host video. Some day I might be interested in trying to make money off of uploaded video that I own entirely but for now that idea is mostly incompatible with my approach to this sort of thing (a tip jar feature with proceeds donated to a charity of my choosing would be nice though).

Myspace is not primarily a video sharing site, but they have a decent video uploading feature. If you have a myspace page and make video you might as well use their hosting instead of embedding someone else's. It can feel a lot more personal even if your page is not your personal page because of the emphasis on friending.

There's a ton of other hosting sites I won't bother with, mostly since their user base is marginal or they are just a dumping ground for copyrighted material, racier stuff if their content policy is more liberal than youtube or google's. Others are completely based around making money for uploaded content, pay per view basically: get thousands of views and make $10 or so. That sort of goal based emphasis probably marginalizes a lot of otherwise interesting content, encourages copy-cats and therefore homogeneity. So does having a view-counter or ratings (read slashdot for a few months to see how their 'karma' promotes pointless regurgitation...), but money is a force-multiplier.

Google project hosting

2007-02-23T07:55:00.000-08:00

I was getting tired of waiting for shell service to come back to sourceforget so I searched for blogs talking about it, maybe they knew something I didn't- instead I found this entry http://icolor2.blogspot.com/2007/02/sourceforge-downtime.html which pointed me to google project hosting.

Almost immediately I created my own project http://code.google.com/p/binarymillenium/ and started uploading code. Initially I thought there was no support for screenshots but I then realized screenshots could be uploaded to svn and linked to from the wiki, although it automatically creates image for 'jpg' extensions but not 'JPG'.

I'm ready to quit sourceforge entirely, and leave all its frustrating user interface problems behind. It's one of those facets of the internet where big sites start up and then become sort of fossilized in whatever web technology was the state of the art or popular at the time, and then a new generation will rise up and replace them with superior web stuff- google web toolkit is the current new thing.

Render-to-texture feedback with OSG

2007-02-21T22:13:00.000-08:00

This actually doesn't use any of the screen capture code from the last post ( I'm still working out how to best use that) - it just does a good old fashioned glReadPixels to current the current opengl screen.

OSG Feedback from binarymillenium on Vimeo.

Screen Capture with wxWindows

2007-02-19T06:55:00.000-08:00

For reasons that will become clear when I post some more software and video, I need to have a general method of capturing any part of the screen and using it as a texture in custom applications.

Google searches for 'screen capture' 'capture screen' or 'screen grab' turn up a lot of other peoples shareware screen grab stuff, and lots of mailing lists where screen captures are provided of one thing or another but it's hardly ever about the code that did the work. How to find some usable code to do what I needed? The answer turned out to be sourceforge, where I found sseditor:

wxWin32ScreenShot.cpp

I was hoping there would have been a platform independent method where you just ask the video card to give you all the pixels, but it turns out to be windowing platform specific. This instance used Windows and wxWindows. I compiled wxWindows 2.8.0 for Cygwin and saw that the examples work. I then copied the screengrab code out of sseditor and put it in a wxWindows sample called 'drawing'. That worked after a little tweaking.

I then took the code and put it into a custom OSG application. There were lots of problems, and I was worried the wxWindows libraries or just using wxWindows in OSG at all was causing instability. I then ran into a problem where calling 'CreateCompatibleBitmap' failed after exactly 153 calls, but it turned out I needed to call DeleteObject on the HBITMAP before calling CreateCompatible repeatedly, I was using up graphics memory and not freeing it.

Also, I had to call wxInitialize() and wxUninitialize() at the beginning and end of my program.


void wxScreenCapture(wxDC& dc)
{
    int sizeX = 0;
    int sizeY = 0;
    sizeX = tex_width; //GetSystemMetrics(SM_CXSCREEN);
    sizeY = tex_height; //GetSystemMetrics(SM_CYSCREEN);

    compat_counter++;
  
    bmp->SetHeight(sizeY);
    bmp->SetWidth(sizeX);
   
    
    HDC mainWinDC = GetDC(GetDesktopWindow());
    HDC memDC = CreateCompatibleDC(mainWinDC);
  
    if (bitmap != NULL) DeleteObject(bitmap);

    bitmap = CreateCompatibleBitmap(mainWinDC,tex_width,tex_height);

    if (bitmap == NULL) {
        std::cerr << "CreateCompatibleBitmap failed at " << 
                    compat_counter << ", " <<
                    mainWinDC << " " << tex_width << " " << 
                    tex_height << std::endl;
        exit(1);
        return;
    }
    
    HGDIOBJ hOld = SelectObject(memDC,bitmap);
    BitBlt(memDC, 0, 0,sizeX,sizeY, mainWinDC, 20, 20, SRCCOPY);
    SelectObject(memDC, hOld);
    DeleteDC(memDC);
    ReleaseDC(GetDesktopWindow(), mainWinDC);
    bmp->SetHBITMAP((WXHBITMAP)bitmap);
    if (bmp->Ok() ) {
        //dc.DrawText( _T("BMP  ok"), 30, 20 );
        
    } else {
        //dc.DrawText( _T("BMP not ok"), 30, 20 );
        std::cerr << "bmp not ok" << std::endl;
        return;
   }

    if (savewximage) {
        bmp->SaveFile( wxT("/cygdrive/b/text.bmp"), wxBITMAP_TYPE_BMP);
    }

My other problem was that I didn't know how to translate between the wxBitmap format of wxWindows and the texture format of the osg::Image. The first way I made it work was by writing the bitmap to disk using a wxWindows function then loading that bmp to a texturing in OSG- this worked but was hard on the disk drive for high rate screen capturing, but writing to a ramdisk can fix that.

But the seeming right way to do it:


/// get image from desktop in wxBitmap format, 
// convert it to osg::Image format
wxAlphaPixelData rawbmp(*bmp, wxPoint(0,0), 
                         wxSize(tex_width, tex_height)); 
wxAlphaPixelData::Iterator p(rawbmp);               

image->allocateImage(tex_width, tex_height, 
                     1, GL_RGBA, GL_FLOAT);

/// image is an osg::Image
float* img_data = (float*)image->data();
for (unsigned i = 0; (i < tex_height); i++) {
for (unsigned j = 0; (j < tex_width); j++) {

    int ind = i*tex_width + j;

    bool flip = ((i > tex_height/4-1) && (i < 3*tex_height/4) 
                && (j > tex_width/4-1)  && (j < 3*tex_width/4));

    img_data[ind*4]   = flip ? 1.0 - p.Red()/255.0 : p.Red()/255.0;

    img_data[ind*4+1] = flip ? 1.0 - p.Green()/255.0 : p.Green()/255.0;

    img_data[ind*4+2] = flip ? 1.0 - p.Blue()/255.0 : p.Blue()/255.0;

    img_data[ind*4+3] = 1.0;

    p.MoveTo(rawbmp, j, tex_height-1-i);
}
}

The flip code seems really wierd, but it was necessary.

Anyway, after all that I felt pretty proud of myself for hacking a working screencap together over a couple of days, not knowing anything about windowing toolkits before that (I've always known enough about them to try and avoid them like the plague, there's no uglier code than window gui widget code).

The only libraries needed for wxWindows are -lwx_base-2.8 and -lwx_msw_core-2.8, though there will be warnings about other wx libs getting auto imported.

VJ Night

2007-01-24T08:03:00.000-08:00

There might have been about 20 people there to watch the show. Around 8 VJ Scobot started out with his opening set, which was mostly fast edits between a few dozen clips. Some of the clips were recognizable, like from Bad Boys or Team America, others from anime sources, and some footage from marginal productions of decades ago. Something from early incarnations of 'The Rocketeer'? Scobot later explained that his software allows keying of clips to keys (and presumably effects and transitions also?).

After 20-30 minutes of that the music wound down and Scobot introduced DJ Deeb, the guest VJ Spyscience, and what VJ Night is all about. An interview with Spyscience followed, video taped by one from the 911 crew (though it's not clear whether the interview will ever be put on the internet or anywhere else, it's just for the archive). Spyscience used some software I forget the name of, which is also clip oriented but drag and drop along with an external USB device with sliders and knobs is used. Most of the clips where provided with the software, one he had made himself. His computer locked up a few of times and Scobot had to take control of the video for a while during rebooting.

After that a Q&A sessions followed, and the event was over by about 10.

Overall the event was interesting, though even as Scobot admitted the traditional role of VJ work is to make a side-show, not to be shown in a theater to an audience focused directly on it. I thought the format works but that VJ sets should be a little shorter, since they can get repetitious. Another nice thing might be to have continuous split-screen, one view showing the VJ output and the other focusing on the VJ and what they're doing, although the layout of the space allows the audience to effectively look over the shoulder of the working VJ.

I found my personal preference is for visuals that are the opposite of recognizable clips from tv or movies, but purely abstract instead. I'd like to see someone do a set with more live-generated visuals (rather than live-editing-plus-effects on a lot of clips).

Gephex

2007-01-10T07:38:00.000-08:00

I created a couple of custom gephex modules (for 0.4.3):

Average

Find the average color or brightness of a framebuffer. (maybe add HSV next)

Slow motion

Play back snippets of framebuffer input in slow motion. The music is from ccmixter, William Berry's Time To Take Out The Trash.

The source code (GPLed of course) and windows dlls are provided in those zip files.
More custom effects are on the way.

New video showing off basic gephex effects:

...but none of those I just created. Maybe next time.

Phoeeb Deform

2006-12-23T17:45:00.000-08:00

Brand new video, made almost from scratch (a stock photograph from deviantart was used as a referenced and applied as a texture).

Phoeeb Deform from binarymillenium on Vimeo.

The music is from ccMixter (http://ccmixter.org/media/files/vincent_vega11/6966), which has supplied me with several good soundtracks for recent videos, although it can be difficult to browse the music- where's the sort by rating button? I recall another CC music site less focused on remixing and more on original works that had more movie like soundtracks I'd like to use in the future but I forget the site.

Deriving The Rocket Equation

2006-11-29T13:19:00.000-08:00

This is probably a job interview question somewhere. I was just thinking about it
and decided to try it out myself, with no regard to convention beyond what is already intuitive for me.

The question is, you have rocket with a certain amount of mass, some of which is fuel. The rocket consumes fuel at a constant rate in order to produce a constant thrust. If the rocket starts at time zero, how fast is it going and what is its displacement at some later time?

So the lets say the fuel consumption rate is a positive number k, in units of kg/sec.
The fueled mass of the rocket is Mo.
Therefore the mass at a later time t is
M(t) = Mo - kt

From physics it is known that F=MA. Our F is constant, we'll just call it F, its units are kg*m/s^2. Acceleration is the thing that will change, intuitively acceleration should increase as the mass goes down as fuel is consume.

A= F/M = F/M(t) = F/(Mo - kt)

Looking at that equation makes sense- the denominator gets smaller therefore A goes up as time increases.

So the trick to finding velocity is integrating that curve and position is found by integrating again. Easy numerically, but of course we want a nice analytical answer.

I've forgotten a lot of the little tricks of calculus, which is just as well since I want to solve this as intuitively as possible anyway. The most I did lookup to remind myself was to find that the integral of 1/x is ln(x) and that ln(x/y) = ln(x) - ln(y), and that to have definite integral over a specified interval (time 0 to time t1 here) you can subtract the indefinite integral of the lower bound from the indefinite integral of the higher bound.

We don't have a 1/x equation, at first glance, we have a 1/(constant- x) equation- so how do you integrate that? The trick is to make our equation into the form of 1/x.

1/x backwards to our acceleration graph:

So we need to reverse our graph to make it look like that and then we'll have the integral. Instead integrating with respect to time, we'll integrate with respect to (Mo-kt) and do it from the lower bound Mo-ktf (tf is the final t) to Mo (Mo - k to, to is zero so it's just Mo).

So we end up with

Integral of F/(Mo-kt) d(Mo-kt) = F * Integral of 1/(Mo-kt) dkt
(since Mo does not change the interval d(Mo-kt), it drops out). Unfortunately the units of this integral is going to be kg*m/s^2 * 1/kg * kg/s * s (the units of the F times the units of the 1/ mass times the units of the k times the units of the t) = kg*m /s^2 - we want velocity not force! In order to cancel out the k in dkt inside the integral we divide F by k outside of the integral.

f/k units is m/s - a velocity which we should return to later.

f/k * ( ln(Mo) - ln (Mo - ktf) ) = f/k * ( ln (Mo/(Mo - ktf)) )

Or if we say Mf = Mo - ktf, then v(t) = f/k * ln(Mo/M(t)).

Exhaust Velocity

Going slight further in, we should look at what is happening in order to accelerate the rocket. Rockets operate by throwing mass out the back end at a high velocity- throwing out a little bit of mass a high velocity adds a little velocity to your rocket with a big mass- and there's an equation of momentum to show
how those four things are related:

m1 * deltaV1 = m2 * deltaV2

We know we're burning m1 = k amount of fuel every second, and that since we're not doing any nuclear reactions the k amount of fuel is still there in mass form- it's just moving out the back end at high velocity. We know the mass of the rocket Mo or M(t) and its acceleration (delta-v) for that moment is a(t). So we have three of the four quantities, the delta-v of the exhaust fuel is what we want. Since the exhaust velocity doesn' t change we can just use Mo = M(0) and a(0)*1 second = F/Mo*1second (to simplify we assume that acceleration doesn't change much over 1 second).

m2*deltaV2 / m1 = deltaV1
Mo*F/Mo* 1sec / ( k * 1sec ) = deltaV1
exhaust velocity deltaV1 = F/k - the same f/k in the above rocket equation. So instead of saying f/k we can say 'exhaust velocity'.

2006 Fremont Solstice Parade Video

2006-11-15T11:36:00.000-08:00

2006 Fremont Solstice Parade highlights, held in Seattle.

Gira Sol, Vamola, the Billion Belly March, and many other groups I haven't identified are featured in the video. (What is the group in the pyramid, or the people in the furry green outfits with drums?)

Gravity

2006-11-05T08:56:00.000-08:00

Gravity from binarymillenium on Vimeo.

Googlepages Test

2006-10-06T09:55:00.000-07:00

I'd like the columns to take up more than the current width, but I don't think it's possible.

Embedding video is a little funky, when I went to three columns it had problems putting in the video in anything but the center 'main content column'. If I can fatten up the single main column I can use the edit html to make my own columns- ideally there would be videos in a grid 3 videos wide, or at least two wide.

Jason Webley - Fremont Solstice Pageant

2006-08-06T19:13:00.000-07:00

After the parade, there's the pageant at Gasworks park. Jason Webley played a set:

VamoLá band warmup

2006-07-28T19:32:00.000-07:00

At the staging area of the Fremont Solstice Parade.

VamoLá Drum & Dance Ensemble

The quality of this one isn't so great because there are many people and details and wider angles in each shot. A solitary subject against a plain background works much better on google video.

parse error at end of input

2006-07-08T22:36:00.000-07:00

I got that error out of gcc 3.3.4 when I forgot a parentheses:


    for (unsigned i = 0; i < troops.size(); i++) {
        if (troops[i]->active) { troops[i]->update(timestep); 
    }

It gives the last line in the file as the line number, so it's not an easy one to track down.

DVD Chapter markers in FCP + iDVD

2006-07-05T07:48:00.000-07:00

I finally learned how to create dvd chapters in Final Cut Pro and iDVD (at least the 2006 version). Pressing the 'm' key, or the '`' key, or going up to the marker menu will add a mark, but you also have to make sure it adds a designation (anyway to do that by default?)- the easiest thing to do is press the '`' key and then the dialog that has a 'add chapter' button is right there, otherwise it takes additional steps.

iDVD, upon importing the exported video (and I think it can be exported as nearly anything in the quicktime export option and the proper markers will survive) will create a 'play movie' and also have a button for selecting chapters from a auto-generated screen full of chapters. You can then change what picture or clip of video is shown for that submenu, but it's not frame-accurate- trying to scroll with the mouse (especially for a longer video) or pressing the left and right arrow keys usually jumps many frames forward or back.

There's also the 'best quality' vs. 'best performance' option, I couldn't fit a 70 minute video with 'best performance' enabled so I had to change it from that default. I probably wanted 'best quality' all along anyway but I didn't know about it until it complained about the length of the video under the old option.

DVD Studio Pro

I have tried making a video with DVD studio pro, mainly to be able to create chapters before I knew the above steps, and also because an earlier version of iDVD was not exporting the 16:9 video correctly- it would look normal widescreen for five minutes and then it squeezed the video a little so there would be black vertical bars on each side. Strange...

DVDSP isn't that hard to use, but it has a major failing for me: it can only use HD video if a HD dvd is being made. iDVD will automatically recode most any video thrown at it, but here an additional very time consuming step is added, one has to export the video from FCP in a SD format ( I chose DV50), the 16:9 aspect ratio won't survive the export but in DVDSP there's an option to force the video back to 16:9. But it's a huge pain to have to do that extra export.

BPDC at Folklife Video

2006-06-27T18:56:00.000-07:00

Breaking Point Dance Company at Seattle's 2006 Folklife.

Event Flyer for 'Boombox' show on Friday 6/30.

Fremont Solstice Parade

2006-06-18T21:47:00.000-07:00

I just uploaded some of my shots of the pre-parade setup area, more will be forthcoming as I find the time- I've got a few of parts of the parade and the pageant at Gasworks. If someone were to sponsor me and buy me a pro account that might speed things up...

On the evening of the 17th there were already hundreds of pictures uploaded to flickr of an event that occurred at noon the same day, and probably plenty more elsewhere on the net. As digital cameras get more inexpensive and the parade become more popular the numbers of pictures made will go up in proportion. Though if you look at the number of people with cameras, the number of them that do upload are a very small proportion- perhaps a couple of orders of magnitude difference or more- but it doesn't matter because there are so many people with cameras.

Is there a point where it becomes completely redundant to upload pictures, or take them at all, because if you want a picture of something you can just go online a day later and find someone else's? And if they have a better camera, a better lens, a better eye for composition- and you feel like an ass because not only are you taking a picture but also everyone next to you also has a camera raised, what's the point, what room for uniqueness and creativity? Of course, there's an infinity of small moments and angles and compositions, there's bound to be someone out there who values your creative effort despite the abundance of similar images available from others.

I also took video of the parade and pageant, which will make it onto google video eventually, but that takes a lot longer to do that upload a few pictures.

Short breakdancing video from Folklife

2006-06-09T23:20:00.000-07:00

These kids were just messing around, there wasn't even any music playing.

Still working on getting something together from the BPDC show...

Mestre Curisco Capoeira Workshop

2006-06-08T20:19:00.000-07:00

This was at the Seattle Center, Folklife 2006. There was a probably much more impressive capoeira show later on this same evening but I was taping the Breaking Point Dance Company at a conflicting time.

The polished wood floor looks great, though most of the effect is lost due to the compression.

Later on this year hopefully I'll get in touch with a local capoeira group and make something much more involved than this.

osgClouds

2006-06-08T06:59:00.000-07:00

Modified osgforest example to produce this simple cloud layer in osg:

With this code (provided here under the GPL):


void ForestTechniqueManager::createTreeList(osg::Node* terrain,const osg::Vec3& origin, const osg::Vec3& size,unsigned int numTreesToCreate,TreeList& trees)
{

    float max_TreeHeight = 15*sqrtf(size.length2()/(float)numTreesToCreate);
    float max_TreeWidth = max_TreeHeight;
    
    float min_TreeHeight = max_TreeHeight*0.3f;
    float min_TreeWidth = min_TreeHeight;

    trees.reserve(trees.size()+numTreesToCreate);

float ceil = origin.z()+900.0f;
float floor = origin.z()+300.0f;

    for(unsigned int i=0;i    {

        Tree* tree = new Tree;
        tree->_position.set(
random(origin.x(),origin.x()+size.x()),
random(origin.y(),origin.y()+size.y()),
random(floor,ceil)
);
        
float colorFactor =  (tree->_position.z()-floor)/(ceil-floor);
colorFactor = 0.3 + colorFactor*(1.0-0.3);
int color = (int) ((1.0 - (1.0-colorFactor)*(1.0-colorFactor) ) *255.0);


tree->_color.set(
color, color, color,
255
);


        tree->_width = random(min_TreeWidth,max_TreeWidth*(1.3f-colorFactor) );
        tree->_height = random(min_TreeHeight,max_TreeHeight*(1.3f-colorFactor) );
        tree->_type = 0;
        
        //if (terrain)
        if (0)
        {
            osgUtil::IntersectVisitor iv;
            osg::ref_ptr segDown = new osg::LineSegment;

            segDown->set(tree->_position,tree->_position+osg::Vec3(0.0f,0.0f,size.z()));
            iv.addLineSegment(segDown.get());
            
            terrain->accept(iv);

            if (iv.hits())
            {
                osgUtil::IntersectVisitor::HitList& hitList = iv.getHitList(segDown.get());
                if (!hitList.empty())
                {
                    osg::Vec3 ip = hitList.front().getWorldIntersectPoint();
                    osg::Vec3 np = hitList.front().getWorldIntersectNormal();
                    tree->_position = ip;
                }
            }
        }
        
        trees.push_back(tree);
    }
}

HDV to DVD

2006-06-04T09:08:00.000-07:00

Exporting 1080i video from FCP is not a good idea if the video is going to going to iDVD. iDVD doesn't deinterlace it itself even though it converts it down to standard definition.

I'm not sure about this, but I think it's best to make the sequence settings set to the resolution desired as well as selecting the right compression when exporting- this will ensure that the aspect ratios are correct and what you see in the output window is mostly correct.

Of course, exporting to HDV 1080i when the source is HDV 1080i is relatively fast, any conversion is not going to be speedy. While editing, it's best to have the timeline in the same settings as the source clips, otherwise FCP will try to convert it on the fly and give a poor quality RT preview or simply say 'unrendered'.

I've ran into a few glitches with this process: On one project, dropping the HDV 720p files into iDVD (from iLife '05) results in the proper anamorphic 16:9 'enhance for wide-screen tvs' type video. Previewing the video in iDVD and everything looks right- the video fills the entire 16:9 preview box. But after burning it, only the first 10 minutes or so is the proper format- although the video is still in 16:9 anamorphic, for some reason the video is squeezed horizontally by about 10%, leaving black bars on the side. This didn't happen on an edit or transition between different timelines, just right in the middle of a continuous shot. WTF?

Another problem, perhaps related to above, is that some text I thought would be on the bottom of the screen got chopped off- this I think is because the conversion from 1080i to 720p chopped some off, or that the glitch above not only squeezes the video horizontally but also stretches it vertically and then cuts off the botton.

So currently I'm googling for better processes to replace all this, one semi-promising one advises to export to DV50 at some point.