2011-11-23

Lunar DTM100 to Blender displacment map

This post is an aggregration of multiple threads created on google plus and additional findings, the scattered nature of those threads made it impossible to find all the information in one place hence this post.   It's also a work in progress with some blank spots- help is welcome!

(I've mostly moved to google plus which is great for mini-blogging (as opposed to the micro blogging of twitter and full size regular blogger blogging) and has very good engagement once you find good people to put in your circles.  I expect greater plus/blogger integration in the future probably starting with comments becoming plusified.)

From LROC Lunar Map




Lunar Elevation Data


Source DTM files are in the IMG files:


http://lroc.sese.asu.edu/data/LRO-L-LROC-5-RDR-V1.0/LROLRC_2001/DATA/SDP/WAC_DTM/ 


The highest resolution maps are in the 100M.IMG files, which means 100 meters/pixel (which seems large for an object so close as the moon- why don't we have 1 meter per pixel, or 0.1 meter yet?).

I haven't written a script for handling the files that don't completely cover the lunar globe, and will probably use other tools to get that right.  (grass gis http://grass.fbk.eu/gdp/index.php?)


TBD detail on file format.






Generating 32-bit elevation tifs with Python


Conversion to viewable

Not a lot of programs can display those 32-bit tifs correctly, and Blender didn't like them.  Get a version of imagemagick with hdri enabled (./configure --enable-hdri when building it from source) so they can be converted to friendlier formats.

Making an easier to view jpg from the 32-bit tifs:


convert -define quantum:scale=255.0 -normalize moon.tif moon_fromtif.jpg



And the following is the result:
From LROC Lunar Map



The jpeg is usable in Blender but doesn't hold up well with a lot of zooming- the 255 levels of elevation possible in a jpeg produce stair step artifacts:
From LROC Lunar Map

Conversion to blender usable

So Blender could use a 16-bit format like openexr for 255x smoother gradations, use this conversion:




convert -define quantum:scale=255.0 moon.tif moon.exr


The quantum scale there seems like it ought to be 65535.0 but the 255.0 works, and imagemagick identify -verbose shows that the range of values is 65535.0.

Setting up image based displacement + bump in Blender 2.6x Cycles (latest svn)


TBD flesh this out in greater detail


Add | Mesh | UV sphere


Object Modifies | Add Modifier | Subdivision Surface | Render 6

Material | Surface | Use Nodes


Shift-A | Texture | Image Texture | Open moon.exr


Connect the color to the diffuse bsdf, then should see texture on sphere if in texture or render view mode (maybe have to do something to force redraw/update).

Edit Mode | Select all edges


Mesh | UV Unwrap | Sphere projection | Align to object


Connect image texture to color to bw converter and then to displacement input on material output.  TBD proper height scaling of craters.









Object Data | Displacement | Method | Both


Link to .blend file:

http://goo.gl/ZrPJ8

From LROC Lunar Map

UV Sphere Projection Polar Problems


The UV spheres generated by blender have the problem of having triangles instead of quads around the poles.  The spherical projection will produce distortion at the poles, smoothing the sphere prior to uv unwrap helps minimize it.  I wonder if there is something problematic about making the pole polygons quads because it would involve multiple polygon points and edges right on top of each other.

You can see the problem areas in the uv image below- the nice quad projections become distorted triangles at the top and bottom.


The result is these pinched areas:





Lunar Visual Mosaics


The moon isn't perfectly grey, there are many interesting light and dark features.  I haven't located a good texture generated from LROC or Clementine data (LROC would be ideal since it would probably guarantee all visual features line up with elevation features).

http://wms.lroc.asu.edu/lroc/wac_mosaic

http://ser.sese.asu.edu/MOON/clem_color.html

There are some random ones to be found on the web but I haven't tried them yet.


Future


Grass gis http://grass.fbk.eu/gdp

gdal - has python bindings    (ubuntu intall python-gdal gdal-bin)  http://www.gdal.org/
Turn python generated tiff images into geotiffs


osgearth - uses geotiff output from gdal to produce lod/paged terrain databases viewable in OpenSceneGraph osgviewer.

Use 100M data (the 256P IMG files).  Parse dtm within python to do this?  Minimum is extracting width x height.



Sources

Original discussions that originated this post:






2011-01-30

Google Docs Storage

I've been playing with google docs storage for about a month. The user interface is inferior to Amazon S3 + S3Fox in every way, but for 1/5th the cost I'm willing to put up with it (though it's only 1/5th the cost if I fill up all the storage for the given price tier, I think they expect most users not to use a large fraction). AVIs and JPEGs can be dragged-and-dropped in quantity, but NEF and .ini files have to use the 'select more pictures' dialog which can only handle 10-15 files at a time (otherwise a strange character appears instead of a list of all the files). Upload speeds seem good (a megabyte every 3 seconds or so), better than I remember S3 being last time I tried it.

There are some linux filesystem programs that can mount my entire google online storage (including blog posts), but only allow uploading of the google docs formats and not any file at all like can be done with the upload dialog. Hopefully this changes.

5/8/2011 update

Folder upload is now possible (no more shift selecting the contents of a folder and having to cut and paste folder names), and it looks like subfolders are uploaded properly. But sometimes a file fails to upload properly, and using the file method it was possible to see which file failed and re-queue for upload. Now after I've uploaded a folder with 90 items and it took 20 minutes, there is an error message saying that one file failed to upload, but no way to know which one. Retry fails repeatedly. Some reports of same, though I haven't seen the other bugs mentioned.

2010-12-14

OpenGameArt

You can't help but want to make a 2d overhead or isometric game after looking at some of the art available on OpenGameArt- I'll settle for a little procedural terrain generation:



http://www.openprocessing.org/visuals/?visualID=16704 (click run if it asks about running old java)

The OGA site could use more contributors, as well as better organization to promote the best work- and of course it could be popularized by making use of the content there. I think I'll use this and some variations at the next VJ event I do- I've used recorded video from games before, but even better to generate some game-like imagery live.

Another useful feature would be for terrain tilesets to have a standard connectivity definition file- an ascii file that says which tiles connect best to which neighbors in which direction. Current that's hardcoded into the processing sketch.

2010-12-07

Gephex 0.4.3b built on 64-bit Ubuntu 10.10

I haven't found a better graph-based VJ tool than Gephex to use in Linux or Windows (but haven't been looking much either), and it's not trivial to get it running on a modern Ubuntu system- but I've worked through all the compiler messages (with the big exception of leaving out ffmpeg) and have an archive of the results available for download.

Go here for details and a link to the download:

http://code.google.com/p/binarymillenium/wiki/Gephex64BitBuild

The reconfiguring may screw it up a little, otherwise let it install to my choice of directories and then move the bin and lib etc. files as appropriate.

I'm hoping there is a new and well-supported tool out there that can take the place of gephex- movid is not intended for VJ work but may be retrofittable.

2010-08-23

First Steps in Bullet physics

I've played with ODE for several small projects, but recently discovered Bullet physics while looking through ROS (Robot Operating System) documentation. The difference in rigid body simulation between the two is not that obvious, the Bullet rigid body demos seem very impressive and perhaps speedier than ODE. But the soft body physics have no equivalent in ODE, and seem very worthy of investigation- and equally intriguing are the decomposition capabilities, where objects can be shattered into smaller pieces with a function call (it may be more difficult than that, I haven't looked at the code yet).



There is a good forum, not a ton of documentation or second-party dispersed know-how in the form of tutorials especially for the soft body physic, but the demo code is great- and beyond that it comes down to experimentation.

Soft bodies from 3d files

A long time ago I made my own soft body physics simulation code where I would load an .obj file and replace all vertex edges with spring-dampers, and also create a grid of internal springs to give the body volume. It worked okay in the best conditions but would frequently explode in others. Now in bullet there is a toolchain to take a 3d object and turn it volumetric, and then simulate much more robustly than in my amateur effort (though explosions still can occur).

The bunny.inl and cube.inl files in Demos/SoftDemo provide the first clues, they have auto-generated comments like '# Generated by tetgen -YY bunny.smesh'.

Installation of tools

Tetgen is in the Ubuntu repositories, but tetview has to be downloaded as a binary. libg2c isn't in Ubuntu 10.04 lucid lynx, I had to download prebuilt version here http://www.fluvial.ch/d/libg2c.tgz , and also libstdc++.so.5 from http://packages.debian.org/lenny/i386/libstdc++5/download (extract with dpkg-deb file.deb .).

Wings3D is a good program for generating 3d meshes, and is in Ubuntu repositories.



The stl it exports in binary and doesn't work for tetgen ("wrong number of vertices"), use meshlab (also in ubuntu repos) to save it as an stl but uncheck binary. Objects with holes in them didn't work right, tetview kind of locks up on them. Concave areas don't seem quite right either.



I used 'tetgen -p file.stl' and it outputs file.1.ele, file.1.face, file.1.node, and file.1.smesh. 'tetview file.1' will view the output. In the bunny.inl file there is a getElements() and getNodes() function, the data there corresponds to the lists in file.1.ele and file.1.node. Cut and paste the lists into the functions, then use a text editor to put the quotes and line breaks in as seen in bunny.inl.

On trying to use the inl in SoftDemo I get this error
./AppSoftBodyDemo
*** glibc detected *** ./AppSoftBodyDemo: malloc(): memory corruption: 0x090f5c38 ***
======= Backtrace: =========
/lib/tls/i686/cmov/libc.so.6(+0x6b591)[0x75f591]
/lib/tls/i686/cmov/libc.so.6(+0x6e395)[0x762395]
/lib/tls/i686/cmov/libc.so.6(__libc_malloc+0x5c)[0x763f9c]
/usr/lib/nvidia-current/libGL.so.1(+0x33e80)[0x2fae80]


Building the debug version helps out (cmake -DCMAKE_BUILD_TYPE=debug), but I haven't been able to figure it out yet.

Here is the backtrace:

Program received signal SIGABRT, Aborted.
0x0012d422 in __kernel_vsyscall ()
(gdb) bt
#0 0x0012d422 in __kernel_vsyscall ()
#1 0x003ff651 in raise () from /lib/tls/i686/cmov/libc.so.6
#2 0x00402a82 in abort () from /lib/tls/i686/cmov/libc.so.6
#3 0x0043649d in ?? () from /lib/tls/i686/cmov/libc.so.6
#4 0x00440591 in ?? () from /lib/tls/i686/cmov/libc.so.6
#5 0x00443395 in ?? () from /lib/tls/i686/cmov/libc.so.6
#6 0x00444f9c in malloc () from /lib/tls/i686/cmov/libc.so.6
#7 0x08157d51 in btAllocDefault (size=1195)
at /home/bm/other/bullet-2.77/src/LinearMath/btAlignedAllocator.cpp:24
#8 0x08157e6f in btAlignedAllocInternal (size=1176, alignment=16)
at /home/bm/other/bullet-2.77/src/LinearMath/btAlignedAllocator.cpp:170
#9 0x080b4803 in btCollisionObject::operator new (sizeInBytes=1176)
at /home/bm/other/bullet-2.77/src/BulletCollision/CollisionDispatch/btCollisionObject.h:115
#10 0x080e9e85 in btSoftBodyHelpers::CreateFromTetGenData (worldInfo=...,
ele=0x81707e4 "48 4 0\n 1 26 28 16 12\n 2 28 21 16 12\n 3 25 19 2 20\n 4 28 27 10 14\n 5 17 3 20 1\n 6 20 24 18 19"...,
face=0x0, node=0x81707bb "# Generated by tetgen -YY test.1.smesh \n",
bfacelinks=false, btetralinks=true, bfacesfromtetras=true)
at /home/bm/other/bullet-2.77/src/BulletSoftBody/btSoftBodyHelpers.cpp:957
#11 0x080b0bc5 in Init_TetraCube (pdemo=0x8309c60)
---Type to continue, or q to quit---
at /home/bm/other/bullet-2.77/Demos/SoftDemo/SoftDemo.cpp:1315
#12 0x080b122c in SoftDemo::clientResetScene (this=0x8309c60)
at /home/bm/other/bullet-2.77/Demos/SoftDemo/SoftDemo.cpp:1451
#13 0x080b383f in SoftDemo::initPhysics (this=0x8309c60)
at /home/bm/other/bullet-2.77/Demos/SoftDemo/SoftDemo.cpp:1849
#14 0x080a5b90 in main (argc=1, argv=0xbffff384)
at /home/bm/other/bullet-2.77/Demos/SoftDemo/main.cpp:28


I put the question to the forum here but no responses- not many people are that adept at the soft body physics yet.

Something functional

But I've been able to create a trivial example from scratch that works:

static const char* getNodes() { return(
"8 3 0 0\n"
" 0 1 1 1\n"
" 1 1 1 -1\n"
" 2 1 -1 -1\n"
" 3 1 -1 1\n"
" 4 -1 -1 1\n"
" 5 -1 1 1\n"
" 6 -1 1 -1\n"
" 7 -1 -1 -1\n"
"# \n"); }

static const char* getElements() { return(
"4 4 0\n"
" 0 0 1 3 5\n"
" 1 4 7 5 3\n"
" 2 6 5 1 7\n"
" 3 2 1 7 3\n"
"# \n"); }




Maybe I should try to corrupt the above into provoking the same kind of crash?

Update- Solution Found

tetgen by default is creating 1-based vertex indices, while Bullet is expecting 0-based indices- the -z flag will make tetgen output zero-based. So no more crashing.

Bullet Soft Body Physics from binarymillenium on Vimeo.