Microsoft Kinect: infrared depth maps for dewarping?

rob · Post by **rob** » 07 Nov 2010, 12:13

univurshul wrote:Where are you guys getting your laser cutters?

I use the one at my local makerspace.

Post by **daniel_reetz** » 10 Nov 2010, 14:18

Well, this turned out to be an interesting story. The guy that made the high quality hacked driver for the PS3 cam wrote code to control it right away, but didn't release it. Instead, he made a fund asking for $10,000 (including the Adafruit money), and when that was hit, he would release code.

http://www.engadget.com/2010/11/07/kine ... ame-video/

Unfortunately for AlexP, the open source approach steamrolled him. Within a few hours of the european launch of Kinect, hacker Hector Martin released working code for Linux.

http://www.engadget.com/2010/11/10/open ... -download/

I hope Adafruit offers Hector the prize. I just ordered a Kinect for myself. Looking forward to making this my first serious coding project.

vitorio · Post by **vitorio** » 10 Nov 2010, 15:22

Both the Kinect and the page-flipping demo use the same techniques for figuring out depth and alignment: structured light.

http://en.wikipedia.org/wiki/Structured_light

The links in the Wikipedia article include other references, such as using it to automatically align projectors (where I first heard about it), and algorithms and software. The Makerbot 3D scanner has its own software that performs the same functions.

The IEEE Spectrum article about the book flipping scanner says that it uses a custom camera that does 1280x1024 at 500fps, but I'm not sure you really need a) that high a framerate or b) a custom camera. Elphel makes ~$1k video cameras that can do at least 640x480 at 100fps, but maybe 1024x768 at 44fps is enough, or something in between. For lower-resolution video, you could theoretically use superresolution techniques to interpolate multiple video frames together into a higher-resolution or more detailed image.

"Just" adding macro lenses to the Kinect hardware to refocus the IR light on a page or across the entire platen might give you enough resolution to dewarp a book, even if you're not using the RGB camera to capture the images, and instead just turning pages and photographing one or both pages with a still camera. Reconstructing two pages from a single photograph of a "spread" alleviates a lot of the issues with dual cameras if you don't mind the resolution loss and it doesn't affect OCR.

Given that there are at least two pieces of software for reference out there for doing this, I'd guess it's probably doable by a software developer with a few weeks of free time, regardless of whether you're using Kinect or making it yourself with a projector like the Makerbot 3D scanner.

rob · Post by **rob** » 10 Nov 2010, 23:41

daniel_reetz wrote:...Within a few hours of the european launch of Kinect, hacker Hector Martin released working code for Linux.

Ha ha HA! Hector indeed got the $3k bounty. I doubt AlexP even had a working driver.

However, the depth output seems rather coarse...

Post by **daniel_reetz** » 10 Nov 2010, 23:45

I know... so great.

Staring at my Kinect now... I'd start hacking on it, but I'm working on another book scanner design.

univurshul · Post by **univurshul** » 11 Nov 2010, 00:58

I'm buying my girlfriend a Kinect for Christmas. hehehe...

Post by **daniel_reetz** » 11 Nov 2010, 01:15

Let me guess... she doesn't have an XBOX? Perfect!

Post by **daniel_reetz** » 12 Nov 2010, 00:52

I just had a meetup with Rob and Spamsickle, and Spamsickle said there was a video of the Kinect infrared lighting... I hadn't seen it, but I went looking and found this: http://www.youtube.com/watch?v=nvvQJxgykcU

Pretty interesting stuff...

Post by **daniel_reetz** » 12 Nov 2010, 02:25

Alright, so I am now working to get the Kinect up and running. Warning, this is going to be super-verbose and mostly notes to myself.

First things first: Need to get the code onto my machine. I'm using Ubuntu Linux for my starting point.

The code is stored on a Git server. Git is a version control system for code. I have never used it before. Using this tutorial as a starting point, I entered the following magic incantation: