Methods To Sense The 3D Surface/Structure Of A Book

DIY Book Scanner Skunk Works. Share your crazy ideas and novel approaches. Home of the "3D structure of a book" thread.

Moderator: peterZ

steve1066d
Posts: 296
Joined: 27 Nov 2010, 02:26
E-book readers owned: PRS-505
Number of books owned: 1250
Location: Minneapolis, MN
Contact:

Re: Methods To Sense The 3D Surface/Structure Of A Book

Post by steve1066d »

Here is a paper on book dewarping. If I follow it, I think it assumes that each spot is warped the same on the top and the bottom, and begins the calculation by estimating the curve of the warped page, then uses that to unwarp the book.

http://www.imlab.jp/cbdar2007/proceedings/papers/P1.pdf

I'm not sure if it is directly applicable to what we are doing, but It might be useful.

Here is another paper, but I don't have access to this one.. Perhaps blender, does...

http://www.springerlink.com/content/h2466581mn1q1765/
Steve Devore
BookScanWizard, a flexible book post-processor.
Anonymous1

Re: Methods To Sense The 3D Surface/Structure Of A Book

Post by Anonymous1 »

Welcome to the book scanning forum, andigit!

My algorithm doesn't work at all, actually. It doesn't account for the perspective distortion present when you take the picture, which radially warps the image. This affects the rest of the process greatly.

steve1066d was working on a fisheye distortion algorithm, and if you go back a few pages, you'll find that it almost works perfectly. The only problem is that the fisheye might not actually be completely symmetric, so I'm still waiting to calibrate the script to deal with my camera.

Could you elaborate a bit about the UV mapping? I didn't think about that before, as I wasn't planning to use Anonymous for anything but visualization, but it does seem like a viable solution.
Anonymous1

Re: Methods To Sense The 3D Surface/Structure Of A Book

Post by Anonymous1 »

@steve1066d, I do have access to those types of papers, but the website doesn't let me login. I downloaded a really nice paper about dewarping using a 3D model of a book, so if you would like to look at it, here's the link: http://www-personal.umich.edu/~nikitan/ ... arping.pdf

I'm still trying to figure out if I can view those papers remotely, but I can't seem to login with my credentials from home. Oh well...
atarkri
Posts: 14
Joined: 08 Jan 2011, 13:29

Re: Methods To Sense The 3D Surface/Structure Of A Book

Post by atarkri »

andigit wrote:I need to assume that camera is parallel and perpendicular to the laser line. If i know the angle I think I can calculate this into the algorithm.
This approach would be less useful in the general case (camera placed wherever), but may give higher quality results. Please go forward with this, and be sure to ask Dan if you need any {test equipment/ pictures satisfying those restraints}.
blender wrote:cv.Split(image, cv.CreateImage(cv.GetSize(image), 8, 1), channelG, channelB, None)
...
cv.InRangeS(imageThreshold, cv.Scalar(15, 15, 15), cv.Scalar(255, 255, 255), imageThreshold)
First of all, this is fantastic work! I was astounded when this worked over a small sample of test images. I was sure the approach of thresholding 15 < x < 255 wouldn't be generally useful...

Some notes:

You can extract channels directly in OpenCV, but it's not as "clean" a function call. In C++ (sorry):

Code: Select all

cv::Mat channelR(cv::Size(image.rows, image.cols), CV_8UC1);
cv::Mat channelG(cv::Size(image.rows, image.cols), CV_8UC1);
cv::Mat channelB(cv::Size(image.rows, image.cols), CV_8UC1);
cv::Mat channels[] = {channelR, channelG, channelB};

int from_to_extract_one[] = {1,0}; //extract the green channel only
cv::mixChannels(&image, 1, &channelG, 1, from_to_extract_one, 1);

int from_to_extract_all[] = { 0,0, 1,0, 2,0 }; // extract three channels
cv::mixChannels(&image, 1, channels, 3, from_to_extract_all, 3);
Most usefully, you may be able to improve the robustness of your algorithm on problem images by performing the "close" operation, again in C++:

Code: Select all

cv::Mat imageClose;
cv::morphologyEx(channelG, imgClose, cv::MORPH_CLOSE, cv::Mat(), cv::Point(-1, -1), 10);
//---------------^---------^---------^----------------^----------^------------------^
//          source,     dest,  operation, defaults to a 3x3, defaults to center,  iterations
The close operation performs a pair of dilate/erode operations in order to merge and strengthen "bright" areas, while retaining feature size. http://en.wikipedia.org/wiki/Closing_%28morphology%29 contains a more thorough explanation.

Last, if noise persists we can do edge detection. Since we are only interested in finding the first point in which we experience a jump (of value) in the color green, we can take the first derivative of the changing pixel values over x and/or over y. This method is quite a bit more complex, but should give more consistent (not necessarily better) results.
atarkri
Posts: 14
Joined: 08 Jan 2011, 13:29

Re: Methods To Sense The 3D Surface/Structure Of A Book

Post by atarkri »

Anonymous wrote:It doesn't account for the perspective distortion present when you take the picture, which radially warps the image.
You mean "lens distortion" http://en.wikipedia.org/wiki/Distortion_%28optics%29 . Perspective distortion means "information lost/modified due to a perspective transform" such as when we take a picture of a plane (book) at an angle.
steve1066d
Posts: 296
Joined: 27 Nov 2010, 02:26
E-book readers owned: PRS-505
Number of books owned: 1250
Location: Minneapolis, MN
Contact:

Re: Methods To Sense The 3D Surface/Structure Of A Book

Post by steve1066d »

Do we know the height of the camera for these images?

From the EXIF data and by calculating it from the change in DPI of the blocks, it looks like the camera height is around 21" above the base. Is that right?

I think the close camera position is causing the spherical and perspective distortions to be more pronounced than they normally would be (though it isn't a bad thing to come up with a solution that works even with the camera close to the page).
Steve Devore
BookScanWizard, a flexible book post-processor.
andigit

Re: Methods To Sense The 3D Surface/Structure Of A Book

Post by andigit »

Anonymous wrote:Could you elaborate a bit about the UV mapping? I didn't think about that before, as I wasn't planning to use Anonymous for anything but visualization, but it does seem like a viable solution.
http://en.wikipedia.org/wiki/UV_mapping

In my case I'm talking about it in 3D texture mapping term. Anyway if you use the ray tracing, you should be able to create a polygon mesh that fits exactly into the viewing cone (in our case the picture) and camera as tho POV.

This makes since in my head but Its hard to explain. Maybe i'll create some sample diagram later to explain it better.
Anonymous1

Re: Methods To Sense The 3D Surface/Structure Of A Book

Post by Anonymous1 »

Okay, I was just thinking you were talking about some other type of UV mapping. I do 3D graphics, so I'm pretty familiar with UV mapping.

Steve, I was thinking, why don't we just straighten the lines out? On a perfectly flat image, the laser lines should be completely straight, and since this is what we want, can't we just use those as guides for a dewarping grid of sorts?

I'm playing with warping in Python, but I'm still trying to decide whether to use pure Python or settle with OpenCV for this. How's your distortion correction coming along? Those test images looked really good.
steve1066d
Posts: 296
Joined: 27 Nov 2010, 02:26
E-book readers owned: PRS-505
Number of books owned: 1250
Location: Minneapolis, MN
Contact:

Re: Methods To Sense The 3D Surface/Structure Of A Book

Post by steve1066d »

Steve, I was thinking, why don't we just straighten the lines out? On a perfectly flat image, the laser lines should be completely straight, and since this is what we want, can't we just use those as guides for a dewarping grid of sorts?
That's a valid approach, and in fact some sort a calibration like that will probably be necessary when we have the process down. But for now, with the barrel corrections I've got reasonably corrected test images which should be good enough to produce reasonable dewarpings. I'm currently working on on the perspective distortion based on calculating each pixel's (or maybe region''s) size, then recalculating the position of everything based on that.

That combined with the perspective due to change in heights I sort of had working before should be enough to get some results. However, the steps involved aren't actually straightforward, so I'm not sure if I'll have something this weekend or not.
Steve Devore
BookScanWizard, a flexible book post-processor.
L0g1cM0del
Posts: 17
Joined: 04 Mar 2014, 00:53

Re: Methods To Sense The 3D Surface/Structure Of A Book

Post by L0g1cM0del »

I was able to get access to the paper that Steve couldn't get. Seems pretty interesting. Here is the link.

http://www.springerlink.com/content/h24 ... lltext.pdf
Post Reply