Daniel Reetz, the founder of the DIY Book Scanner community, has recently started making videos of prototyping and shop tips. If you are tinkering with a book scanner (or any other project) in your home shop, these tips will come in handy. https://www.youtube.com/channel/UCn0gq8 ... g_8K1nfInQ

Methods To Sense The 3D Surface/Structure Of A Book

DIY Book Scanner Skunk Works. Share your crazy ideas and novel approaches. Home of the "3D structure of a book" thread.
Anonymous1

Re: Methods To Sense The 3D Surface/Structure Of A Book

Post by Anonymous1 » 22 Feb 2011, 15:19

When I ordered a speedcube from China, the receipt said "6 to 8 weeks". Seven months later, a dented packages showed up at my front door with a disassembled cube inside. And I paid $10.00 for shipping. Ha.

Shipping isn't a problem, Dan. As long as the mailman doesn't destroy the package, I'm happy.

steve1066d
Posts: 296
Joined: 27 Nov 2010, 02:26
E-book readers owned: PRS-505
Number of books owned: 1250
Location: Minneapolis, MN
Contact:

Re: Methods To Sense The 3D Surface/Structure Of A Book

Post by steve1066d » 22 Feb 2011, 16:08

Nice! With luck the post office will be a bit faster and I'll be able to play with it this weekend. Thanks again for putting these together.
Steve Devore
BookScanWizard, a flexible book post-processor.

steve1066d
Posts: 296
Joined: 27 Nov 2010, 02:26
E-book readers owned: PRS-505
Number of books owned: 1250
Location: Minneapolis, MN
Contact:

Re: Methods To Sense The 3D Surface/Structure Of A Book

Post by steve1066d » 22 Feb 2011, 23:41

Anonymous,

I was looking at you code and I realized that your ModelToPlane method could be simplified.. It always has a theta of 0. So this method works just as well:

Code: Select all

def ModelToPlane(x, y, z, cz = 1, vz = 1):
  dz = z - cz
  bx = x * vz / dz
  by = y * vz / dz
  return (bx, by) 
Last edited by steve1066d on 23 Feb 2011, 09:32, edited 1 time in total.
Steve Devore
BookScanWizard, a flexible book post-processor.

StevePoling
Posts: 290
Joined: 20 Jun 2009, 12:19
E-book readers owned: SONY PRS-505, Kindle DX
Number of books owned: 9999
Location: Grand Rapids, MI
Contact:

Re: Methods To Sense The 3D Surface/Structure Of A Book

Post by StevePoling » 23 Feb 2011, 02:01

Sorry to ask an ignorant question...

When I see the flattened images, I still see the shading gradient. But it appears that you all have an estimate of <x,y,z> of the paper for every pixel in the image. It seems that you can estimate the angle of the paper with respect to the illumination source, by differencing adjacent pixels' z-coordinates. I further presume that brightness varies with the cosine of the angle of incidence between light and paper. Can you then normalize the brightness of each pixel accordingly?

steve1066d
Posts: 296
Joined: 27 Nov 2010, 02:26
E-book readers owned: PRS-505
Number of books owned: 1250
Location: Minneapolis, MN
Contact:

Re: Methods To Sense The 3D Surface/Structure Of A Book

Post by steve1066d » 23 Feb 2011, 09:31

Its probably possible to do something like that, but it will be a lot of trial and error. I've tried doing things like adjusting for lighting in BSW, and its not that easy.. there isn't a simple relationship between the brightness of a picture and the actual brightness value of the pixel. I also think that not only the angle would matter, but at certain angles you get some reflection from the page itself that adds to the difference in lighting.

It may be simpler to play with lighting to try to illuminate the whole page evenly to begin with, but in any case, it is worth playing around with.
Steve Devore
BookScanWizard, a flexible book post-processor.

U235
Posts: 2
Joined: 04 Mar 2014, 00:53

Re: Methods To Sense The 3D Surface/Structure Of A Book

Post by U235 » 23 Feb 2011, 12:23

daniel_reetz wrote:It resembles an idea that Spamsickle proposed before, except that he suggested to have two cameras looking at the edge of the book.
Hi.
I slightly changed the great idea Spamsickle. Easy hardware: Photo camera + web camera (low resolution), no plates, no trigger, speed ~20 shot/min.
Photobook.pdf
(1.26 MiB) Downloaded 306 times
(maybe it will be interesting)

User avatar
Gerard
Posts: 154
Joined: 17 Oct 2010, 07:15
Number of books owned: 0
Location: Berlin (Germany)

Re: Methods To Sense The 3D Surface/Structure Of A Book

Post by Gerard » 23 Feb 2011, 13:07

the continues mode is a nice idea, with software you can delete duplicated images, with qr-code cards it would be possible to say "i am going some pages back, try to find how far i was going" (even without qr codes this would be possible, just comparing the page to the last 5 pages made)

so it would be possible to scan in a continues mode without headache

the dewrapping from the side with a webcam needs to be implemented and proof if is working, but maybe with laser it is still an option, the laser needs an sound toggle that is reacting to the click of the cam maybe an photodiode on the lcd screen and "photo review"-off is a better solution

the concept with the sound is maybe to unreliable, maybe the time of the picures is enough, and the time when the video was started

or lets buy a second webcam and point it directly on the lcd screen of the cam thats making the pictures, if you turn on a cannon the "review after shoot" option off, the screen will be black every time a photo was taken

Anonymous1

Re: Methods To Sense The 3D Surface/Structure Of A Book

Post by Anonymous1 » 23 Feb 2011, 16:29

steve1066d wrote:Anonymous,

I was looking at you code and I realized that your ModelToPlane method could be simplified.. It always has a theta of 0. So this method works just as well:

Code: Select all

def ModelToPlane(x, y, z, cz = 1, vz = 1):
  dz = z - cz
  bx = x * vz / dz
  by = y * vz / dz
  return (bx, by) 
I didn't bother to change all those sines and cosines because the camera's position isn't perfectly above the book (it's a bit off on the y axis), which skews my lines a little bit, which makes the dewarping off on the top.

I'm going to calculate the relative y offset by finding the difference between the midpoint of the two lasers and the actual image's height. This gives me absolutely no units, but if I knew some exact numbers, I could make a bit more use out of it.

StevePoling wrote:Sorry to ask an ignorant question...

When I see the flattened images, I still see the shading gradient. But it appears that you all have an estimate of <x,y,z> of the paper for every pixel in the image. It seems that you can estimate the angle of the paper with respect to the illumination source, by differencing adjacent pixels' z-coordinates. I further presume that brightness varies with the cosine of the angle of incidence between light and paper. Can you then normalize the brightness of each pixel accordingly?
You could try to apply a brightness and contrast boost to the seam area, but that would look funky in the end. I'm actually playing with adaptive thresholding, and I must say, it works well for bitonalizing text. After a quick edge detection pass and a mask, I've got some good text thresholding. I'm not really sure how ST handles it, but it's thresholding seems uniform (even if a dark area has contrast between the text and background, ST still makes it all black) and not adaptive...

As for the non-interactive triggering, have you looked at optical flow or motion detection? I played with optical flow (again with OpenCV), and it was pretty sweet to watch the little vectors wiggle around. I setup a webcam next to the camera, and it was detecting page flips pretty well (I got it to tell me the direction of the flip. This would be a helpful if you have to redo a page), but I found it to be a pain (it was pretty slow).

Maybe motion detection would be good (if your area of interest is contained within the book's content area, you don't have to worry much about other objects interfering). When I have a bit more time, I'll post a Python script demonstrating what I mean. I already have my optical flow script floating around the Software section (in the Gphoto2 thread), so I'll tinker with that.

Even with all this automation stuff, I still like scanning with a trigger. I get 12 pages/minute with only a remote trigger, so I don't think a webcam trigger would help me much. But it's kind of fun to play with computer vision, so I'll contribute if I can.

But that mirror idea is pretty cool. I'm going to try doing what Snapter did, and dewarp with only the profile of the book while I'm waiting for Dan's epic lasers.

steve1066d
Posts: 296
Joined: 27 Nov 2010, 02:26
E-book readers owned: PRS-505
Number of books owned: 1250
Location: Minneapolis, MN
Contact:

Re: Methods To Sense The 3D Surface/Structure Of A Book

Post by steve1066d » 23 Feb 2011, 19:37

I'm going to calculate the relative y offset by finding the difference between the midpoint of the two lasers and the actual image's height. This gives me absolutely no units, but if I knew some exact numbers, I could make a bit more use out of it.
You can get real numbers by using the DPI, which can be derived from the graph paper scans. I think it came out to 193 DPI when I measured it.
Steve Devore
BookScanWizard, a flexible book post-processor.

steve1066d
Posts: 296
Joined: 27 Nov 2010, 02:26
E-book readers owned: PRS-505
Number of books owned: 1250
Location: Minneapolis, MN
Contact:

Re: Methods To Sense The 3D Surface/Structure Of A Book

Post by steve1066d » 23 Feb 2011, 23:29

I think I'm finally getting good enough results to use. Here's an image that I took right from the conversion, all I did was crop it (no keystone correction):
IMG_0880_5.jpg
To get around having to do a spherical correction on the image first, I'm using the difference from the center of the object instead of the same height over the whole object:

Code: Select all

 
double z = cameraDistance - pointHeight;
// this is the change.. in the past I just used the value above to calculate the multiplier
z = Math.sqrt(z * z + fromCenterX * fromCenterX + fromCenterY * fromCenterY);
mult = (float) (cameraDistance / z);
That seemed to do the trick. There still is a bit of keystoning in the bottom of the image, which is due to something in the setup being off a little. My guess is that it will basically be impossible to get everything aligned perfectly, but because each image will have the same error, the same keystone fix should work for all the pages in the book.

One thing that isn't perfect is that I still haven't found the best way to figure out the height of the destination point. Its easy enough to figure out the height of a point on the source image, but the way dewarping works, you have to work backwards. That is, you know the pixel you want to paint, and you need to figure out where it was on the original image. So there's a bit of the chicken and the egg problem. If I knew where the source was for a point I'd be able to correctly determine the height, but to get the source, I need the height.

I worked around it by using the source height as an estimation for the destination height, doing a temporary warp, and recalculating the heights based on the warp that was just generated. (and then did the process one more time). It still isn't quite right, and it does seem like there should be a better way, but I'm out of ideas. Luckily, it does seem to be good enough.
Steve Devore
BookScanWizard, a flexible book post-processor.

Post Reply