Progress on dewarping functionality

[Tulon]

I've made some progress on dewarping recently, so I thought I'd give you a status update.
As I am writing this, the screencast is still in a queue to be encoded, so you may have to wait like half an hour. But hey, the stuff I am demonstrating is well worth a wait!

[eL_PuSHeR]

Amazing. It seems very promising. Thank you.

[daniel_reetz]

Yeah! Congratulations! This is going to be excellent. Thanks!

[spamsickle]

I see how you're manually setting the top and bottom horizontal curves. As you do so, you're also dragging the vertical grid so that it's more dense near the inner margin, where it curves away from the viewer. Your commentary didn't mention this second adjustment, but seemed to suggest that detecting the top and bottom curves would be done by software (presumably still enabling the manual adjustment which you are doing). Is it your plan currently to make that vertical flat-to-curved density adjustment automatic as well?

This seems like something that will be more useful to those using flatbed scanners -- one of the big advantages to the DIY scanner is that the pages are generally pretty flat. Still, as someone who still has a few raw flatbed texts (yes, I was already scanning whole books before discovering Dan's design, it took HOURS), I can definitely appreciate the need. I haven't used the "warp" tool in Photoshop, but I think it has a grid like the one you're using. The problem with a Photoshop solution is that the grid MUST be specified manually, and since the warping varies as one moves through the book on a flatbed scanner, the grid you specify on page 20 may make things worse on page 220. As far as I know, Photoshop scripting doesn't make it easy to interpolate from one grid to another, setting keypoints and animating the dewarping. I wonder if After Effects would facilitate such an interpolation?

In any case, if your software can automatically detect and correct the warping on each individual page, that will obviously be a huge benefit to the flatbed users. Best of luck in your continuing development work!

[daniel_reetz]

I can also see it being very useful for bodged-together DIY scanners, which don't press pages so perfectly, and also for just taking snapshots with no scanner at all -- something I've done dozens of times. Dewarping pages in this way is something I've tried to do many times and had little success at. Can't wait to see it, but I am waiting patiently regardless.

[Tulon]

As you do so, you're also dragging the vertical grid so that it's more dense near the inner margin, where it curves away from the viewer.

Horizontal density is not a result of me moving the control points. Density is derived from the shape of top and bottom curves, plus a constant I hardcoded for now. In case of photo shots, this constant has a physical meaning - it's the distance to the camera divided by straight line distance between the vertical text boundaries. So this constant is pretty constant and you won't have to change it all the time.

BTW, trapezoid distortion can also be handled using this method, but in this case text lines should better be straight.

[StevePoling]

Is trapezoid distortion the same thing as keystoning? The shape of keystones is generally trapezoidal...

I think the photographic guys made this a jargon term. I recall when I had a job setting up projectors when i worked in my college library's AV division, my boss used keystoning to describe an improper alignment of projector and screen resulting in a trapezoidal distortion of the projected image. And this alignment issue is the dual of the misalignment of the page and the photographic imaging plane.

Keep in mind, I'm neither a projectionist nor photographer by trade, so I may have misheard all this.

[StevePoling]

Tulon, I just watched your video. It is an absolute marvel. You are definitely on the right track.

Here's something to think about. the lighting on the gutter edge of your page is not as bright as on the flat part of the page near the edge of the book. However, the transition from poor lighting to good lighting is a continuous one. I'd imagine it's proportional to the cosine of the angle between a line tangent to the paper and a ray drawn from the light source. If there's any relationship between this angle and the curvature of the distortion-line the user drags, it can be used to normalize whiteness of the image. OR if normalizing the whiteness is easier, the normalization can provide an estimated tangent angle and then in turn an estimated distortion-line curvature.

In all, this is marvelous stuff. I'm going to be in London in a fortnight, and I'd love to buy you a round of your favorite beverage as a token of my gratitude. I'm sure I'm not the only guy here who shares that sentiment.

[Tulon]

Is trapezoid distortion the same thing as keystoning? The shape of keystones is generally trapezoidal...

I suppose they are the same. The distortion model I implemented can't handle arbitrary perspective distortion plus line curling though. The camera is expected to be perpendicular to the vertical boundaries of page content. If it's also perpendicular to a plane formed by those two, that's the ideal case. If it's perpendicular to the lines but not their plane (which is the case for most DIY setups), then lines will be uncurled correctly, but horizontal density will be a bit off, unless the page is flat.

Here's something to think about. the lighting on the gutter edge of your page is not as bright as on the flat part of the page near the edge of the book. However, the transition from poor lighting to good lighting is a continuous one. I'd imagine it's proportional to the cosine of the angle between a line tangent to the paper and a ray drawn from the light source.

Unfortunately even on a scanner, the light comes at an angle, which we don't know. In case of DIY setups, there is probably more than one light source, which makes it even worse.

In all, this is marvelous stuff. I'm going to be in London in a fortnight, and I'd love to buy you a round of your favorite beverage as a token of my gratitude.

Sure, PM me when you are there.

[Misty]

Wow, this looks excellent. Great work, Tulon! I'm looking forward to this.