Original ~600pg/hr, very portable scanner now achieving ~900pg-1100pg/hr

Built a scanner? Started to build a scanner? Record your progress here. Doesn't need to be a whole scanner - triggers and other parts are fine. Commercial scanners are fine too.

Moderator: peterZ

User avatar
Mohib
Posts: 107
Joined: 05 Apr 2014, 21:15
Number of books owned: 0
Country: Canada

Re: Original ~600pg/hr, very portable scanner now achieving ~900pg-1100pg/hr

Post by Mohib »

dpc wrote: 30 Mar 2017, 12:35Please don't take offense to my suggestion, but if your goal is to produce a single page scanner that can be easily setup in remote locations such as a library, I'd encourage you to build something like this.
None taken. All ideas welcome, and that's a nice scanner and does an excellent job.

The objective the builder had for the design was to handle text very close to the book's spine and for that, our imaging methods are the same (mine vertical, their's angled) and so we get the same results as my scanner meets that key objective the designer had.

As far as meeting my objectives, here's why that scanner doesn't work for me (though may be fine for others):
dpc wrote: 30 Mar 2017, 12:35 It's dirt simple to build
That is relative to your skill set and tool box. It may be "dirt simple" to build for some with the skills, tools and space to build it, but even if "dirt simple", it does require a lot, lot more construction than mine (which requires all of drilling 6 holes and making 5 cuts of plastic pipe and 1 of threaded rod -- no painting, accurate wood work, etc.) and even this some find too much!

Also that scanner will require some good wood working skills and tools (the builder mentioned at one point: "I used a table saw for the 3 sided frame surrounding the glass") and as well as accurate work to make sure the platen square with the camera so there are no key-stoning effects. By comparison I use a ball joint and the iPhone's levelling tool (second panel of the compass app) to make sure the camera is square with the table (or rather I place the phone on the table to see its tilt (if any) and then match that after mounting the phone to the support; just takes a minute to get done).
dpc wrote: 30 Mar 2017, 12:35less expensive
Yes that's probably true, but at around $80ish for mine, it's not going to break the bank. And besides, the Manfrotto clamp can be used for general picture taking if you don't want to lug a tripod around.
dpc wrote: 30 Mar 2017, 12:35far more portable
I would have to disagree with you there. It doesn't look like it knocks down, but even if it does, it's got some very large parts, and looks like it weighs quite a bit. I really would not want to walk into a library and set that up.

For comparison, here's my scanner dismantled and packed for transport. Assembly requires attaching the handle to the platen (wrapped in the white towel) (2 bolts needed) and attaching the horizontal post to the vertical post (1 thumb nut needed).
Scanner 900 - Dismantled for transport - small.jpg
Scanner 900 - Dismantled for transport - small.jpg (318.29 KiB) Viewed 9568 times
dpc wrote: 30 Mar 2017, 12:35 doesn't have the varying image size problems that your design has
That is true, but the image size problem is not a "real problem" but more an aesthetic one. The difference in size is only about 5% for 230 page book (see viewtopic.php?f=14&t=3007#p17556) and easily solved with the macro focus bar to the point it requires a 2 second adjustment every 40 pages to virtually eliminate.

However, the last commenter (MJM) in the thread makes some very important points (viewtopic.php?f=14&t=2549&start=10#p19577). In particular, he/she mentioned:
With it you can get more text when the text is too close to the gutter, but the tricky part of it is scanning without harming the book, and most of them will be if they're opened more than 90º. In any case, one key is to get the pages to be flatten from the book spine itself, so flattening the text with a glass is not enough, you must get the entire book in a position where the pages block is not turned or curled at any point. You make this at the same moment you choose the angle of scanning. Its the way to do it specially if you want to OCR the text lated, in order to avoid distorsions.
One of my design considerations, was to be able to scan paperbacks without cracking or damaging the spine (and even trickier if the text is close to the spine). As you'll see with the the scanner you suggested, books are opened well beyond 90 degrees (it looks like 135 degrees) to scan and that will almost certainly damage the spine, as the commenter noted also.

By comparison, with mine you don't need to open books much more than if you were reading the book. See the schematic I posted earlier in this thread. Notice the book is not bent back beyond 90 degrees and it can be reduced to perhaps 75 degrees as long as it clears the field of view (FOV) as shown by the diagonal lines from iPhone holder to the left and right edges of the book. If you drop the camera on the macro rail the field of view allows you to "look deeper" into the spine so even books that curl a lot are less of a problem and the curl does not block the page being scanned.

Furthermore, since you're not having to support the weight of the book (see below) you can effortlessly control the book to ensure the spine is not damaged. I've scanned many paperback books without creating so much as a crease in the spine as the book was never opened much more than you would for reading and I think if I had the bevel on the left edge of the platen, I mentioned earlier and shown in the schematic, things would be even better.

Besides this problem, the other real problem (for me) with the design you suggested is its operation: it will be slower, less efficient and more tiring to use because of step 4 (steps 1, 2, 3 and 5 are the same as my scanner):

1) You have to lift the platen with say your right hand (and lets assume you even have a convenient handle and shutter trigger attached to it -- which it doesn't so more construction).

2) You have to turn the page.

3) Drop the platen back.

4) But now, instead of just taking the image (as with mine), you have to lift **and** press the entire book with your left hand against the platen to flatten the page, and some times press quite hard depending on the binding and especially so to get text close to the spine. As the scanning progresses, the book will get heavier and heaver as more pages fall to the left side, and it will get more tiring to press. Also the scanner itself will have to be heavy or clamped down to the table so the it doesn't end up sliding on the table if you have a book with a spine that requires a lot of pressure to flatten the page close to the spine.

By comparison, with mine, there is no extra tiring step to lift and press the book flat against the platen, instead when you drop the platen back (step 3) you just put a little downward pressure on it (rather than lifting the book) to flatten the page -- if more pressure is needed for some spines, it's not tiring as the platen handle actually acts like a lever to amplify the pressure down and so just a little more pressure (and that is downwards not upwards) is all that is needed.

Essentially, since my design uses the table as part of the scanner, it has less parts and construction and since it uses gravity to assist you, rather than work against you, it's less tiring and more efficient to use.

5) Take the image. However, since there is no dark background behind the book (as there is with mine, between the book and the table) you'll probably have to tape a large dark card to the front cover of the book (and then the back cover) and worry about it.
Last edited by Mohib on 30 Mar 2017, 23:31, edited 4 times in total.
User avatar
Mohib
Posts: 107
Joined: 05 Apr 2014, 21:15
Number of books owned: 0
Country: Canada

Re: Original ~600pg/hr, very portable scanner now achieving ~900pg-1100pg/hr

Post by Mohib »

Dtic

Thanks for that script. I'll give it a test.

There is, however, a subtle nuance to the way the scaling must be applied which I elaborated on in a new post on the Scan Tailor GitHub here: https://github.com/scantailor/scantailor/issues/89

Specifically I wrote:
As an update to my last comment about using a calibration page, the scale adjustment factor has to be applied differently, depending on whether 1 or 2 cameras were used for the scanning. So there would need to be an additional option to say if calibration pages were for 1 or two cameras.

- For a 1 camera setup (like mine), if there were 100 pages scanned, then pages 1, 3, 5, etc start big and go smaller, while pages 2, 4, 6, etc. start small and go bigger (because even pages are scanned in reverse with the book upside down). So for odd pages the images have to be gradually scaled bigger (start with no scaling on page 1, and maximum scaling for the last page 99). But for even pages, the images have to the scale applied in reverse, so maximum scaling on page 2 and no scaling on page 100.

- For a 2 camera setup, a start and end calibration page should be used for each camera (just in case camera-platen distance is no the same for left and right pages). So there would be two calibration pages at the start (one for the left and one for the right camera) and two at the end, and two scale factors. Since with two cameras both left and right are scanned at the same time and in the same direction, then the scaling for all pages would be done starting with no scaling for the first page and maximum scaling on the last page, but using the left scale factor on the left pages and the right scale factor on the right pages.
With respect to your point:
dtic wrote: 30 Mar 2017, 07:44 The chance of this niche feature being added to Scan Tailor is very small I think.
You are probably right, however there is actually a broader advantage to this feature: it eliminates the need for precise and accurate camera-platen distance even in two camera setups so makes construction simpler and easier. In the bigger picture, the feature allows for one more degree of freedom in the scanner design and so may allow for more creative designs -- both single and dual camera -- to be developed.
dtic
Posts: 464
Joined: 06 Mar 2010, 18:03

Re: Original ~600pg/hr, very portable scanner now achieving ~900pg-1100pg/hr

Post by dtic »

Mohib wrote: 30 Mar 2017, 16:26 if there were 100 pages scanned, then pages 1, 3, 5, etc start big and go smaller, while pages 2, 4, 6, etc. start small and go bigger (because even pages are scanned in reverse with the book upside down)
Right, but that should be no problem. First shoot the right side pages starting with the first page in the book: 1 3 5 7 ... 99. Then flip the book 180 degrees and shoot the left side pages starting with the last page in the book: 100 98 96 ... 2.

When done put the saved right and left side images in separate folders and don't rename the files yet. Then run the script once on each folder. Since page 100 was the first left side page to be shot the phone presumably gave it a filename (a datestamp or incremental number) that makes it sort first among the left side page images. In that case the script should work correctly also on the left side images.

After the script has finished you can rename the left side images to reverse their sort order.
User avatar
Mohib
Posts: 107
Joined: 05 Apr 2014, 21:15
Number of books owned: 0
Country: Canada

Re: Original ~600pg/hr, very portable scanner now achieving ~900pg-1400pg/hr!

Post by Mohib »

Here are some better pictures and a video of it in operation for a 1 minute test. I just did this straight after a single test video to see if the camera was recording ok and the view was clear. So no practice runs!

And ... I scanned 24/25 pages (depending on how you count) in 1 minute. That translates to 1440-1500 pages per hour (with a one camera scanner)! Needless to say I was sure surprised!

In the video I start a 1 min timer (just visible on the table at the start) just before I start shooting. You can hear the camera shutter and the timer beep at the end. On one page I triggered the camera twice (the extra shot is not part of the count). You will also notice a few times there is a delay after I press the remote trigger and the shutter trips. I think this is because Camera+ pauses sometimes to transfer pictures out of a cache but the delay is all included in the 1 minute time.

Video link: http://po.st/scanner900video

In the pictures you'll notice various configurations:

a) Normal
b) With light
c) With extended camera arm for using the light with larger books (to avoid a hot spot reflection at the edge of the platen)
d) Mounted at rear of table.

I've also posted a sample page scanned (reduced size), the page after processing with ScanTailor (reduced size converted to jpg) and the a list of the EXIF time stamps of the scans.

I'll post construction details shortly too.
Scanner 900 - overview - P4020165 - small.jpg
Scanner 900 - front view - P4020164 - small.jpg
Scanner 900 - front with light - P4020167 - small.jpg
Scanner 900 - iPhone closeup - P4020169 - small.jpg
Scanner 900 - extended arm for light with larger books - P4020173 - small.jpg
Scanner 900 - iPhone closeup with extended arm - P4020174 - small.jpg
Scanner 900 - mounted at rear of table - P4020175 - small.jpg
IMG_1729 - small.jpg
IMG_1729 - small from tif.jpg
Scanner 900 - 1 min, 25 pages - exif date stamps.png
Scanner 900 - 1 min, 25 pages - exif date stamps.png (65.6 KiB) Viewed 9517 times
Last edited by Mohib on 03 Apr 2017, 13:16, edited 1 time in total.
dpc
Posts: 379
Joined: 01 Apr 2011, 18:05
Number of books owned: 0
Location: Issaquah, WA

Re: Original ~600pg/hr, very portable scanner now achieving ~900pg-1100pg/hr

Post by dpc »

Do you have an image of a scanned black page that you could post?
User avatar
Mohib
Posts: 107
Joined: 05 Apr 2014, 21:15
Number of books owned: 0
Country: Canada

Re: Original ~600pg/hr, very portable scanner now achieving ~900pg-1400pg/hr!

Post by Mohib »

dpc wrote: 03 Apr 2017, 13:09 Do you have an image of a scanned black page that you could post?
A black page? Are you trying to see reflections?

As you can see from the black background in the pics, there are and will of course be reflections, the same as there would be with any scanner that's not shielded from ambient light by a dark shroud and one reason lights can be important.

However, my objective with this scanner was to image text pages, not pictures, and for text minor, almost imperceptible reflections on normal white pages are unimportant as Scan Tailor drops them out beautifully (see the sample above which was taken without the light next to bright, sunlit window).

That being said, with my original scanner (using a camera not iPhone), I did post extensive tests and samples here:

viewtopic.php?f=14&t=3007&start=20#p17723

to show the reflections on normal white pages, white pages with photos and then again when using the single light setup.

As I explained there, the single light is not optimal for several reasons (mainly uneven illumination, as expected, if it is the sole source of light), but it does eliminate reflections and now, with the new extension bar, I can eliminate the hot-spot it creates in the platen, as it pushes the camera a few inches further away. However, as I also mentioned there, when combined with sunlight to ensure even lighting, the single light does it's job and eliminates reflections.
dpc
Posts: 379
Joined: 01 Apr 2011, 18:05
Number of books owned: 0
Location: Issaquah, WA

Re: Original ~600pg/hr, very portable scanner now achieving ~900pg-1100pg/hr

Post by dpc »

Yes, the concern is about reflections showing up in the scanned images. It would be good for people that might be considering building a scanner such as yours to understand this. Scanning a solid black page under your platen would reveal any potential problems with reflections.
User avatar
Mohib
Posts: 107
Joined: 05 Apr 2014, 21:15
Number of books owned: 0
Country: Canada

Re: Original ~600pg/hr, very portable scanner now achieving ~900pg-1100pg/hr

Post by Mohib »

dpc wrote: 04 Apr 2017, 12:38 It would be good for people that might be considering building a scanner such as yours to understand this. Scanning a solid black page under your platen would reveal any potential problems with reflections.
The reflections are no worse nor better than from any scanner that does not have dedicated lighting and/or is not protected by a dark shroud overhead and other surrounding ambient lighting (such as the scanner you suggested earlier, David Landlin's excellent and popular scanner, and many, many of the other excellent and productive scanner builds here).

Nevertheless, for regular text pages, processed by Scan Tailor, the reflections are of no consequence as Scan Tailor removes them. See, for example, the test image posted above (taken during the video) was imaged **without** the light right next to a bright, sun-lit window (as seen in the video). The reflection (which the light would have removed) is barely visible in the original image and the copy I posted here (if you know where to look) and, of course, totally absent after Scan Tailor processing, as seen in the post processed sample posted above.

Nevertheless, the reflections from my scanner are easily removed with the single lamp, per the very extensive tests I performed 3 years ago and their results posted here:
viewtopic.php?f=14&t=3007&start=20#p17723
User avatar
Mohib
Posts: 107
Joined: 05 Apr 2014, 21:15
Number of books owned: 0
Country: Canada

Re: Original ~600pg/hr, very portable scanner now achieving ~900pg-1100pg/hr

Post by Mohib »

OK I've done some tests with my single LED light in a dark room to minimise reflections as any other kind of reflection test would not be helpful.

To ensure the lamp does not reflect directly in the platen, I added the extension pipe to the camera support (see pictures above).

I also noticed in the pictures I posted above, I actually had the platen the wrong way and the 3 holes should be towards the spine not towards the edge of the page (the video is correct). It's not a huge difference as the platen is 11" wide and hole #1 (see screenshot below) at 5" from the edge of the platen (and so almost at the centre of the platen) and should be 5" from the spine when attached correctly to the handle (not 6" as in the earlier pictures above). So having it backwards (so it's 6" from the spine) makes no difference on regular sized books, but it does push the handle further into the field of view and so, with wider books, it starts to show up in the frame.

However, even with platen attached the right way (as shown in the screenshot below, and the handle is an inch further out so it no longer appears in the field of view), the handle does show up as a reflection at the very top and bottom left corners and down the left edge of the frame when scanning tall and/or wide books. Consequently for these (and if reflections are critically important, i.e. not being processed by Scan Tailor), the handle should be moved to hole #2 or hole #3 (and optionally painted black or made from black pipe), so it's attached closer to the spine and further out.
3 platen handle mounting holes
3 platen handle mounting holes
3 platen handle mounting holes.png (362.4 KiB) Viewed 9432 times
Alternatively the handle could be made longer, as it was in my original scanner, but that creates a larger arc when lifting the platen (which means more arm motion and slower page turning). The large arm motion, from both the longer handle and it being attached near the spine, feels noticeably slower when scanning. Better, I think is to drill extra platen handle mounting holes for the odd times you might want to scan large and/or wide books with photos and that will not be Scan Tailor processed to black and white, and just remount the handle (a 5 min job).

These are the tests in the order the images appear below (the images are at 33% of the original size).

1) Scanner 900 - 9 inch vertical ruler - black background - with light at night.jpg
(Platen attachment hole is unimportant with small books but hole #1 is most efficient.)

Typical sized book, edge of platen (where spine would be) is at left edge of frame.

2) Scanner 900 - 12 inch vertical ruler - black background - with light at night (middle hole).jpg
(Used hole #2 -- good for wide books. Tall books should use hole #3.)

The edge of platen (where spine would be) is at left edge of frame. This image was taken with the platen handle attached to hole #2 and there's small reflection from platen handle supports at top and bottom left but the long part of the handle does not reflect down the left edge of image where the spine of the book would be. Consequently, (as explained above) the handle should be moved out to hole #3 if scanning a very tall book and eliminating reflections is critically important. Otherwise, the hole #2, used here, is sufficient to eliminate the platen reflection for a wide book.

3) Scanner 900 - 8 inch book - horizontal ruler - full frame.jpg
(Used hole #1 which not the best hole for wide books. I should have used hole #2.)

Tall book 12" book (8" wide). The platen handle just barely clips the left corners of the page, but is almost impossible to notice in the original. It's only possible to know this by tracking the reflection in the dark area and see it intersects the very corners of the page and runs down the edge near the spine, but if I had the handle in hole #2 (as in the previous dark frame image), it would have missed the page altogether.

4) Scanner 900 - 10.5 inch book - horizontal ruler - full frame.jpg
5) Scanner 900 - 10.5 inch book - horizontal ruler - square frame.jpg
6) Scanner 900 - 10.5 inch book - horizontal ruler (missing)- dark picture - full frame.jpg
7) Scanner 900 - 10.5 inch book - horizontal ruler (missing) - dark picture - square frame.jpg
(All of these used hole #1 which not the best hole for wide books. I should have used hole #2.)

All these 4 were from an ultra wide (10.5") book -- with dark pictures (so to most strenuously test reflections). I shot the pages in both full frame and square frame modes.

8) Scanner 900 - 9.25 inch book - horizontal ruler (missing) - full frame.jpg
(Used hole #1 which not the best hole for wide books. I should have used hole #2.)

Another wide book (between the above two books) with a regular (not dark) picture. Again, because the platen was attached to the wrong hole the handle has a small reflection near the spine.

All in all I'm quite impressed with the colour (camera was set to auto white balance) from the iPhone 4s. It is much better than the old camera I was using and I'm sure newer smart phones will be even better. As expected with a single light, illumination is not even. A second light would be good if perfect illumination is necessary. I'll give it a shot one day (although with a different light as the second one as I don't have two of these LED lamps) to see if shadows from the platen handle are a problem.
Scanner 900 - 9 inch vertical ruler - black background - with light at night - @33%
Scanner 900 - 9 inch vertical ruler - black background - with light at night - @33%
Scanner 900 - 12 inch vertical ruler - black background - with light at night (middle hole) - @33%
Scanner 900 - 12 inch vertical ruler - black background - with light at night (middle hole) - @33%
Scanner 900 - 8 inch book - horizontal ruler - full frame - @33%
Scanner 900 - 8 inch book - horizontal ruler - full frame - @33%
Scanner 900 - 10.5 inch book - horizontal ruler - full frame - @33%
Scanner 900 - 10.5 inch book - horizontal ruler - full frame - @33%
Scanner 900 - 10.5 inch book - horizontal ruler - square frame - @33%
Scanner 900 - 10.5 inch book - horizontal ruler - square frame - @33%
Scanner 900 - 10.5 inch book - horizontal ruler - square frame - @33%.jpg (496.17 KiB) Viewed 9450 times
Scanner 900 - 10.5 inch book - horizontal ruler (missing)- dark picture - full frame - @33%
Scanner 900 - 10.5 inch book - horizontal ruler (missing)- dark picture - full frame - @33%
Scanner 900 - 10.5 inch book - horizontal ruler (missing) - dark picture - square frame - @33%
Scanner 900 - 10.5 inch book - horizontal ruler (missing) - dark picture - square frame - @33%
Scanner 900 - 10.5 inch book - horizontal ruler (missing) - dark picture - square frame - @33%.jpg (437.92 KiB) Viewed 9450 times
Scanner 900 - 9.25 inch book - horizontal ruler (missing) - full frame - @33%
Scanner 900 - 9.25 inch book - horizontal ruler (missing) - full frame - @33%
dpc
Posts: 379
Joined: 01 Apr 2011, 18:05
Number of books owned: 0
Location: Issaquah, WA

Re: Original ~600pg/hr, very portable scanner now achieving ~900pg-1100pg/hr

Post by dpc »

Your scanner got a mention here.
Post Reply