Original ~600pg/hr, very portable scanner now achieving ~900pg-1100pg/hr

Built a scanner? Started to build a scanner? Record your progress here. Doesn't need to be a whole scanner - triggers and other parts are fine. Commercial scanners are fine too.

Moderator: peterZ

Post Reply
User avatar
Mohib
Posts: 107
Joined: 05 Apr 2014, 21:15
Number of books owned: 0
Country: Canada

Re: Original ~600pg/hr, very portable scanner now achieving ~900pg-1100pg/hr

Post by Mohib »

Thanks and thanks for the tips. I'll try them out soon. In the mean time, here's an update to the script (V1.2).

a) It now asks you for both left and right directories (you can provide either or both) and processes them (one after the other).

b) It also creates the output (excluding the first and last calibration images) in sub-directories under each saving the hassle of backing things up and/or over-writing your original files by mistake.

Code: Select all


#singleinstance, force

; Normalize Scanned Pages Scale V1.2
;
; Autohotkey script using graphics magic library
;
; Rescale pages from a book with varying camera/platen distance (i.e. difference in distance from camera to first page and last page scanned is thickness of book)
; to normalize page scale so content on each page is same as first page (i.e. last page is scaled the most amount).
;
; File modified date and time of each image must increase from first (largest) to last (smallest) page scanned.
;
; ALL pages must be scanned (including blank) with equal number of pages in separate directories for left and right hand pages;
; Output is placed in a subdirectory of each (calibration pages are not processed or copied)

; Full path to graphicsmagick gm.exe
; Example: C:\Program Files\GraphicsMagick-1.3.25-Q8\gm.exe
gm := "C:\Program Files (x86)\GraphicsMagick-1.3.25-Q16\gm.exe"

if !FileExist(gm) {
 MsgBox ,,ERROR,graphicsmagick's gm.exe file not found here:`n`n%gm%
 exitapp
}

GetImageDirectories:

; Get path to folder with right and/or right hand page jpg images to process

rightOutput := "\right-adjusted"
leftOutput := "\left-adjusted"
rightImagesFolder := ""
leftImagesFolder := ""

FileSelectFolder, rightImagesFolder,,2,Choose folder with ALL right hand pages (or enter cancel to skip processing right hand pages)
FileSelectFolder, leftImagesFolder,,2,Choose folder with ALL right hand pages (or enter cancel to skip processing left hand pages)

If (rightImagesFolder = "" and leftImagesFolder = "") {
  ExitApp
}

If (rightImagesFolder = leftImagesFolder) {
  MsgBox ,,ERROR,You have chosen the same directory for both left and right hand pages. This is not allowed. Please choose the directories again.
  goto GetImageDirectories
}

If (rightImagesFolder <> "") {
 if !FileExist(rightImagesFolder) {
	MsgBox ,,ERROR,Folder given for right hand images does not exist. Please choose both image directories again
   goto GetImageDirectories
 }
 
 ; Femove trailing slashes
 rightImagesFolder := RegExReplace(rightImagesFolder, "\\$")
 
 if FileExist(rightImagesFolder . rightOutput) {
  FileRemoveDir, %rightImagesFolder%%rightOutput%
 }
 FileCreateDir, %rightImagesFolder%%rightOutput%
}

If (leftImagesFolder <> "") {
 if !FileExist(leftImagesFolder) {
	MsgBox ,,ERROR,Folder given for left hand images does not exist. Please choose both image directories again
   goto GetImageDirectories
 }
 
 ; Femove trailing slashes
 leftImagesFolder := RegExReplace(leftImagesFolder, "\\$")

 if FileExist(leftImagesFolder . leftOutput) {
  FileRemoveDir, %leftImagesFolder%%leftOutput%
 }
 FileCreateDir, %leftImagesFolder%%leftOutput%
}

; Get number of pixels of calibration image (6" ruler) placed on first page and last page of book 
GetCalibrationSizes:
firstImageSize := 0
While firstImageSize = 0 Or firstImageSize = "" {
 InputBox, firstImageSize, First Calibration Page, Enter number of pixels taken by a 6`" ruler placed on the first page of the book (first callibration page),,450,130
 if (ErrorLevel = 1) {
   ExitApp
  }
If (firstImageSize = 0 Or firstImageSize = "") {
  MsgBox ,,ERROR,Zero or blank is not allowed
  }
 }

lastImageSize :=  0
While lastImageSize = 0 Or lastImageSize = "" {
 InputBox, lastImageSize, Last Calibration Page, Enter number of pixels taken by a 6`" ruler placed on the last page of the book (second callibration page),,450,130
 if (ErrorLevel = 1) {
   ExitApp
  }
 If (lastImageSize = 0 Or lastImageSize = "") {
  MsgBox ,,ERROR,Zero or blank is not allowed
  }
 If (firstImageSize < lastImageSize) {
  MsgBox ,,ERROR,Last/smallest image is greater than first/largest image. Please enter the values again.
  goto GetCalibrationSizes
 }
}

; Calculate scale difference (percentage) between first and last pages
totalScaleFactor := ( firstImageSize / lastImageSize ) * 100

; 8228=4+32+8192
MsgBox, 8228, WARNING, Processing images as follows`n`nRighthand images: %rightImagesFolder%`n`nLefthand images: %leftImagesFolder%`n`nPixel count (first image): %firstImageSize%`nPixel count (last image): %lastImageSize%`nFirst to last image scale factor: %totalScaleFactor%`n`nOk to proceed?
IfMsgBox, No
 ExitApp

If (rightImagesFolder <> "") {
 ProcessDirectory(rightImagesFolder, rightImagesFolder . rightOutput, "RIGHT")
}
If (leftImagesFolder <> "") {
 ProcessDirectory(leftImagesFolder, leftImagesFolder . leftOutput, "LEFT")
}

exitapp

ProcessDirectory(currentInputImagesFolder, currentOutputImagesFolder, currentPageHand)
{

 global totalScaleFactor
 global gm

 ; noOfFiles number of pages in directory to scale can be distributed across all pages
 ; Create file list sorted by modified date/time so script can be used after renaming and is not depenedent on original image file name.
 ; Using time stamp also ensures processing is not random if directory listing is not alphabetic
 fileList := ""
 noOfFiles := 0
 Loop, files, %currentInputImagesFolder%\*.jpg 
 {
  ; ignore first calibration image
  if (noOfFiles <> 0) {
   fileList = %FileList%%A_LoopFileTimeModified%`t%A_LoopFileName%`n
  }
  noOfFiles++
 }

 ; remove last calibration page from file list (search for second `n)
 fileList := Substr(fileList,1,Instr(fileList,"`n",false,-1,1))
 ; reduce count by 2 becuse 2 calibration images removed
 noOfFiles := noOfFiles - 2

 ; Sort file list (date is appeneded to front of each entry of FileList - so list will be sorted by time first and then by file name)
 ; This lets many files with the same time (i.e. seconds didn't change) still sort in the right order and not dependent on
 ; Dir listings that may be in random order. Timestamp is not really needed and can be removed.
 Sort, FileList

 ; Initalize the progress bar style
 Progress, B2 M FM10 FS10 WM10 WS10 W700, Starting, PROCESSING %currentPageHand% HAND PAGES -- PRESS ESCAPE TO ABORT

 Loop, Parse, fileList, `n
 {
  
  ; Omit the last linefeed (blank item) at the end of the list.
  if (A_LoopField = "") 
   break
  
  ; Split into two parts at the tab char.
  StringSplit, FileItem, A_LoopField, %A_Tab%  

  ; ignore first image coz not to be scaled
  if (a_index = 1) {
   FileCopy, %currentInputImagesFolder%\%FileItem2%, %currentOutputImagesFolder%\%FileItem2%
   continue
  }

  thisImageScaleFactor := (totalScaleFactor - 100) * (a_index / noOfFiles) + 100
  
  amountDone := floor(a_index/noOfFiles*100)
  ; Update the progress bar value
  Progress, %amountDone%
  ; Display the progress bar at the new value with new text
  Progress, ,Processing: %a_index%/%noOfFiles% -- Scale: %thisImageScaleFactor% -- %FileItem2%
  RunWait "%gm%" convert "%currentInputImagesFolder%\%FileItem2%" -resize %thisImageScaleFactor%`% -quality 100 "%currentOutputImagesFolder%\%FileItem2%",,hide
  
  ; Just make sure we actually exit when done, just to make sure  last file is not processed twice in case don't exit with the blank above (as that seems to fail sometimes)
  if (a_index = noOfFiles)
   break
 }

 Progress, Off
 return
}


; Allow escape to cancel
Escape::ExitApp
User avatar
Mohib
Posts: 107
Joined: 05 Apr 2014, 21:15
Number of books owned: 0
Country: Canada

Re: Original ~600pg/hr, very portable scanner now achieving ~900pg-1100pg/hr

Post by Mohib »

Ok I ran 3 focus tests to see the difference between manual focus and auto-focus.

TESTS:

All with iPhone 4s at 2x zoom (different zoom or cameras may yield different results).

a) 260 pages, imaged every 20 pages with focus lock set to the first page and left locked from start to finish.

b) 260 pages, imaged every 20 pages with auto-focus

c) 1 min test on auto-focus to see if I went at full speed could auto-focusing lock keep up.

RESULTS:

a) Marginal difference between first page (page 3) and last page (page 263) (full pages reduced to 25% and cropped portions at 100% attached)

b) No discernible difference between last page (page 3) and last page (page 263) and marginal difference with last page with focus lock. Overall, though auto focus seems sharper (even on first image which should be the same). (full pages reduced to 25% and cropped portions at 100% attached)

c) I did 25 pages in one min (=1500 pages/hour) and all are equally sharp so it looks like auto-focus can keep up with the speed.

CONCLUSIONS:

At 2x zoom it seems there's enough depth of field at the distance imaged (about 18") that on fixed focus there's hardly any difference between the first and last page of a 250 page book so perhaps it would best would be to lock focus in the middle of the book if you want to use manual focus. For larger books I would suggest placing a small piece of paper (say a business card) at pages 200, 400, etc. to remind you to re-set the focus for the next 200 pages at but reset it at page 300, 500 etc. so you get +/-100 pages (total 200) around each focus lock.

However, since it seems the iPhone 4s auto-focus can keep up (probably because it's electronic and not mechanical), there's no downside to leaving auto-focus on. Newer phones -- which have faster processors than the 4s (now 5+ years old) -- can probably auto-focus even faster.

This means despite the variable platen/camera distance with my scanner, while scanning I do not need to use the macro focus rail (which facilitated rapid mitigation of both focus and the image scale problems sufficiently to not be an issue) because smart-phone auto-focusing seems to be fast enough (and/or focus lock in the middle of the book is sufficient for a 250 page book, and so one would could use the above process for larger books) and the image scaling script, in this thread, normalises content scale. In other words, I can put my full attention to page turning just jogging the book back in view every 30-40 pages.

--------------------------------

FIXED FOCUS
IMG_1806 - page 3 - fixed focus (unchanged) - 25%.jpg
IMG_1806 - page 3 - fixed focus (unchanged) - 25%.jpg
IMG_1806 - page 3 - fixed focus (unchanged) - 25%.jpg (326.27 KiB) Viewed 28541 times
IMG_1819 - page 263 - fixed focus (unchanged) - 25%.jpg
IMG_1819 - page 263 - fixed focus (unchanged) - 25%.jpg
IMG_1819 - page 263 - fixed focus (unchanged) - 25%.jpg (302.01 KiB) Viewed 28541 times
IMG_1806 - page 3 - fixed focus (unchanged) - 100%.jpg
IMG_1806 - page 3 - fixed focus (unchanged) - 100%.jpg
IMG_1806 - page 3 - fixed focus (unchanged) - 100%.jpg (401.08 KiB) Viewed 28541 times
IMG_1819 - page 263 - fixed focus (unchanged) - 100%.jpg
IMG_1819 - page 263 - fixed focus (unchanged) - 100%.jpg
IMG_1819 - page 263 - fixed focus (unchanged) - 100%.jpg (343.63 KiB) Viewed 28541 times
--------------------------------

AUTO FOCUS
IMG_1820 - page 3 - auto focus - 25%.jpg
IMG_1820 - page 3 - auto focus - 25%.jpg
IMG_1820 - page 3 - auto focus - 25%.jpg (327.39 KiB) Viewed 28541 times
IMG_1833 - page 263 - auto focus - 25%.jpg
IMG_1833 - page 263 - auto focus - 25%.jpg
IMG_1833 - page 263 - auto focus - 25%.jpg (301.04 KiB) Viewed 28541 times
IMG_1820 - page 3 - auto focus - 100%.jpg
IMG_1820 - page 3 - auto focus - 100%.jpg
IMG_1820 - page 3 - auto focus - 100%.jpg (407.72 KiB) Viewed 28541 times
IMG_1833 - page 263 - auto focus - 100%.jpg
IMG_1833 - page 263 - auto focus - 100%.jpg
IMG_1833 - page 263 - auto focus - 100%.jpg (345.49 KiB) Viewed 28541 times
Last edited by Mohib on 10 Apr 2017, 11:25, edited 2 times in total.
User avatar
Mohib
Posts: 107
Joined: 05 Apr 2014, 21:15
Number of books owned: 0
Country: Canada

Re: Original ~600pg/hr, very portable scanner now achieving ~900pg-1100pg/hr

Post by Mohib »

I also finally figured out how to get rid of all the ugly hardware -- which has been evolving (see below) -- that holds the vertical/horizontal assembly together.

I had always wanted to use a threaded rod inside the vertical pipe, in the same way I had done with the horizontal pipe, but the problem was how to handle the two rods intersecting with each other and I could not figure out how to get around that ... until now! :) Using a threaded rod in the vertical pipe would also simplify construction and assembly as it's a lot easier than trying to cut a steel cable the right length to work and also assembling using a cable.

How did I manage it? Well for the last 3 years I've always thought I'd drill a 1/4" hole in the top of the T joint for the vertical threaded rod and of course that would intersect with the horizontal threaded rod. I could offset the hole a bit but that wouldn't look very nice. Then I realised the horizontal threaded rod has a lot play in where it can sit (i.e. it doesn't have to run down the centre line all the way and so can be offset without doing anything). It has the play because there's a lot of room in the T-joint and then it pops out through a washer with a big hole (not a 1/4" hole). So it struck me that if I used a double T-joint, then a vertical threaded rod would also have that play also (and not pop out the top in a 1/4" hole forcing it in place) and so each threaded rod could just be offset from their centre lines slightly (1/8") and pass each other without really looking like they are off set. It requires a little bit of pressure to flex them a little since my hand knobs fit inside the T-joint openings with little room to spare (and so tend to re-centre the rods), but now the whole support assembly looks very clean and assembling it is as simple as possible!

Here's the result.
DSC03515-side - 25% - grey.jpg
DSC03515-side - 25% - grey.jpg (244.58 KiB) Viewed 28541 times
DSC03523-light - 25% - grey.jpg
DSC03523-light - 25% - grey.jpg (295.19 KiB) Viewed 28541 times
DSC03524-light extension - 25% - grey.jpg
DSC03524-light extension - 25% - grey.jpg (310.24 KiB) Viewed 28541 times
DSC03528-back mount - 25% - grey.jpg
DSC03528-back mount - 25% - grey.jpg (257.35 KiB) Viewed 28541 times
DSC03545-twin T assembled - 25% - grey.jpg
DSC03545-twin T assembled - 25% - grey.jpg (295.35 KiB) Viewed 28541 times
DSC03548-full-post-dismantled - 25% - grey.jpg
DSC03548-full-post-dismantled - 25% - grey.jpg (386.05 KiB) Viewed 28541 times
DSC03556-top end of post dismantled - 25% - grey.jpg
DSC03556-top end of post dismantled - 25% - grey.jpg (368.52 KiB) Viewed 28541 times
DSC03572-platen dismantled - 25% - grey.jpg
DSC03572-platen dismantled - 25% - grey.jpg (248.41 KiB) Viewed 28541 times
Scanner 900 - Support assembly - evolution.jpg
User avatar
Mohib
Posts: 107
Joined: 05 Apr 2014, 21:15
Number of books owned: 0
Country: Canada

Re: Original ~600pg/hr, very portable scanner now achieving ~900pg-1100pg/hr

Post by Mohib »

I did a more comprehensive depth of field test to see how well focused images remain on fixed focus.

THE TEST AT 1.5X zoom , iPhone 4S
--------------------
1) Page 501 is the reference page. For fixed focus, focus was set on this page. For auto focus, focus was of course automatically recalculated on each page.

2) Focus check as pages are further away from camera relative to p501:
Images taken at +50 pages (p551), +100 (p601), +150 (p651) and +200 (p701) on both fixed and auto.

3) Focus check as pages are closer to camera relative to p501;
Images taken at -50 pages (p451), -100 (p401), -150 (p351) and -200 (p301) on both fixed and auto.

Side by side images below at 100% original size.

CONCLUSION
------------------
Frankly, I can't see any changes and focus through out the 400 page range stays pretty much the same and pretty much identical to auto-focus. I'm not sure if the depth of field changes much at 2.5x zoom, but when I get time I'll repeat the test.

Other smart-phones/cameras may, of course, give different results from the iPhone 4S I used.
Depth of Field test - 1.5x zoom - auto v fixed - +0.jp
Depth of Field test - 1.5x zoom - auto v fixed - +0.jp
Depth of Field test - 1.5x zoom - auto v fixed - +0.jpg (653.42 KiB) Viewed 28484 times
Depth of Field test - 1.5x zoom - auto v fixed - +50.jpg
Depth of Field test - 1.5x zoom - auto v fixed - +50.jpg
Depth of Field test - 1.5x zoom - auto v fixed - +50.jpg (651.32 KiB) Viewed 28484 times
Depth of Field test - 1.5x zoom - auto v fixed - +100.jpg
Depth of Field test - 1.5x zoom - auto v fixed - +100.jpg
Depth of Field test - 1.5x zoom - auto v fixed - +100.jpg (664.97 KiB) Viewed 28484 times
Depth of Field test - 1.5x zoom - auto v fixed - +150.jpg
Depth of Field test - 1.5x zoom - auto v fixed - +150.jpg
Depth of Field test - 1.5x zoom - auto v fixed - +150.jpg (663.42 KiB) Viewed 28484 times
Depth of Field test - 1.5x zoom - auto v fixed - +200.jpg
Depth of Field test - 1.5x zoom - auto v fixed - +200.jpg
Depth of Field test - 1.5x zoom - auto v fixed - +200.jpg (641.43 KiB) Viewed 28484 times
Depth of Field test - 1.5x zoom - auto v fixed - -50.jpg
Depth of Field test - 1.5x zoom - auto v fixed - -50.jpg
Depth of Field test - 1.5x zoom - auto v fixed - -50.jpg (649.86 KiB) Viewed 28484 times
Depth of Field test - 1.5x zoom - auto v fixed - -100.jpg
Depth of Field test - 1.5x zoom - auto v fixed - -100.jpg
Depth of Field test - 1.5x zoom - auto v fixed - -100.jpg (640.4 KiB) Viewed 28484 times
Depth of Field test - 1.5x zoom - auto v fixed - -150.jpg
Depth of Field test - 1.5x zoom - auto v fixed - -150.jpg
Depth of Field test - 1.5x zoom - auto v fixed - -150.jpg (626.83 KiB) Viewed 28484 times
Depth of Field test - 1.5x zoom - auto v fixed - -200.jpg
Depth of Field test - 1.5x zoom - auto v fixed - -200.jpg
Depth of Field test - 1.5x zoom - auto v fixed - -200.jpg (651.81 KiB) Viewed 28484 times
Konos93a
Posts: 195
Joined: 19 Sep 2016, 10:00
E-book readers owned: kobo aura,kindle 1,kindle pw3,pocketbook inkpad 2
Number of books owned: 3000
Country: greece

Re: Original ~600pg/hr, very portable scanner now achieving ~900pg-1100pg/hr

Post by Konos93a »

Mohib wrote: 06 Apr 2017, 23:52

Firstly let me explain the "problem." It's not the platen that's the real problem but the book drifting left and/or tilting a bit. This seems to be generally caused by pressing the platen into the spine. This causes the book to nudge left. Now this is not as big a problem as it seems because, firstly, you don't need to zoom too tight into the page so can tolerate some drift and, secondly, you can "roll the spine" right to shove the page into the view for several pages. So basically you only have to move the book back every 30 pages or so and it just takes a few seconds to do, so it's not a big delay.
but you can solve this movement with Deskew images in abbyfinereader ,or if is it driff in the capture time reducing the capture speed or with a sticker down of the book, right?

about the script that you use .it is used to change pixels of every page with a scale .so if the pages be cropped the results ought to have the same pixel size with the same wight of context right?
i ask you cause i figure out that if a broom handle cut i the middle . with some screws ,a protractor and a spirit level someone can make a perfect 100 degrees for v scanner. now with my scanner i have,4 degrees the v is not in the middle and the result is that one camera is nearest in the opposite glass than the other. so the half images when i crop are larger than others.i can solve that with scale in scantaylor. do u know any way i can resize them in the same pixels?

you know in the age of tweeter a north-american dude its rare to be so analytic .thanks a lot
User avatar
Mohib
Posts: 107
Joined: 05 Apr 2014, 21:15
Number of books owned: 0
Country: Canada

Re: Original ~600pg/hr, very portable scanner now achieving ~900pg-1100pg/hr

Post by Mohib »

Konos93a wrote: 14 Apr 2017, 19:59 but you can solve this movement with Deskew images in abbyfinereader ,or if is it driff in the capture time reducing the capture speed or with a sticker down of the book, right?
No, this is not about deskew, it's about the book actually slowly moving left on the table so it's shifting **out** of the camera view, so left margin is not even in the image.
Konos93a wrote: 14 Apr 2017, 19:59 or if is it driff in the capture time reducing the capture speed or with a sticker down of the book, right?
Yes, you need to "fix" the book to the table. It's not a very big problem and only takes a fraction of a second to push the book back.
Konos93a wrote: 14 Apr 2017, 19:59 about the script that you use .it is used to change pixels of every page with a scale .so if the pages be cropped the results ought to have the same pixel size with the same wight of context right?
Yes the script is used **after** scanning, and **before** using Scan Tailor or any other software so that text on all pages is the same size when you start processing.
Konos93a wrote: 14 Apr 2017, 19:59 i can solve that with scale in scantaylor. do u know any way i can resize them in the same pixels?
I'm not sure if Scan Tailor can do this -- unless the newer test versions have a feature for it. But, otherwise you can just change the calculation in the script to process your images in a similar way (instead of changing the scale for each page, just fix the scale for all pages).
Mohib wrote: 14 Apr 2017, 13:20 you know in the age of tweeter a north-american dude its rare to be so analytic .
LOL! Thanks, but I find everyone on this forum very analytic -- perhaps because what they build either works or doesn't!
jlev
Posts: 24
Joined: 04 Mar 2014, 00:52
Number of books owned: 0
Country: USA
Contact:

Re: Original ~600pg/hr, very portable scanner now achieving ~900pg-1100pg/hr

Post by jlev »

Mohib, huge congratulations on such a simplified, elegant design! Your documentation is inspiring, and the results you're getting look great. I especially like how you've settled on using an iPhone 4 -- new enough that the camera is good, but old enough that it's (I presume) inexpensive to pick up used.

I have two suggestions for the issue of pages getting farther from the camera as the scan process progresses across a book:
  • If the book were elevated at even a slight angle, and the camera pointed at it (at an angle, rather than perpendicular), the cradle could be made to slide laterally to keep the book's top page in a fixed position to the camera. This could be accomplished even by using something like a line on the table to mark a fixed spot; the operator could then watch to keep the spine of the book aligned with that line.
  • Would using something like Voussoir (Forum post here; GitHub repo.
    here) be useful in this case? I just posted it this evening; it's open-source (permissively-licensed),
    and watches for markers in an image to automatically de-keystone and crop (and thus normalize) images based on the location of those markers. That sort of software approach (using fiducial markers) could be a good alternative or complement to the approaches you've reported here.
Do either of those approaches sound useful or feasible to you?
User avatar
Mohib
Posts: 107
Joined: 05 Apr 2014, 21:15
Number of books owned: 0
Country: Canada

Re: Original ~600pg/hr, very portable scanner now achieving ~900pg-1100pg/hr

Post by Mohib »

jlev wrote: 15 Apr 2017, 22:53 Mohib, huge congratulations on such a simplified, elegant design! Your documentation is inspiring, and the results you're getting look great.
Thanks!
jlev wrote: 15 Apr 2017, 22:53 I especially like how you've settled on using an iPhone 4 -- new enough that the camera is good, but old enough that it's (I presume) inexpensive to pick up used.
Actually this is partially right. I did pick up my iPhone 4S used, but not for this project. It was only after I got I realised smart-phones could be a break through for my scanner by:

a) eliminating a lot of camera specific construction for the trigger (assuming you didn't have a camera with remote trigger -- I didn't) and
b) eliminating the auto-focus delay (my point-and-shoot didn't have focus lock)

and would generally provide a more user friendly, more efficient setup.

That being said, the iPhone 4s has its limitations. The resolution is really not high enough to image small print or newspapers and then OCR because, even though I'm using Camera+ Free's Fine Mode and ISO 50, the images still have a fair amount of artefacts around the text. It's not affecting OCR (after Scan Tailor processing) of regular sized text at all, but the image quality is probably not good enough for those wanting better quality scans in the first place (like those reasonable point-and-shoots give).

However, for creating clean, OCRd copies of typical hardback and paperbacks, the iPhone 4s works perfectly and I'm sure many/most already have newer model smartphones that give higher resolution and better quality images (and better optics) which they can use, so really don't need to get another camera. I don't have one so can't test this.
jlev wrote: 15 Apr 2017, 22:53 I have two suggestions for the issue of pages getting farther from the camera as the scan process progresses across a book:
Thanks for the suggestions, and I've elaborated my agreement to your general suggestion of fiducial markers in your Impost Scanner thread here: viewtopic.php?f=14&t=3411#p20590. I added calibration pages in dtic's script, suggested their use when I first posted my scanner and note others have mentioned they also use calibration pages to automatically perform various corrections and/or normalisations to the images.

WRT to your suggestion, if I understand it correctly, there would be a hardware component for the book and then a post-processing phase to correct a deliberately introduced error. Leaving aside as to whether the hardware solution would work (which it might very well do), my experience has shown any additional components are distracting and break the smooth tilt-flip-click scanning cycle rhythm and slow scanning speed down. That being said, dtic's post-processing script (which I simply made more user friendly for regular use) totally corrects the problem without the additional hardware and without any changes to the scanning cycle.
User avatar
Mohib
Posts: 107
Joined: 05 Apr 2014, 21:15
Number of books owned: 0
Country: Canada

Re: Original ~600pg/hr, very portable scanner now achieving ~900pg-1100pg/hr

Post by Mohib »

Well I've finally got a decent video of the scanner -- which I'm calling the TIFLIC Book Scanner (for tilt, flip, click) v2.1 -- on YouTube, in glorious HD, here:
https://www.youtube.com/watch?v=MU6H7dEzTMY

And in 360p on Vimeo here:
https://vimeo.com/213003462

The video includes 3 different 1 minute tests:

a) hardback (25 pages; 1500 pages/hour),
b) paperback (22 pages, showing no damage to the spine; 1320 pages/hour),
c) large format book (15 pages; 900 pages/hour).

Here's a graph of the number of seconds taken for each page for each of the tests. As is evident (and from the video), once you've got your rhythm, scanning proceeds very smoothly and rapidly, and why I've noticed any extra hardware that interferes with your rhythm reduces the speed dramatically. The long delay at the start for each is due to starting the timer and getting the book ready to start scanning. I then commence with a page turn before triggering the camera, so it's a complete tilt-flip-click cycle on the first page.
camera trigger time indexes - graph.jpg
camera trigger time indexes - graph.jpg (83.75 KiB) Viewed 28364 times
Also included in the video are lots of pictures of:

d) results (original scan, Scan Tailor output, OCR output),
e) results from books with fragile bindings and tight gutters,
f) construction pictures.

Attached is the updated parts list and construction diagrams for v2.1 of the build. Enjoy!
User avatar
Mohib
Posts: 107
Joined: 05 Apr 2014, 21:15
Number of books owned: 0
Country: Canada

Re: Original ~600pg/hr, very portable scanner now achieving ~900pg-1100pg/hr

Post by Mohib »

I decided to do some tests to compare original images, Scan Tailor output and OCR results between:

- an iPhone 4s
- an Olympus VG-130 D-71 point and shoot
- a Sony DSC RX100

I wasn't concerned too much about lighting and white balance as I was with other aspects of image quality (artifacts, etc.).

My purpose was to see how much lower the iPhone image quality was compared to a cheap point and shoot and a high-end, pro/am compact. My cameras are older and not WiFi or NFC enabled so I used an much improved (and still evolving) version of the mechanical trigger (using the bike-break method) I originally had developed. However, almost all newer cameras (even cheap point and shoots) are WiFi/NFC enabled and can be remotely controlled from a smart phone.

As an aside, I tested Sony's PlayMemories app with an A5000 and it worked very well. However, there was a delay of about 4 seconds after triggering the shutter while the app seem to recycle itself before another shot could be taken (and that was after turning off all the download, preview and other functions). I'm not sure what caused the delay and couldn't get rid of it, nor do I know if other manufacturer's smart-phone apps have the same problem, but if that's what it is, then it appears there would be a hard limit on speed with Sony cameras at 15 pages/min (900 per hour). Perhaps a custom written app could avoid the delay.

So below are 100% crops of the original images, followed by the Scan Tailor output, and two 3 way comparisons of Abbyy OCR generated from the original images and from the Scan Tailor output.

As expected the iPhone 4s original image quality is clearly not up to the Olympus or Sony, and the Sony clearly better than the Olympus.

However, Scan Tailor output had marginal differences between the three.

But when it came OCR output, there was no difference in results and all produced perfect OCR, save for a 1 character error -- whether OCR was done of the original scans or of the Scan Tailor output.
.
.
iPhone 4s - 8MB - Fine - IMG_2213 - crop.jpg
iPhone 4s - 8MB - Fine - IMG_2213 - crop.jpg
iPhone 4s - 8MB - Fine - IMG_2213 - crop.jpg (412.45 KiB) Viewed 28239 times
.
.
Olympus VG-130 D-71 - 8MB - Fine - P4240251 +0 - crop.jpg
Olympus VG-130 D-71 - 8MB - Fine - P4240251 +0 - crop.jpg
Olympus VG-130 D-71 - 8MB - Fine - P4240251 +0 - crop.jpg (441.57 KiB) Viewed 28239 times
.
.
Sony DSCRX100 - 10MB - Fine - DSC03687 +0 - crop.jpg
Sony DSCRX100 - 10MB - Fine - DSC03687 +0 - crop.jpg
Sony DSCRX100 - 10MB - Fine - DSC03687 +0 - crop.jpg (439.9 KiB) Viewed 28239 times
.
.
iPhone 4s v Olympus VG-130 D-71 v Sony DSCRX100 - JPGs.png
iPhone 4s v Olympus VG-130 D-71 v Sony DSCRX100 - JPGs.png
.
.
iPhone 4s - 8MB - Fine - IMG_2213 - Scan Tailor - crop - tif converted to 100% jpg.jpg
iPhone 4s - 8MB - Fine - IMG_2213 - Scan Tailor - crop - tif converted to 100% jpg.jpg
iPhone 4s - 8MB - Fine - IMG_2213 - Scan Tailor - crop - tif converted to 100% jpg.jpg (321.94 KiB) Viewed 28239 times
.
.
Olympus VG-130 D-71 - 8MB - Fine - P4240251 +0 - Scan Tailor - crop - tif converted to 100% jpg.jpg
Olympus VG-130 D-71 - 8MB - Fine - P4240251 +0 - Scan Tailor - crop - tif converted to 100% jpg.jpg
Olympus VG-130 D-71 - 8MB - Fine - P4240251 +0 - Scan Tailor - crop - tif converted to 100% jpg.jpg (302.84 KiB) Viewed 28239 times
.
.
Sony DSCRX100 - 10MB - Fine - DSC03687 +0 - Scan Tailor - crop - tif converted to 100% jpg.jpg
Sony DSCRX100 - 10MB - Fine - DSC03687 +0 - Scan Tailor - crop - tif converted to 100% jpg.jpg
Sony DSCRX100 - 10MB - Fine - DSC03687 +0 - Scan Tailor - crop - tif converted to 100% jpg.jpg (291.18 KiB) Viewed 28239 times
.
.
iPhone 4s v Olympus VG-130 D-71 v Sony DSCRX100 - Scan Tailor.png
iPhone 4s v Olympus VG-130 D-71 v Sony DSCRX100 - Scan Tailor.png
Post Reply