Original ~600pg/hr, very portable scanner now achieving ~900pg-1100pg/hr

Built a scanner? Started to build a scanner? Record your progress here. Doesn't need to be a whole scanner - triggers and other parts are fine. Commercial scanners are fine too.

Moderator: peterZ

Post Reply
User avatar
Mohib
Posts: 107
Joined: 05 Apr 2014, 21:15
Number of books owned: 0
Country: Canada

Re: Original ~600pg/hr, very portable scanner now achieving ~900pg-1100pg/hr

Post by Mohib »

Mohib wrote: 27 Apr 2017, 20:22 As an aside, I tested Sony's PlayMemories app with an A5000 and it worked very well. However, there was a delay of about 4 seconds after triggering the shutter while the app seem to recycle itself before another shot could be taken (and that was after turning off all the download, preview and other functions). I'm not sure what caused the delay and couldn't get rid of it, nor do I know if other manufacturer's smart-phone apps have the same problem, but if that's what it is, then it appears there would be a hard limit on speed with Sony cameras at 15 pages/min (900 per hour). Perhaps a custom written app could avoid the delay.
After some googling, I came across this post by a PlayMemories user which indicates that it is possible to shoot time pics with the Timelapse App at 1 second intervals:
Now I just use timelapse to take 60 shots 1 second apart, then just pose and play with my son, hands free. The only issue is that the app will lock the focus. -- https://www.dpreview.com/forums/reply?parent=55457946
So, given this, there doesn't seem to be a technical limitation and perhaps the PlayMemories free app doesn't allow faster shooting, there's a limit with the A5000 or some other issue causing the delay.
User avatar
Mohib
Posts: 107
Joined: 05 Apr 2014, 21:15
Number of books owned: 0
Country: Canada

Re: Original ~600pg/hr, very portable scanner now achieving ~900pg-1100pg/hr

Post by Mohib »

In the spirit of wondering why the obvious doesn't strike you before it does, I've updated the image scaling script to also number the pages so the left and right pages will interleave correctly once both sets of images are copied to a single directory. This avoids using extra software (like the one I used which I described here: viewtopic.php?f=14&t=3401#p20511) and the inevitable errors with any manual processing and fiddling.

Assumptions about this renumbering script to understand since it was designed to process pages from a single camera scanner.

a) It assumes right hand pages (odd numbered) are scanned from page 1 up to x (i.e. from start of the book to the end, up to a maximum of 9997) and are in their own directory.

b) It assumes left hand pages (even numbered) were scanned from x+1 down to 2 (i.e from the end of the book to the start) and are in their own directory.

c) It checks there are the same number of files in left and right directories (if both are specified), i.e. it assumes you've scanned all pages from 1 up to x and x+1 down to 2.

d) If you don't want to perform image scaling and just number the pages, enter the same pixel value for the first and last pages (i.e. 100% scale).

Code: Select all

#singleinstance, force

; Normalize Scanned Pages Scale V1.3
;
; Autohotkey script using graphics magic library
;
; 1) Rescale pages from a book with varying camera/platen distance (i.e. difference in distance from camera to first page and last page scanned is thickness of book)
; to normalize page scale so content on each page is same as first page (i.e. last page is scaled the most amount).
;
; 2) File modified date and time of each image must increase from first (largest) to last (smallest) page scanned.
; 
; a) so for right hand pages (odd numbered), largest and first scanned page should be page 1 and smallest and last scanned should be page x.
; b) and for left hang pages (even numbered), largest and first scanned page should be page x+1 and smallest and last scanned should be page 2.
;
; 3) ALL pages must be scanned (including blank) with equal number of pages in separate directories for left and right hand pages.
; 
; 4) Output is placed in a subdirectory of each (calibration pages are not processed or copied).
; 
; 5) Filenames of righthand (odd) and lefthand (even) page are also numbered as follows:
;
; a) right hand pages (odd numbered) are scanned from page 1 up to x (i.e. from start of the book to the end, up to a maximum of 9997).
; b) left hand pages (even numbered) were scanned from x+1 down to 2 (i.e from the end of the book to the start).
; c) same number of files in left and right directories (if both are specified), i.e it assumes you've scanned all pages from 1 up to x and x+1 down to 2.
; d) to skip scaling and just number pages, enter the same pixel value for the first and last pages (i.e. 100% scale).
; 
; Change log:
; V1.3 -- Added -- Insert page numbering of righthand (odd) and lefthand (even) pages into file name
;


; Full path to graphicsmagick gm.exe
; Example: C:\Program Files\GraphicsMagick-1.3.25-Q8\gm.exe
gm := "C:\Program Files (x86)\GraphicsMagick-1.3.25-Q16\gm.exe"

if !FileExist(gm) {
 MsgBox ,,ERROR,graphicsmagick's gm.exe file not found here:`n`n%gm%
 exitapp
}

GetImageDirectories:

; Get path to folder with right and/or right hand page jpg images to process

rightOutput := "\right-adjusted"
leftOutput := "\left-adjusted"
rightImagesFolder := ""
leftImagesFolder := ""

FileSelectFolder, rightImagesFolder,,2,Choose folder with all RIGHT hand pages (or enter cancel to skip processing right hand pages).
FileSelectFolder, leftImagesFolder,,2,Choose folder with all LEFT hand pages (or enter cancel to skip processing left hand pages).

If (rightImagesFolder = "" and leftImagesFolder = "") {
  ExitApp
}

If (rightImagesFolder = leftImagesFolder) {
  MsgBox ,,ERROR,You have chosen the same directory for both left and right hand pages. This is not allowed. Please choose the directories again.
  goto GetImageDirectories
}

If (rightImagesFolder <> "") {
 if !FileExist(rightImagesFolder) {
	MsgBox ,,ERROR,Folder given for right hand images does not exist. Please choose both image directories again.
   goto GetImageDirectories
 }
 
 ; Femove trailing slashes
 rightImagesFolder := RegExReplace(rightImagesFolder, "\\$")

 if FileExist(rightImagesFolder . rightOutput) {
  ; need to specify recurse else FileRemoveDir doesn't work.
  FileRemoveDir, %rightImagesFolder%%rightOutput%, 1
 }
 FileCreateDir, %rightImagesFolder%%rightOutput%
 
 Loop, files, %rightImagesFolder%\*.jpg 
  {
   noOfRightFiles++
  }

}

If (leftImagesFolder <> "") {
 if !FileExist(leftImagesFolder) {
	MsgBox ,,ERROR,Folder given for left hand images does not exist. Please choose both image directories again.
   goto GetImageDirectories
 }
 
 ; Femove trailing slashes
 leftImagesFolder := RegExReplace(leftImagesFolder, "\\$")

 if FileExist(leftImagesFolder . leftOutput) {
  ; need to specify recurse else FileRemoveDir doesn't work.
  FileRemoveDir, %leftImagesFolder%%leftOutput%, 1
 }
 FileCreateDir, %leftImagesFolder%%leftOutput%

 Loop, files, %leftImagesFolder%\*.jpg 
  {
   noOfLeftFiles++
  }

}

; check number of left and right files are the same
If (noOfRightFiles <> noOfLeftFiles And leftImagesFolder <> "" And rightImagesFolder <> "") {
  MsgBox ,,ERROR, The left and right directories don't have the same number of files`n`nRight directory: %noOfRightFiles%`nLeft directory: %noOfLeftFiles%
  ExitApp
}

; Get number of pixels of calibration image (6" ruler) placed on first page and last page of book 
GetCalibrationSizes:
firstImageSize := 0
While firstImageSize = 0 Or firstImageSize = "" {
 InputBox, firstImageSize, First Calibration Page, Enter number of pixels taken by a 6`" ruler placed on the first page of the book (first callibration page).,,450,150
 if (ErrorLevel = 1) {
   ExitApp
  }
If (firstImageSize = 0 Or firstImageSize = "") {
  MsgBox ,,ERROR,Zero or blank is not allowed
  }
 }

lastImageSize :=  0
While lastImageSize = 0 Or lastImageSize = "" {
 InputBox, lastImageSize, Last Calibration Page, Enter number of pixels taken by a 6`" ruler placed on the last page of the book (second callibration page). Enter same number as the first page to bypass scaling and only number the files.,,450,150
 if (ErrorLevel = 1) {
   ExitApp
  }
 If (lastImageSize = 0 Or lastImageSize = "") {
  MsgBox ,,ERROR,Zero or blank is not allowed
  }
 If (firstImageSize < lastImageSize) {
  MsgBox ,,ERROR,Last/smallest image is greater than first/largest image. Please enter the values again.
  goto GetCalibrationSizes
 }
 }

; Calculate scale difference (percentage) between first and last pages
totalScaleFactor := ( firstImageSize / lastImageSize ) * 100

; 8228=4+32+8192
MsgBox, 8228, WARNING, Processing images as follows`n`nRight hand images: %rightImagesFolder%`n`nLeft hand images: %leftImagesFolder%`n`nPixel count (first image): %firstImageSize%`nPixel count (last image): %lastImageSize%`nFirst to last image scale factor: %totalScaleFactor%`n`nOk to proceed?
IfMsgBox, No
 ExitApp

If (rightImagesFolder <> "") {
 ProcessDirectory(rightImagesFolder, rightImagesFolder . rightOutput, "RIGHT")
}
If (leftImagesFolder <> "") {
 ProcessDirectory(leftImagesFolder, leftImagesFolder . leftOutput, "LEFT")
}

exitapp

ProcessDirectory(currentInputImagesFolder, currentOutputImagesFolder, currentPageHand)
{

 global totalScaleFactor
 global gm

 ; noOfFiles number of pages in directory to scale can be distributed across all pages
 ; Create file list sorted by modified date/time so script can be used after renaming and is not depenedent on original image file name.
 ; Using time stamp also ensures processing is not random if directory listing is not alphabetic
 fileList := ""
 noOfFiles := 0
 Loop, files, %currentInputImagesFolder%\*.jpg 
 {
  ; ignore first calibration image
  if (noOfFiles <> 0) {
   fileList = %FileList%%A_LoopFileTimeModified%`t%A_LoopFileName%`n
  }
  noOfFiles++
 }

 ; remove last calibration page from file list (search for second `n)
 fileList := Substr(fileList,1,Instr(fileList,"`n",false,-1,1))
 ; reduce count by 2 becuse 2 calibration images removed
 noOfFiles := noOfFiles - 2
 
 ; Sort file list (date is appeneded to front of each entry of FileList - so list will be sorted by time first and then by file name)
 ; This lets many files with the same time (i.e. seconds didn't change) still sort in the right order and not dependent on
 ; Dir listings that may be in random order. Timestamp is not really needed and can be removed.
 Sort, FileList

 ; Initalize the progress bar style
 Progress, B2 M FM10 FS10 WM10 WS10 W700, Starting, PROCESSING %currentPageHand% HAND PAGES -- PRESS ESCAPE TO ABORT
 
 If (currentPageHand = "RIGHT") {
  currentFileNumber := 1 
  fileNumberIncrement := 2
 } Else {
  currentFileNumber := noOfFiles * 2
  fileNumberIncrement := -2
 }
 
 Loop, Parse, fileList, `n
 {
  
  ; Omit the last linefeed (blank item) at the end of the list.
  If (A_LoopField = "") {
   break
  }
  
  ; Split into two parts at the tab char.
  StringSplit, FileItem, A_LoopField, %A_Tab%  

  currentPageNumber := Format("PAGE#{:04}", currentFileNumber)
  
  ; ignore first image coz not to be scaled
  If (a_index = 1) {
   FileCopy, %currentInputImagesFolder%\%FileItem2%, %currentOutputImagesFolder%\%FileItem2%
  } else {
   If (totalScaleFactor <> 100) {
    thisImageScaleFactor := (totalScaleFactor - 100) * (a_index / noOfFiles) + 100
    amountDone := floor(a_index/noOfFiles*100)
    ; Update the progress bar value
    Progress, %amountDone%
    ; Display the progress bar at the new value with new text
    Progress, ,Processing: %a_index%/%noOfFiles% -- Scale: %thisImageScaleFactor% -- %currentPageNumber% -- %FileItem2%
    RunWait "%gm%" convert "%currentInputImagesFolder%\%FileItem2%" -resize %thisImageScaleFactor%`% -quality 100 "%currentOutputImagesFolder%\%FileItem2%",,hide
   } else {
    FileCopy %currentInputImagesFolder%\%FileItem2%, %currentOutputImagesFolder%\%FileItem2%
   } 
  }
 
  FileMove %currentOutputImagesFolder%\%FileItem2%,%currentOutputImagesFolder%\%currentPageNumber% -- %FileItem2%
  currentFileNumber := currentFileNumber + fileNumberIncrement

  ; Just make sure we actually exit when done, just to make sure  last file is not processed twice in case don't exit with the blank above (as that seems to fail sometimes)
  if (a_index = noOfFiles) {
   break
  }
 }

 Progress, Off
 return
}

; Allow escape to cancel
Escape::ExitApp
Konos93a
Posts: 195
Joined: 19 Sep 2016, 10:00
E-book readers owned: kobo aura,kindle 1,kindle pw3,pocketbook inkpad 2
Number of books owned: 3000
Country: greece

Re: Original ~600pg/hr, very portable scanner now achieving ~900pg-1100pg/hr

Post by Konos93a »

Normalize Scanned Pages Scale V1.3

Can you please ,when you have free time to make a video tutorial about your script .how to use it , how to install and how to face with errors ?

and if you can uploading a zip file with 10 pages for processing of your work
User avatar
Mohib
Posts: 107
Joined: 05 Apr 2014, 21:15
Number of books owned: 0
Country: Canada

Re: Original ~600pg/hr, very portable scanner now achieving ~900pg-1100pg/hr

Post by Mohib »

I've attached the instructions (the script file is in the PDF).
.
Image Scale-Page Numbering Script instructions.pdf
(1.25 MiB) Downloaded 467 times
.
.
And here's a before (from my 2014 post about this issue here: viewtopic.php?f=14&t=3007#p17556 ) and after comparison of two facing pages after running the script and processing in ScanTailor (no manual fiddling done). As expected, the pages are now the same scale (i.e. page 2's content is not smaller).
.
.
BEFORE
BEFORE
A Short History of the Ismailis - p 010 v p 011 - 35%.jpg (390.28 KiB) Viewed 11063 times
.
.
AFTER
AFTER
Facing page scale comparison -- PAGE#0010 -- P000255 v PAGE#0011 -- P000007 - 50%.jpg (456.45 KiB) Viewed 11063 times
.
Konos93a
Posts: 195
Joined: 19 Sep 2016, 10:00
E-book readers owned: kobo aura,kindle 1,kindle pw3,pocketbook inkpad 2
Number of books owned: 3000
Country: greece

Re: Original ~600pg/hr, very portable scanner now achieving ~900pg-1100pg/hr

Post by Konos93a »

aw fail ,fail,fail

find out gm was installed on program files, not program files *86 :oops: :oops: :oops:

anyway thnks alot i will try someday your method . seems less complicated than other scanners and more accessible due phones .

have a great day
User avatar
Mohib
Posts: 107
Joined: 05 Apr 2014, 21:15
Number of books owned: 0
Country: Canada

Re: Original ~600pg/hr, very portable scanner now achieving ~900pg-1100pg/hr

Post by Mohib »

Konos93a wrote: 14 May 2017, 06:47 aw fail ,fail,fail
Ah yes, sorry about that. Attached are better instructions with typos fixed! :)

The script is attached INSIDE the PDF (you'll find it on the "Attachments" panel on the left).
.
.
Image Scale-Page Numbering Script instructions.pdf
(1.35 MiB) Downloaded 453 times
User avatar
Mohib
Posts: 107
Joined: 05 Apr 2014, 21:15
Number of books owned: 0
Country: Canada

Re: Original ~600pg/hr, very portable scanner now achieving ~900pg-1100pg/hr

Post by Mohib »

I was asked how to get the cable around the second cross-dowel (as that's not clear in my original or new docs), so thought I'd share an animated GIF made from some additional pics I have taken (eventually for a video) to show:

a) how the platen handle and cable is assembled and
b) how the vertical post is assembled.

For the platen handle, as you assemble it, make sure all the pipes are pushed firmly into the elbows, so they are not taking extra length away from the cable (and make it appear too short).

Then, to get the 2nd cross-dowel through the second loop in the cable, assuming you've got the cable the right length, thread a thin, strong string/ribbon (I've got a strong piece of ribbon in the pics) through the loop, finish assembling it all (again pushing hard so everything is firmly seated all the way in), and then pull hard on the string/ribbon. That will stretch the cable to its full length and then, while looking through the hole where the cross dowel goes, get the loop lined up with the hole and push the cross dowel through. It takes a bit of fiddling but I think you'll see what I'm getting at once you try.
.
.
PLATEN--TIME-LAPSE-ASSEMBLY.gif
PLATEN--TIME-LAPSE-ASSEMBLY.gif (1.01 MiB) Viewed 11024 times
.
.
SUPPORT--TIME-LAPSE-ASSEMBLY.gif
SUPPORT--TIME-LAPSE-ASSEMBLY.gif (889.3 KiB) Viewed 11024 times
Last edited by Mohib on 15 May 2017, 22:56, edited 1 time in total.
dpc
Posts: 379
Joined: 01 Apr 2011, 18:05
Number of books owned: 0
Location: Issaquah, WA

Re: Original ~600pg/hr, very portable scanner now achieving ~900pg-1100pg/hr

Post by dpc »

What's the purpose of that cable that's threaded through the PVC tubing? Is that just to hold the individual pieces together?
User avatar
Mohib
Posts: 107
Joined: 05 Apr 2014, 21:15
Number of books owned: 0
Country: Canada

Re: Original ~600pg/hr, very portable scanner now achieving ~900pg-1100pg/hr

Post by Mohib »

Yes, to avoid gluing parts so they can be reused with a larger platen (i.e. a longer cross bar and cable is all that's needed) and also let me remove the break leaver since it's not stuck in the platen handle (which is why in my current incarnation with the blue-tooth trigger the break lever is gone!).

Although, I think if you glue everything except the 90 degree elbows and the horizontal cross bar (i.e. the piece you grip with your hand), the cable would not be needed to hold it all together, but still let it be dismantled to remove the brake leaver or use a larger platen).

Mainly, I didn't glue anything because I wanted to

a) avoid having to ensure everything is perfectly square for gluing (because I can never do it properly! :) ), and

b) I wanted to be able to experiment with various configurations, handle sizes (which I reduced by 1" from my original version when I moved to the 45 degree elbows), platen sizes, vertical post heights (which turned out to be very useful and let me do this: viewtopic.php?f=14&t=3411#p20590 ), horizontal post length (which I actually extend when using a lamp to keep it's reflection off the platen), etc..
User avatar
Mohib
Posts: 107
Joined: 05 Apr 2014, 21:15
Number of books owned: 0
Country: Canada

Re: Original ~600pg/hr, very portable scanner now achieving ~900pg-1100pg/hr

Post by Mohib »

Well I finally decided to download and try ScanTailor 64bit Experimental (I know, big mistake to have not done this long ago) and wow, what a difference in speed and quality!

More importantly, I now understand what Konos93a was getting at when he mentioned "match size by scaling" way back here:
Mohib wrote: 08 Apr 2017, 10:54
Konos93a wrote: 08 Apr 2017, 06:05 after you take photos you process with scantaylor and "match size by scaling"
is that enough?
Scan Tailor **does not** fix the scaling problem as far as I've seen (I'm using v0.9.11.1 of Scan Tailor) and I've been discussing about how it **could** fix the problem. But the script in my last post, updated from the original version kindly provided by dtic, **does solve the problem** from what I can see.
and had I just asked "What's that?" it would have saved all the trouble of developing the image-scaling script because Tulon was ahead of me (or because of me, I'm not sure when this feature was added) and had implemented the exact content scaling feature that I had been hoping for in ScanTailor Experimental. So this is much appreciated and makes things much simpler for people using TIFLIC type scanners, which don't have a fixed platen/camera distance. Thanks Tulon!

So below is a test of the same two pages (from the same scans I'd taken in 2014), but using ScanTailor Experimental's "match size by scaling" on all the content pages of the book. The few others pages (title, copyright and what not) were set to use the normal, "match by growing margins" or "don't match size." As for the scaling, the results are basically the same as using the image scaling script, but the quality of the text (I used 2x Resolution Enhancement) is much smoother and of course, one extra step of processing (and image degradation) is saved.

The script is still useful for doing the page numbering, and so for those using ScanTailor Experimental (and don't need the image scaling parts of the script), or who just want page numbering, I've made a new simplified script without the image scaling parts that doesn't need GraphicsMagick. Both scripts are inside the PDF with instructions:
. .
.
ScanTailor Margins - match size by scaling.png
ScanTailor Margins - match size by scaling.png (21.67 KiB) Viewed 10988 times
.
.
A Short History of the Ismailis - p 010 v p 011 - ScanTailor 64bit Exp - 25%.jpg
A Short History of the Ismailis - p 010 v p 011 - ScanTailor 64bit Exp - 25%.jpg (443.6 KiB) Viewed 10988 times
Last edited by Mohib on 16 May 2017, 18:12, edited 1 time in total.
Post Reply