Page 1 of 1

New OS X Post-processing Software

Posted: 14 Dec 2012, 18:19
by Shaknum
Paginator is a new post-processing package I’m currently putting together. This basis for the project is to use QR-Codes to mark the borders of the scanned book and crop/deskew based on that data, the DPI is also calculated and written to the file. Current functionality is limited to this narrow purpose, but I would like to add some fancy image processing in the future and perhaps further processing to bind the images together into a complete pdf (or maybe even DjVU). It only runs on OS X 10.7 and later, this is a requirement of ZXingObjC (it depends on NSRegularExpresion in the foundation framework from 10.7 and later).
Screen Shot 2012-12-14 at 5.12.05 PM.png
Screen Shot 2012-12-14 at 5.12.05 PM.png (407.4 KiB) Viewed 10156 times
--Current Workflow:
The source can be downloaded from https://github.com/shaknum/Paginator, the compiled app is stored there as well and it should be runnable as is (OS X 10.7 and later). The software is alpha and somewhat of a prototype right now, but I would be glad for some real world testing and advice.
When you run the program, you can click on “File” the “Print QR Codes...” to print out the necessary QR codes. It would probably be best to attach these to a ruler or something with the top QR Codes being slidable up and down the ruler (I'll do this soon). Then capture the images of your book pages like this:
SAM_8508 copy.jpg
Then you can return to Paginator and click on either “Load Page Images” on the bottom left or “File” then “open”, they both work the same. Then select the directory where your left page images are stored (I’ve only tested with jpg files, other types may work and I will soon start testing with raw), only portrait files work presently, landscape will fail. Now an alert will pop up asking if you want to load right page images, you can either say yes and select the folder with the right images, or if you keep both pages in one directory say no, they will all appear in the left hand column.
Now you can reorder any of the images in the browser columns (original order numbering is preserved in the title for your reference). You can delete images from either column and you can drag and drop new images onto the browser window as well to add pages you may have missed. When everything looks right, set your desired output DPI, choose color, grayscale, or black and white image output and hit “Process” in the bottom middle. Now there will be a pop up asking if you want to change the default output directory (this is based on the location of the left page image directory you selected earlier), you can say “NO” and accept the default or “YES” and select a new directory. Now sit back, relax, and watch the progress bar go. Failures will probably be silent (minimal reports will be output to the Console), hopefully everything will just work right and your processed files will appear in the output directory.
Example output:
Processed - Small.jpg

Re: New OS X Post-processing Software

Posted: 14 Dec 2012, 18:25
by Shaknum
--Current Issues:
Problem 1:
I have a number of math issues and would love some input. First, the way I calculate the crop point is a bit hacky. ZXing gives me the center of each position block on the QR Code:
QR Code - position code.jpg
(90.87 KiB) Downloaded 7375 times
I can get the distance between two on the x-axis and two on the z-axis and average them getting something close to the dpi for that part of the image (I average all four corners to calculate the DPI for the whole image). I then use a formula to get from the center of the position block to the corner of the QR Code; for instance: cropPoint.x = point.x - dpi*0.45 and cropPoint.y = point.y + dpi*0.45 (+ or - depends on which QR Code I’m reading):
QR Code - position code - adjusted.jpg
QR Code - position code - adjusted.jpg (91.04 KiB) Viewed 10155 times
There must be a more precise way for me to get from the center of the QR Code position block to the corner, but I don’t know it.

Problem 2:
Currently the program only works if the images are in portrait (not landscape). There must be some issue with my math, but I don’t get it:

Code: Select all

NSPoint bottomLeft;
    NSPoint bottomRight;
    NSPoint topLeft;
    NSPoint topRight;
                       
    if ([pagePosition isEqualToString:@"left page"]) {
        bottomLeft = [[dictData objectForKey:@"left page bottom left"] pointValue];
        bottomRight = [[dictData objectForKey:@"left page bottom right"] pointValue];
        topLeft = [[dictData objectForKey:@"left page top left"] pointValue];
        topRight = [[dictData objectForKey:@"left page top right"] pointValue];
    }
    
    if ([pagePosition isEqualToString:@"right page"]) {
        bottomLeft = [[dictData objectForKey:@"right page bottom left"] pointValue];
        bottomRight = [[dictData objectForKey:@"right page bottom right"] pointValue];
        topLeft = [[dictData objectForKey:@"right page top left"] pointValue];
        topRight = [[dictData objectForKey:@"right page top right"] pointValue];
    }
    
    //Collect the DPI data
    float dpi = [[dictData objectForKey:@"DPI"] floatValue];
    
    cv::Mat original = [self cvMatFromNSImage:initialImage];
    
    //Calculate proper width and height
    //Not sure why this doesn't work for landscape images
    CGFloat w1 = sqrt( pow(bottomRight.x - bottomLeft.x , 2) + pow(bottomRight.x - bottomLeft.x, 2));
    CGFloat w2 = sqrt( pow(topRight.x - topLeft.x , 2) + pow(topRight.x - topLeft.x, 2));
     
    CGFloat h1 = sqrt( pow(topRight.y - bottomRight.y , 2) + pow(topRight.y - bottomRight.y, 2));
    CGFloat h2 = sqrt( pow(topLeft.y - bottomLeft.y , 2) + pow(topLeft.y - bottomLeft.y, 2));
     
    CGFloat maxWidth = (w1 < w2) ? w1 : w2;
    CGFloat maxHeight = (h1 < h2) ? h1 : h2;
    
    //Adjust width and height for DPI (dpi is calculated from the QR Code, targetDPI is set by the user)
    maxWidth = (maxWidth/dpi)*targetDPI;
    maxHeight = (maxHeight/dpi)*targetDPI;
    
    //Setup matrix
    cv::Point2f src[4], dst[4];
    //src has the points that I found in relation to the QR Code
    src[0].x = topLeft.x;
    src[0].y = topLeft.y;
    src[1].x = topRight.x;
    src[1].y = topRight.y;
    src[2].x = bottomRight.x;
    src[2].y = bottomRight.y;
    src[3].x = bottomLeft.x;
    src[3].y = bottomLeft.y;
    
    //dst is the points I want the src points to move to (i.e. skew)
    dst[0].x = 0;
    dst[0].y = 0;
    dst[1].x = maxWidth;
    dst[1].y = 0;
    dst[2].x = maxWidth;
    dst[2].y = maxHeight;
    dst[3].x = 0;
    dst[3].y = maxHeight;
    
    //Perform undistortion
    cv::Mat undistorted = cv::Mat( cvSize(maxWidth,maxHeight), CV_8UC1);
    cv::warpPerspective(original, undistorted, cv::getPerspectiveTransform(src, dst), cvSize(maxWidth, maxHeight));

Not sure why this doesn’t just work for any orientation (90degrees, 180, 270, 66.45, etc...), but we should be able to process the file at any orientation if we know the bottom left, bottom right, top left, top right points (which we do since the QR Codes are labelled and that data is read here).

Problem 3
ZXing fails if the image isn’t the right size. My 14 megapixel images need to be resized to a third the size (~5 megapixels) to avoid detection errors, and this resizing is hardcoded into the program. I think a more automatic resizing method is easily possible, but I need time to work it out.

Re: New OS X Post-processing Software

Posted: 14 Dec 2012, 18:26
by Shaknum
--Image processing
I could use some help with image processing. I have used openCV since it has a nice perspective warp, though I think that can be done natively with Core Image. OpenCV can convert to grayscale or even B&W with a nice adaptive threshold, nevertheless it can’t do incredibly well with the fairly uneven images from our image scans. Scantailor has some great processing ability here, perhaps there are some C methods/libs that I could pull from that project, or perhaps there are even better things. The sky is the limit here, we are just basically limited by the libraries we can use in objective c (or import from c).

Re: New OS X Post-processing Software

Posted: 14 Dec 2012, 18:27
by Shaknum
--Help and openness
I have used open source libraries and this project is completely open. Feel free to use the source for anything at all, it is all rather hacked, though I will try to clean it up over time. I especially need to fix the hierarchy issue with ImageBrowser and ImageBrowser2 which should be controlled by one super class, not telephoning each other back and forth (a remnant of an earlier program I had written earlier). I appreciate any help anyone might want to offer. As I said the main concept is here and working, but I need to polish things and fix some things up.

Re: New OS X Post-processing Software

Posted: 28 Dec 2012, 14:18
by steve1066d
As far as calculating that corner, its difficult to get at that information with zxing, unless you not the version of code it is that you are reading (which last time I looked wasn't returned in the metadata).

The key is that the white space around the code is 3 segments log. So if you know the size of a segment you can calculate where the white space should end. If you know the com.google.zxing.qrcode.decoder.Version, you can get the number of segments by using the getDimensionForVersion() method.

Then to determine the number of segments between the two position blocks, subtract 7 from that value.

Then you can take the distance between position blocks that you have divide by the number of segments between blocks, and you should have the segment size, which you can use to find the corner including the whitespace.

You might want to look at the BSW code where I calculate where the 4th position block would be if it was in the same position as the other 3:

https://bookscanwizard.svn.sourceforge. ... RData.java

Re: New OS X Post-processing Software

Posted: 29 Dec 2012, 12:43
by Shaknum
steve1066d wrote:As far as calculating that corner, its difficult to get at that information with zxing, unless you not the version of code it is that you are reading (which last time I looked wasn't returned in the metadata).

The key is that the white space around the code is 3 segments log. So if you know the size of a segment you can calculate where the white space should end. If you know the com.google.zxing.qrcode.decoder.Version, you can get the number of segments by using the getDimensionForVersion() method.

Then to determine the number of segments between the two position blocks, subtract 7 from that value.

Then you can take the distance between position blocks that you have divide by the number of segments between blocks, and you should have the segment size, which you can use to find the corner including the whitespace.

You might want to look at the BSW code where I calculate where the 4th position block would be if it was in the same position as the other 3:

https://bookscanwizard.svn.sourceforge. ... RData.java
That makes a lot of sense, thanks for the help. I'll thumb through and see if I can get that method to return the right data.

Re: New OS X Post-processing Software

Posted: 17 May 2013, 12:10
by Misty
This looks great! I'm going to give it a try.

One thing I'd find useful is colour correction. Using the open-source Argyll-CMS tool, it would be possible to photograph a colour chart with each of the left and right cameras, then automatically apply it to all pages uniformly. See my thread on it here: http://diybookscanner.org/forum/viewtop ... =20&t=2676