Scantailor preprocess script
Moderator: peterZ
-
- Posts: 79
- Joined: 15 Sep 2010, 15:33
- Number of books owned: 2000
- Country: USA
- Location: Ohio
Re: Scantailor preprocess script
There's no problem with noob questions that are asked clearly like you did!
Have you compiled ppmunwarp yet? The cpp file is just the source. Compile it on Linux using: g++ -o ppmunwarp ppmunwarp.cpp
That will produce a file called ppmunwarp (no extension) Put it wherever you want. There are three ways of doing it that I see:
1. Put it in a folder that the system looks too when calling a binary, like /usr/bin. Nothing needs added to the script for that.
2. A folder of your choice, such as a subdirectory called software in your home directory, add ~/software/ to the script whenever ppmunwarp is called. (It's in there four times, I think. Lines 109, 118, 129, 152.)
3. Place the ppmunwarp binary in the working directory. Nothing needs changed there either.
I think it's just easier to put it in a binary folder. (I use ~/bin/)
Maybe it would be worth adding a configuration option for a path to ppmunwarp. (As in option 2) I never really thought about it! Thanks for pointing that out to me.
Have you compiled ppmunwarp yet? The cpp file is just the source. Compile it on Linux using: g++ -o ppmunwarp ppmunwarp.cpp
That will produce a file called ppmunwarp (no extension) Put it wherever you want. There are three ways of doing it that I see:
1. Put it in a folder that the system looks too when calling a binary, like /usr/bin. Nothing needs added to the script for that.
2. A folder of your choice, such as a subdirectory called software in your home directory, add ~/software/ to the script whenever ppmunwarp is called. (It's in there four times, I think. Lines 109, 118, 129, 152.)
3. Place the ppmunwarp binary in the working directory. Nothing needs changed there either.
I think it's just easier to put it in a binary folder. (I use ~/bin/)
Maybe it would be worth adding a configuration option for a path to ppmunwarp. (As in option 2) I never really thought about it! Thanks for pointing that out to me.
-
- Posts: 6
- Joined: 04 Jan 2013, 23:55
- E-book readers owned: kindle
- Number of books owned: 0
- Country: USA
Re: Scantailor preprocess script
note to self. install GNU C++ compiler... SMH. This is what happens when you have a Windows user dual booting for the first time.
Thank you for your help!
Thank you for your help!
-
- Posts: 6
- Joined: 04 Jan 2013, 23:55
- E-book readers owned: kindle
- Number of books owned: 0
- Country: USA
Re: Scantailor preprocess script
I have dpi set to automatic.
when I set it to manual, and use gimp to input dpi
what am I doing wrong?
Code: Select all
Simple calibration: ppmunwarp [-m check.ppm] calib.ppm > deform.bin
Simple unwarping: ppmunwarp -d deform.bin skewed.ppm > unwarped.ppm
Calculated PPI is:
ImageMagick will now prepare images for Scantailor...
mogrify.im6: invalid argument for option `-units': -density @ error/mogrify.c/MogrifyImageCommand/4325.
done
Code: Select all
calibration_corrected.ppm is being opened in GIMP
Manually determine the PixelsPerInch
PixelsPerInch?:
PPI is set to: 127
ImageMagick will now prepare images for Scantailor...
mogrify.im6: improper image header `01_L_corrected.ppm' @ error/pnm.c/ReadPNMImage/293.
-
- Posts: 79
- Joined: 15 Sep 2010, 15:33
- Number of books owned: 2000
- Country: USA
- Location: Ohio
Re: Scantailor preprocess script
Could I see the entire log file? I don't think I have enough information to answer properly. I'll speculate though, because that's what men do even when we don't have enough data. (I'm suddenly picturing the guys on King of the Hill all giving different reasons why a truck won't start) While you are at it, I guess I can check your config section too from the main script.
The first problem:
The first question I have is if it is unwarping things correctly up to the DPI stage.
Did you change anything in the script? I'm not seeing the output from ppmunwarp that I get with the commands I run.
My log gives:
In order to have it give the DPI, you need to have the correct -mul option set in the UNWARP_OPT section of the config. (If using the half centimeter sheet, it should be UNWARP_OPT="-mul 5.08")
The imagemagick error there is because there is no argument being given to density option. (because the variable isn't being read)
The second error:
That looks like the script is working properly, but imagemagick is choking somewhere. It looks to me like a memory leak of some sort. In your package manager, you may be able to try installing a different version of imagemagick to see if it solves it. (I don't know enough about imagemagick or c to really understand that error message. It comes from here somewhere: http://trac.imagemagick.org/browser/Ima ... ders/pnm.c )
I'm also curious if things work properly if you just leave the DPI config option blank. In that instance, it will just try to convert the unwarped images into tif files without inserting DPI. That would isolate whether it is an imagemagick problem or a script problem. Also, the DPI value you entered seems pretty low. (Scantailor won't even work with files below 150 dpi) I have no idea why that could throw imagemagick off, though.
The first problem:
The first question I have is if it is unwarping things correctly up to the DPI stage.
Did you change anything in the script? I'm not seeing the output from ppmunwarp that I get with the commands I run.
My log gives:
Code: Select all
Calculating DPI from image: calibration_corrected.ppm
Number of detected points: 1938
Average: 64.579329, calculated from 1900 of 1900 data points. PPI: 328
Deskewed picture size: 3355 x 2516 (99.91% x 99.93%)
Calculated PPI is: 328
The imagemagick error there is because there is no argument being given to density option. (because the variable isn't being read)
The second error:
That looks like the script is working properly, but imagemagick is choking somewhere. It looks to me like a memory leak of some sort. In your package manager, you may be able to try installing a different version of imagemagick to see if it solves it. (I don't know enough about imagemagick or c to really understand that error message. It comes from here somewhere: http://trac.imagemagick.org/browser/Ima ... ders/pnm.c )
I'm also curious if things work properly if you just leave the DPI config option blank. In that instance, it will just try to convert the unwarped images into tif files without inserting DPI. That would isolate whether it is an imagemagick problem or a script problem. Also, the DPI value you entered seems pretty low. (Scantailor won't even work with files below 150 dpi) I have no idea why that could throw imagemagick off, though.
-
- Posts: 6
- Joined: 04 Jan 2013, 23:55
- E-book readers owned: kindle
- Number of books owned: 0
- Country: USA
Re: Scantailor preprocess script
Currently running Ubuntu 13.04 in virtual machine. I am using the red dot (cm) calibration page.
log (ran with fewer images so I could post the entire log)
preprocess: I did change file extension to CR2
log (ran with fewer images so I could post the entire log)
Code: Select all
Your camera images are now being prepared for Scantailor
Determining RGB values of color.CR2...
Detected RGB values are 43, 26, 12
RGB values to be adjusted by 3.7674418604, 6.2307692307, 13.3333333333
Running Imagemagick to adjust colors and convert CR2 to ppm...
done
Calculating geometry calibration data from image: calibration.ppm
Usage: ppmunwarp [options] [--] [inpname or stdin]
Options:
--version Print program version.
-h Print program usage.
-q Suppress normal program messages.
-c Enforce calibration mode.
-m Set output PPM picture with checked calibration points.
-cc (inpname) Set input PPM picture with calibration points.
-cp Set file name for textual calibration point list.
-d (stdout) Set file name for binary deformation grid.
-i (inpname) Set input file name.
-o (stdout) Set ouput file name.
-gm (1) Set additional margin around calibration points.
-gs (2) Set scale factor for deformation grid.
-pc (ff0000) Point color for calibration point detection.
-ph (85) Range [%] around point color for detection.
-ps (50) Saturation [%] for point detection.
-pv (50) Intensity volume threshold [%] for point detection.
Simple calibration: ppmunwarp [-m check.ppm] calib.ppm > deform.bin
Simple unwarping: ppmunwarp -d deform.bin skewed.ppm > unwarped.ppm
done
Correcting geometry. This will take some time...
Usage: ppmunwarp [options] [--] [inpname or stdin]
Options:
--version Print program version.
-h Print program usage.
-q Suppress normal program messages.
-c Enforce calibration mode.
-m Set output PPM picture with checked calibration points.
-cc (inpname) Set input PPM picture with calibration points.
-cp Set file name for textual calibration point list.
-d (stdout) Set file name for binary deformation grid.
-i (inpname) Set input file name.
-o (stdout) Set ouput file name.
-gm (1) Set additional margin around calibration points.
-gs (2) Set scale factor for deformation grid.
-pc (ff0000) Point color for calibration point detection.
-ph (85) Range [%] around point color for detection.
-ps (50) Saturation [%] for point detection.
-pv (50) Intensity volume threshold [%] for point detection.
Simple calibration: ppmunwarp [-m check.ppm] calib.ppm > deform.bin
Simple unwarping: ppmunwarp -d deform.bin skewed.ppm > unwarped.ppm
Usage: ppmunwarp [options] [--] [inpname or stdin]
Options:
--version Print program version.
-h Print program usage.
-q Suppress normal program messages.
-c Enforce calibration mode.
-m Set output PPM picture with checked calibration points.
-cc (inpname) Set input PPM picture with calibration points.
-cp Set file name for textual calibration point list.
-d (stdout) Set file name for binary deformation grid.
-i (inpname) Set input file name.
-o (stdout) Set ouput file name.
-gm (1) Set additional margin around calibration points.
-gs (2) Set scale factor for deformation grid.
-pc (ff0000) Point color for calibration point detection.
-ph (85) Range [%] around point color for detection.
-ps (50) Saturation [%] for point detection.
-pv (50) Intensity volume threshold [%] for point detection.
Simple calibration: ppmunwarp [-m check.ppm] calib.ppm > deform.bin
Simple unwarping: ppmunwarp -d deform.bin skewed.ppm > unwarped.ppm
Usage: ppmunwarp [options] [--] [inpname or stdin]
Options:
--version Print program version.
-h Print program usage.
-q Suppress normal program messages.
-c Enforce calibration mode.
-m Set output PPM picture with checked calibration points.
-cc (inpname) Set input PPM picture with calibration points.
-cp Set file name for textual calibration point list.
-d (stdout) Set file name for binary deformation grid.
-i (inpname) Set input file name.
-o (stdout) Set ouput file name.
-gm (1) Set additional margin around calibration points.
-gs (2) Set scale factor for deformation grid.
-pc (ff0000) Point color for calibration point detection.
-ph (85) Range [%] around point color for detection.
-ps (50) Saturation [%] for point detection.
-pv (50) Intensity volume threshold [%] for point detection.
Simple calibration: ppmunwarp [-m check.ppm] calib.ppm > deform.bin
Simple unwarping: ppmunwarp -d deform.bin skewed.ppm > unwarped.ppm
Usage: ppmunwarp [options] [--] [inpname or stdin]
Options:
--version Print program version.
-h Print program usage.
-q Suppress normal program messages.
-c Enforce calibration mode.
-m Set output PPM picture with checked calibration points.
-cc (inpname) Set input PPM picture with calibration points.
-cp Set file name for textual calibration point list.
-d (stdout) Set file name for binary deformation grid.
-i (inpname) Set input file name.
-o (stdout) Set ouput file name.
-gm (1) Set additional margin around calibration points.
-gs (2) Set scale factor for deformation grid.
-pc (ff0000) Point color for calibration point detection.
-ph (85) Range [%] around point color for detection.
-ps (50) Saturation [%] for point detection.
-pv (50) Intensity volume threshold [%] for point detection.
Simple calibration: ppmunwarp [-m check.ppm] calib.ppm > deform.bin
Simple unwarping: ppmunwarp -d deform.bin skewed.ppm > unwarped.ppm
Usage: ppmunwarp [options] [--] [inpname or stdin]
Options:
--version Print program version.
-h Print program usage.
-q Suppress normal program messages.
-c Enforce calibration mode.
-m Set output PPM picture with checked calibration points.
-cc (inpname) Set input PPM picture with calibration points.
-cp Set file name for textual calibration point list.
-d (stdout) Set file name for binary deformation grid.
-i (inpname) Set input file name.
-o (stdout) Set ouput file name.
-gm (1) Set additional margin around calibration points.
-gs (2) Set scale factor for deformation grid.
-pc (ff0000) Point color for calibration point detection.
-ph (85) Range [%] around point color for detection.
-ps (50) Saturation [%] for point detection.
-pv (50) Intensity volume threshold [%] for point detection.
Simple calibration: ppmunwarp [-m check.ppm] calib.ppm > deform.bin
Simple unwarping: ppmunwarp -d deform.bin skewed.ppm > unwarped.ppm
Usage: ppmunwarp [options] [--] [inpname or stdin]
Options:
--version Print program version.
-h Print program usage.
-q Suppress normal program messages.
-c Enforce calibration mode.
-m Set output PPM picture with checked calibration points.
-cc (inpname) Set input PPM picture with calibration points.
-cp Set file name for textual calibration point list.
-d (stdout) Set file name for binary deformation grid.
-i (inpname) Set input file name.
-o (stdout) Set ouput file name.
-gm (1) Set additional margin around calibration points.
-gs (2) Set scale factor for deformation grid.
-pc (ff0000) Point color for calibration point detection.
-ph (85) Range [%] around point color for detection.
-ps (50) Saturation [%] for point detection.
-pv (50) Intensity volume threshold [%] for point detection.
Simple calibration: ppmunwarp [-m check.ppm] calib.ppm > deform.bin
Simple unwarping: ppmunwarp -d deform.bin skewed.ppm > unwarped.ppm
done
Calculating DPI from image: calibration_corrected.ppm
Usage: ppmunwarp [options] [--] [inpname or stdin]
Options:
--version Print program version.
-h Print program usage.
-q Suppress normal program messages.
-c Enforce calibration mode.
-m Set output PPM picture with checked calibration points.
-cc (inpname) Set input PPM picture with calibration points.
-cp Set file name for textual calibration point list.
-d (stdout) Set file name for binary deformation grid.
-i (inpname) Set input file name.
-o (stdout) Set ouput file name.
-gm (1) Set additional margin around calibration points.
-gs (2) Set scale factor for deformation grid.
-pc (ff0000) Point color for calibration point detection.
-ph (85) Range [%] around point color for detection.
-ps (50) Saturation [%] for point detection.
-pv (50) Intensity volume threshold [%] for point detection.
Simple calibration: ppmunwarp [-m check.ppm] calib.ppm > deform.bin
Simple unwarping: ppmunwarp -d deform.bin skewed.ppm > unwarped.ppm
Calculated PPI is:
ImageMagick will now prepare images for Scantailor...
mogrify.im6: invalid argument for option `-units': -density @ error/mogrify.c/MogrifyImageCommand/4325.
done
Deleting temporary files...
done
Your images are ready for Scantailor
Code: Select all
#!/bin/bash
(
###Scantailor Preprocessor###
#Fixes color with a gray card and geometry with ppmunwarp and a calibration grid
#Dependencies: Imagemagick, ppmunwarp as modified by royeven, Ufraw (only needed for RAW), Ufraw-batch (only needed for RAW), GIMP (if manual DPI calculation is desired)
#Written by abmartin and royeven at www.diybookscanner.org
###Configuration###
# Global "variables"
INPUT_FORMAT="CR2" #Change as appropriate
OUTPUT_FORMAT="ppm" #Do not change
COLOR_IMAGE="color" # The file name (without extension) of the gray card photo
CALIBRATION_IMAGE="calibration" # The file name (without extension) of the geometry calibration file
CONTROL_IMAGE="check" # The name of the deskew control file created with the -m attribute of ppmunwarp
CONTROL_PPI="check_ppi" # The name of the ppi control file created with the -m attribute of ppmunwarp
CALIBRATION_FILE="calibration.bin" # The full file name of the binary calibration data
UNWARP_OPT="-mul 5.08" # Options to ppmunwarp. Change according to your setup. Note: the mul attribute is a multiplier to the pixel-count. Since there are 5.08 calibration points pr. inch (i.e. 2*2.54), we must multiply the pixel count between two adjacent calibration dots with 5.08 to get the PPI
CORRECTED_POSTFIX="_corrected" # the postfix of corrected images. E.g. the file "PAGE01_R.PPM" becomes "PAGE_01_POSTFIX.PPM"
COLOR_CORRECTION_METHOD="imagemagick" #options for imagemagick and ufraw - leave blank if no color correction is desired
DPI_CALCULATION_METHOD="automatic" #options are automatic, manual, and autocheck
IM_VERBOSE_OPTION="" #use "-verbose" if imagemagick details are desired, otherwise leave blank
IM_FINAL_OUTPUT_OPTION="-format tif -compress lzw" #set desired output format and compression options -- for tif with lzw compression, use "-format tif -compress lzw"
#Imagemagick Color Correction Options
#True RGB values of Gray Card
TRUE_RED=162
TRUE_GREEN=162
TRUE_BLUE=160
COLOR_CROP_AREA="500x500" #crop area for the gray card in the center in order to find the average color of the gray card. If the crop area extends beyond the gray card, color correction will be incorrect.
# Include current folder in search path for program ppmunwarp
PATH=$PATH:`pwd`
echo -e "\nYour camera images are now being prepared for Scantailor"
###Color Correction###
##Using Imagemagick to Fix Colors
if [ "$COLOR_CORRECTION_METHOD" == "imagemagick" ];
then
echo -e "\nDetermining RGB values of $COLOR_IMAGE.$INPUT_FORMAT..."
#Convert RAW file extension and crop an area in the center of calibration image
convert $COLOR_IMAGE.$INPUT_FORMAT -gravity center -crop "$COLOR_CROP_AREA"+0+0 color-test.png
#Determine average colors in cropped area
SOURCE_RED=$(convert color-test.png -resize 1x1 -format "%[fx:int(255*p{10,10}.r)]" info:)
SOURCE_GREEN=$(convert color-test.png -resize 1x1 -format "%[fx:int(255*p{10,10}.g)]" info:)
SOURCE_BLUE=$(convert color-test.png -resize 1x1 -format "%[fx:int(255*p{10,10}.b)]" info:)
echo "Detected RGB values are $SOURCE_RED, $SOURCE_GREEN, $SOURCE_BLUE"
#Calculate necessary adjustments
RED_ADJUST="$(echo "scale=10; $TRUE_RED/$SOURCE_RED" | bc)"
GREEN_ADJUST="$(echo "scale=10; $TRUE_GREEN/$SOURCE_GREEN" | bc)"
BLUE_ADJUST="$(echo "scale=10; $TRUE_BLUE/$SOURCE_BLUE" | bc)"
echo "RGB values to be adjusted by $RED_ADJUST, $GREEN_ADJUST, $BLUE_ADJUST"
#Adjust Colors and Output Raw images as ppm files
echo -e "\nRunning Imagemagick to adjust colors and convert $INPUT_FORMAT to $OUTPUT_FORMAT..."
mogrify -format ppm $IM_VERBOSE_OPTION -color-matrix "$RED_ADJUST 0 0 0 $GREEN_ADJUST 0 0 0 $BLUE_ADJUST" *.$INPUT_FORMAT
#Remove temporary file
rm color-test.png
echo "done"
##Using UFRaw to Fix Colors
elif [ "$COLOR_CORRECTION_METHOD" == "ufraw" ];
#1. select an area on your gray card,
#2. click the eyedropper, which equalizes RGB values
#3. adjust the exposure control to get RGB values to their ultimate goal.
#4. Input values in the script
then
#Interactive variables
echo -e "\nThe color calibration image is being loaded in UFRaw. Enter the following values for color correction"
ufraw $COLORIMAGE.$INPUT_FORMAT &
echo "Color temperature?: "
read UFRAW_TEMP
echo "Green Value?: "
read UFRAW_GREEN
echo "Exposure change?: "
read UFRAW_EXPOSURE
##Convert all files of INPUT_FORMAT to OUTPUT_FORMAT
echo -e "\nRunning ufraw-batch, this will take a while..."
ufraw-batch --out-type=$OUTPUT_FORMAT *.$INPUT_FORMAT #--temperature=$UFRAW_TEMP --green=$UFRAW_GREEN --exposure=$UFRAW_EXPOSURE
echo "done"
##No Color Correction
else
echo -e "\nNo color correction will be done"
echo "Preparing images for ppmunwarp by converting $INPUT_FORMAT to $OUTPUT_FORMAT"
mogrify -format ppm $IM_VERBOSE_OPTION *.$INPUT_FORMAT
echo "done"
fi
###Geometry Correction###
##Calibration
echo -e "\nCalculating geometry calibration data from image: $CALIBRATION_IMAGE.$OUTPUT_FORMAT"
ppmunwarp $UNWARP_OPT -m "$CONTROL_IMAGE.$OUTPUT_FORMAT" "$CALIBRATION_IMAGE.$OUTPUT_FORMAT" > "$CALIBRATION_FILE"
echo -e "done"
##Correction
echo -e "\nCorrecting geometry. This will take some time..."
for i in *.ppm; do
if [ -e "$i" ]; then
FILE=`basename "$i" .ppm`
ppmunwarp $UNWARP_OPT -d calibration.bin "$i" > "$FILE$CORRECTED_POSTFIX.$OUTPUT_FORMAT"
fi
done
echo -e "done"
###DPI Calculation###
if [ "$DPI_CALCULATION_METHOD" == "automatic" ];
then
##Calculating PPI in software
echo -e "\nCalculating DPI from image: $CALIBRATION_IMAGE$CORRECTED_POSTFIX.$OUTPUT_FORMAT"
PPI=$(( (ppmunwarp $UNWARP_OPT -m "$CONTROL_PPI.$OUTPUT_FORMAT" "$CALIBRATION_IMAGE$CORRECTED_POSTFIX.$OUTPUT_FORMAT") 1>/dev/null ) 2>&1)
echo "$PPI"
PPI=`echo "$PPI" | grep -o "Average: [0-9\.]*, calculated from [0-9]* of [0-9]* data points. PPI: [0-9]*" | sed -r 's/.*PPI: ([0-9]*).*/\1/g'`
echo "Calculated PPI is: $PPI"
elif [ "$DPI_CALCULATION_METHOD" == "manual" ];
then
##Open Image in GIMP
echo -e "\n$CALIBRATION_IMAGE$CORRECTED_POSTFIX.$OUTPUT_FORMAT is being opened in GIMP"
echo "Manually determine the PixelsPerInch"
gimp "$CALIBRATION_IMAGE$CORRECTED_POSTFIX.$OUTPUT_FORMAT" &
##Enter PPI
echo -e "\nPixelsPerInch?: "
read PPI
echo "PPI is set to: $PPI"
elif [ "$DPI_CALCULATION_METHOD" == "autocheck" ];
##Autodetecting DPI and comparing with a manual measurment in GIMP
then
##Calculating PPI in software
echo -e "\nCalculating calibration data from image: $CALIBRATION_IMAGE$CORRECTED_POSTFIX.$OUTPUT_FORMAT"
PPI=$(( (ppmunwarp $UNWARP_OPT -m "$CONTROL_PPI.$OUTPUT_FORMAT" "$CALIBRATION_IMAGE$CORRECTED_POSTFIX.$OUTPUT_FORMAT") 1>/dev/null ) 2>&1)
echo "$PPI"
##Checking with GIMP
gimp "$CALIBRATION_IMAGE$CORRECTED_POSTFIX.$OUTPUT_FORMAT" &
##Compare automatic and manual
PPI=`echo "$PPI" | grep -o "Average: [0-9\.]*, calculated from [0-9]* of [0-9]* data points. PPI: [0-9]*" | sed -r 's/.*PPI: ([0-9]*).*/\1/g'`
echo "Calculated PPI is: $PPI"
echo "Is this correct? If not, insert corrected value now. If correct, leave empty and press enter"
read PPI_CORRECTED
if [ "$PPI_CORRECTED" != "" ]
then
PPI=$PPI_CORRECTED
fi
echo "PPI is set to: $PPI"
else
#If config option is left blank or is incorrect, no DPI will be entered
echo -e "\nDPI information has neither been calculated nor manually entered."
echo "ImageMagick will now prepare images for Scantailor..."
mogrify $IM_VERBOSE_OPTION $IM_FINAL_OUTPUT_OPTION *[$CORRECTED_POSTFIX].ppm
rm *.ppm
echo "done"
exit
fi
###Preparing Images for Scantailor###
echo -e "\nImageMagick will now prepare images for Scantailor..."
mogrify $IM_VERBOSE_OPTION $IM_FINAL_OUTPUT_OPTION -density $PPI -units PixelsPerInch *[$CORRECTED_POSTFIX].ppm
echo "done"
###Housekeeping###
#Delete temporary files
echo -e "\nDeleting temporary files..."
rm *.ppm
echo "done"
echo -e "\nYour images are ready for Scantailor"
exit
) 2>&1 | tee preprocess.log
-
- Posts: 79
- Joined: 15 Sep 2010, 15:33
- Number of books owned: 2000
- Country: USA
- Location: Ohio
Re: Scantailor preprocess script
Hmmm...
So it appears that the conversion to ppm is going okay. If my next suggestion is wrong, could you please confirm that the initial conversion is happening properly. (imagemagick should spit out an error message if it wasn't) Those files would appear in the working folder.
The first failure occurs when ppmunwarp tries to calibrate itself. The message it is giving us tells us that something is wrong with the syntax. I wonder if it is not able to find the calibration image. What have you named the calibration file? The script is trying to feed ppmunwarp with calibration.ppm which was converted from the original calibration.CR2. If you chose to call that file something different, you can change that name in the script config.
I sure hope it's that simple, or else I have some thinking to do.
I really should go back in and add exits on errors. (but I doubt that I will)
So it appears that the conversion to ppm is going okay. If my next suggestion is wrong, could you please confirm that the initial conversion is happening properly. (imagemagick should spit out an error message if it wasn't) Those files would appear in the working folder.
The first failure occurs when ppmunwarp tries to calibrate itself. The message it is giving us tells us that something is wrong with the syntax. I wonder if it is not able to find the calibration image. What have you named the calibration file? The script is trying to feed ppmunwarp with calibration.ppm which was converted from the original calibration.CR2. If you chose to call that file something different, you can change that name in the script config.
I sure hope it's that simple, or else I have some thinking to do.
I really should go back in and add exits on errors. (but I doubt that I will)
-
- Posts: 6
- Joined: 04 Jan 2013, 23:55
- E-book readers owned: kindle
- Number of books owned: 0
- Country: USA
Re: Scantailor preprocess script
It appears to be able to find the calibration image without issue.
http://img14.imageshack.us/img14/5696/ea3i.jpg
I added verbose to the script... here is the log
http://img14.imageshack.us/img14/5696/ea3i.jpg
I added verbose to the script... here is the log
Code: Select all
Your camera images are now being prepared for Scantailor
Determining RGB values of color.CR2...
Detected RGB values are 43, 26, 12
RGB values to be adjusted by 3.7674418604, 6.2307692307, 13.3333333333
Running Imagemagick to adjust colors and convert CR2 to ppm...
"ufraw-batch" --silent --create-id=also --out-type=png --out-depth=16 "--output=/tmp/magick-gexOomHz.png" "/tmp/magick-SNC9KnXw"
/tmp/magick-gexOomHz.png PNG 3921x2611 3921x2611+0+0 16-bit DirectClass 47.19MB 1.140u 0:01.179
calibration.CR2=>/tmp/magick-gexOomHz.png CR2 3921x2611 3921x2611+0+0 16-bit DirectClass 47.19MB 0.010u 0:00.030
calibration.CR2=>calibration.ppm CR2 3921x2611 3921x2611+0+0 16-bit DirectClass 61.42MB 0.480u 0:00.469
"ufraw-batch" --silent --create-id=also --out-type=png --out-depth=16 "--output=/tmp/magick-2d9YDotg.png" "/tmp/magick-uyvHArZZ"
/tmp/magick-2d9YDotg.png PNG 4770x3177 4770x3177+0+0 16-bit DirectClass 71.08MB 1.160u 0:01.160
color.CR2=>/tmp/magick-2d9YDotg.png CR2 4770x3177 4770x3177+0+0 16-bit DirectClass 71.08MB 0.010u 0:00.009
color.CR2=>color.ppm CR2 4770x3177 4770x3177+0+0 16-bit DirectClass 90.92MB 0.720u 0:00.510
"ufraw-batch" --silent --create-id=also --out-type=png --out-depth=16 "--output=/tmp/magick-q45yXviB.png" "/tmp/magick-TxGIdCBc"
/tmp/magick-q45yXviB.png PNG 4770x3177 4770x3177+0+0 16-bit DirectClass 72.53MB 1.340u 0:01.339
IMG_0003.CR2=>/tmp/magick-q45yXviB.png CR2 4770x3177 4770x3177+0+0 16-bit DirectClass 72.53MB 0.010u 0:00.019
IMG_0003.CR2=>IMG_0003.ppm CR2 4770x3177 4770x3177+0+0 16-bit DirectClass 90.92MB 0.730u 0:01.140
"ufraw-batch" --silent --create-id=also --out-type=png --out-depth=16 "--output=/tmp/magick-Vum1VR6o.png" "/tmp/magick-KuFJB9BB"
/tmp/magick-Vum1VR6o.png PNG 4770x3177 4770x3177+0+0 16-bit DirectClass 72.72MB 1.410u 0:01.419
IMG_0005.CR2=>/tmp/magick-Vum1VR6o.png CR2 4770x3177 4770x3177+0+0 16-bit DirectClass 72.72MB 0.020u 0:00.010
IMG_0005.CR2=>IMG_0005.ppm CR2 4770x3177 4770x3177+0+0 16-bit DirectClass 90.92MB 0.720u 0:00.480
"ufraw-batch" --silent --create-id=also --out-type=png --out-depth=16 "--output=/tmp/magick-Biz0aDku.png" "/tmp/magick-zasyt82m"
/tmp/magick-Biz0aDku.png PNG 4770x3177 4770x3177+0+0 16-bit DirectClass 72.68MB 1.390u 0:01.390
IMG_0007.CR2=>/tmp/magick-Biz0aDku.png CR2 4770x3177 4770x3177+0+0 16-bit DirectClass 72.68MB 0.010u 0:00.010
IMG_0007.CR2=>IMG_0007.ppm CR2 4770x3177 4770x3177+0+0 16-bit DirectClass 90.92MB 0.700u 0:00.509
"ufraw-batch" --silent --create-id=also --out-type=png --out-depth=16 "--output=/tmp/magick-FaXfyvP4.png" "/tmp/magick-tEntiUBM"
/tmp/magick-FaXfyvP4.png PNG 4770x3177 4770x3177+0+0 16-bit DirectClass 72.89MB 1.390u 0:01.399
IMG_0009.CR2=>/tmp/magick-FaXfyvP4.png CR2 4770x3177 4770x3177+0+0 16-bit DirectClass 72.89MB 0.010u 0:00.019
IMG_0009.CR2=>IMG_0009.ppm CR2 4770x3177 4770x3177+0+0 16-bit DirectClass 90.92MB 0.730u 0:00.980
Found a generic profile, type exif
Found a generic profile, type exif
Found a generic profile, type exif
Found a generic profile, type exif
Found a generic profile, type exif
Found a generic profile, type exif
done
Calculating geometry calibration data from image: calibration.ppm
Usage: ppmunwarp [options] [--] [inpname or stdin]
Options:
--version Print program version.
-h Print program usage.
-q Suppress normal program messages.
-c Enforce calibration mode.
-m Set output PPM picture with checked calibration points.
-cc (inpname) Set input PPM picture with calibration points.
-cp Set file name for textual calibration point list.
-d (stdout) Set file name for binary deformation grid.
-i (inpname) Set input file name.
-o (stdout) Set ouput file name.
-gm (1) Set additional margin around calibration points.
-gs (2) Set scale factor for deformation grid.
-pc (ff0000) Point color for calibration point detection.
-ph (85) Range [%] around point color for detection.
-ps (50) Saturation [%] for point detection.
-pv (50) Intensity volume threshold [%] for point detection.
Simple calibration: ppmunwarp [-m check.ppm] calib.ppm > deform.bin
Simple unwarping: ppmunwarp -d deform.bin skewed.ppm > unwarped.ppm
done
Correcting geometry. This will take some time...
Usage: ppmunwarp [options] [--] [inpname or stdin]
Options:
--version Print program version.
-h Print program usage.
-q Suppress normal program messages.
-c Enforce calibration mode.
-m Set output PPM picture with checked calibration points.
-cc (inpname) Set input PPM picture with calibration points.
-cp Set file name for textual calibration point list.
-d (stdout) Set file name for binary deformation grid.
-i (inpname) Set input file name.
-o (stdout) Set ouput file name.
-gm (1) Set additional margin around calibration points.
-gs (2) Set scale factor for deformation grid.
-pc (ff0000) Point color for calibration point detection.
-ph (85) Range [%] around point color for detection.
-ps (50) Saturation [%] for point detection.
-pv (50) Intensity volume threshold [%] for point detection.
Simple calibration: ppmunwarp [-m check.ppm] calib.ppm > deform.bin
Simple unwarping: ppmunwarp -d deform.bin skewed.ppm > unwarped.ppm
Usage: ppmunwarp [options] [--] [inpname or stdin]
Options:
--version Print program version.
-h Print program usage.
-q Suppress normal program messages.
-c Enforce calibration mode.
-m Set output PPM picture with checked calibration points.
-cc (inpname) Set input PPM picture with calibration points.
-cp Set file name for textual calibration point list.
-d (stdout) Set file name for binary deformation grid.
-i (inpname) Set input file name.
-o (stdout) Set ouput file name.
-gm (1) Set additional margin around calibration points.
-gs (2) Set scale factor for deformation grid.
-pc (ff0000) Point color for calibration point detection.
-ph (85) Range [%] around point color for detection.
-ps (50) Saturation [%] for point detection.
-pv (50) Intensity volume threshold [%] for point detection.
Simple calibration: ppmunwarp [-m check.ppm] calib.ppm > deform.bin
Simple unwarping: ppmunwarp -d deform.bin skewed.ppm > unwarped.ppm
Usage: ppmunwarp [options] [--] [inpname or stdin]
Options:
--version Print program version.
-h Print program usage.
-q Suppress normal program messages.
-c Enforce calibration mode.
-m Set output PPM picture with checked calibration points.
-cc (inpname) Set input PPM picture with calibration points.
-cp Set file name for textual calibration point list.
-d (stdout) Set file name for binary deformation grid.
-i (inpname) Set input file name.
-o (stdout) Set ouput file name.
-gm (1) Set additional margin around calibration points.
-gs (2) Set scale factor for deformation grid.
-pc (ff0000) Point color for calibration point detection.
-ph (85) Range [%] around point color for detection.
-ps (50) Saturation [%] for point detection.
-pv (50) Intensity volume threshold [%] for point detection.
Simple calibration: ppmunwarp [-m check.ppm] calib.ppm > deform.bin
Simple unwarping: ppmunwarp -d deform.bin skewed.ppm > unwarped.ppm
Usage: ppmunwarp [options] [--] [inpname or stdin]
Options:
--version Print program version.
-h Print program usage.
-q Suppress normal program messages.
-c Enforce calibration mode.
-m Set output PPM picture with checked calibration points.
-cc (inpname) Set input PPM picture with calibration points.
-cp Set file name for textual calibration point list.
-d (stdout) Set file name for binary deformation grid.
-i (inpname) Set input file name.
-o (stdout) Set ouput file name.
-gm (1) Set additional margin around calibration points.
-gs (2) Set scale factor for deformation grid.
-pc (ff0000) Point color for calibration point detection.
-ph (85) Range [%] around point color for detection.
-ps (50) Saturation [%] for point detection.
-pv (50) Intensity volume threshold [%] for point detection.
Simple calibration: ppmunwarp [-m check.ppm] calib.ppm > deform.bin
Simple unwarping: ppmunwarp -d deform.bin skewed.ppm > unwarped.ppm
Usage: ppmunwarp [options] [--] [inpname or stdin]
Options:
--version Print program version.
-h Print program usage.
-q Suppress normal program messages.
-c Enforce calibration mode.
-m Set output PPM picture with checked calibration points.
-cc (inpname) Set input PPM picture with calibration points.
-cp Set file name for textual calibration point list.
-d (stdout) Set file name for binary deformation grid.
-i (inpname) Set input file name.
-o (stdout) Set ouput file name.
-gm (1) Set additional margin around calibration points.
-gs (2) Set scale factor for deformation grid.
-pc (ff0000) Point color for calibration point detection.
-ph (85) Range [%] around point color for detection.
-ps (50) Saturation [%] for point detection.
-pv (50) Intensity volume threshold [%] for point detection.
Simple calibration: ppmunwarp [-m check.ppm] calib.ppm > deform.bin
Simple unwarping: ppmunwarp -d deform.bin skewed.ppm > unwarped.ppm
Usage: ppmunwarp [options] [--] [inpname or stdin]
Options:
--version Print program version.
-h Print program usage.
-q Suppress normal program messages.
-c Enforce calibration mode.
-m Set output PPM picture with checked calibration points.
-cc (inpname) Set input PPM picture with calibration points.
-cp Set file name for textual calibration point list.
-d (stdout) Set file name for binary deformation grid.
-i (inpname) Set input file name.
-o (stdout) Set ouput file name.
-gm (1) Set additional margin around calibration points.
-gs (2) Set scale factor for deformation grid.
-pc (ff0000) Point color for calibration point detection.
-ph (85) Range [%] around point color for detection.
-ps (50) Saturation [%] for point detection.
-pv (50) Intensity volume threshold [%] for point detection.
Simple calibration: ppmunwarp [-m check.ppm] calib.ppm > deform.bin
Simple unwarping: ppmunwarp -d deform.bin skewed.ppm > unwarped.ppm
done
Calculating DPI from image: calibration_corrected.ppm
Usage: ppmunwarp [options] [--] [inpname or stdin]
Options:
--version Print program version.
-h Print program usage.
-q Suppress normal program messages.
-c Enforce calibration mode.
-m Set output PPM picture with checked calibration points.
-cc (inpname) Set input PPM picture with calibration points.
-cp Set file name for textual calibration point list.
-d (stdout) Set file name for binary deformation grid.
-i (inpname) Set input file name.
-o (stdout) Set ouput file name.
-gm (1) Set additional margin around calibration points.
-gs (2) Set scale factor for deformation grid.
-pc (ff0000) Point color for calibration point detection.
-ph (85) Range [%] around point color for detection.
-ps (50) Saturation [%] for point detection.
-pv (50) Intensity volume threshold [%] for point detection.
Simple calibration: ppmunwarp [-m check.ppm] calib.ppm > deform.bin
Simple unwarping: ppmunwarp -d deform.bin skewed.ppm > unwarped.ppm
Calculated PPI is:
ImageMagick will now prepare images for Scantailor...
mogrify.im6: invalid argument for option `-units': -density @ error/mogrify.c/MogrifyImageCommand/4325.
done
Deleting temporary files...
done
Your images are ready for Scantailor
-
- Posts: 79
- Joined: 15 Sep 2010, 15:33
- Number of books owned: 2000
- Country: USA
- Location: Ohio
Re: Scantailor preprocess script
My instinct is suggesting that the ppm files that imagemagick is making are somehow not compatible with ppmunwarp. Since your file manager doesn't seem to be thumbnailing them, I figure that might be the problem. (Of course, I'm not sure if your file manager thumbs ppm files anyway) Since we also seem to be getting 0 byte _corrected.ppm files, I think that backs this up even more. If you can send me the calibration.CR2 image, I'll play around with imagemagick to see if there is something special that needs done. I've sent you a PM with my e-mail.
*edit
I just RTFM....http://www.imagemagick.org/www/formats.html
It does look like imagemagick needs an additional piece of information when handling CR2 files.
"Requires an explicit image format otherwise the image is interpreted as a TIFF image (e.g. cr2:image.cr2)."
(It may be as simple as just manually editing line 64 -- at the end just make it cr2:*.$INPUT_FORMAT --- That may also need done to line 48: cr2:$COLOR_IMAGE.$INPUT_FORMAT)
*edit
I just RTFM....http://www.imagemagick.org/www/formats.html
It does look like imagemagick needs an additional piece of information when handling CR2 files.
"Requires an explicit image format otherwise the image is interpreted as a TIFF image (e.g. cr2:image.cr2)."
(It may be as simple as just manually editing line 64 -- at the end just make it cr2:*.$INPUT_FORMAT --- That may also need done to line 48: cr2:$COLOR_IMAGE.$INPUT_FORMAT)
-
- Posts: 79
- Joined: 15 Sep 2010, 15:33
- Number of books owned: 2000
- Country: USA
- Location: Ohio
Re: Scantailor preprocess script
To report about the bug: The problem was happening in the initial conversion from the raw format. Then every other step failed too. (I really should add exit codes and errors, I just don't want too...) The solution, however, was pretty simple.
I just spent a few minutes playing with imagemagick and CR2 images. I've concluded that using imagemagick to do the initial raw conversion just isn't a great idea. The output that I get as ppm files have issues with ppmunwarp. (incorrect header)
The solution, and this seems to increase speed anyway, is to use ufraw-batch to do the initial raw conversion.
In this version, I have added two new color correction/conversion options. One for raw images without color correction (raw_no_color) and the other for raw images with automatic color correction (raw_color). The manual raw color correction option using ufraw and ufraw-batch is still available. (ufraw) The two remaining options are for non-raw images, like camera jpgs, one for automatic color correction (imagemagick) and the other for no color correction (leave config option blank)
In the new raw_color option, ufraw-batch makes tif files from the raw input. Imagemagick then determines appropriate color correction and applies that correction while converting into ppm files. (Then the intermediate tif files are deleted)
In raw_no_color, ufraw-batch just does the final conversion to ppm.
In all of the UFraw steps, I have added a --silent option. If you desire verbose output, delete it in the configuration.
I hope that this fixes everyone's RAW issues!
It still needs testing though!
(And one of these days, I'll get around to more sophisticated color profiling. See Misty's excellent thread about how that can be accomplished)
I just spent a few minutes playing with imagemagick and CR2 images. I've concluded that using imagemagick to do the initial raw conversion just isn't a great idea. The output that I get as ppm files have issues with ppmunwarp. (incorrect header)
The solution, and this seems to increase speed anyway, is to use ufraw-batch to do the initial raw conversion.
In this version, I have added two new color correction/conversion options. One for raw images without color correction (raw_no_color) and the other for raw images with automatic color correction (raw_color). The manual raw color correction option using ufraw and ufraw-batch is still available. (ufraw) The two remaining options are for non-raw images, like camera jpgs, one for automatic color correction (imagemagick) and the other for no color correction (leave config option blank)
In the new raw_color option, ufraw-batch makes tif files from the raw input. Imagemagick then determines appropriate color correction and applies that correction while converting into ppm files. (Then the intermediate tif files are deleted)
In raw_no_color, ufraw-batch just does the final conversion to ppm.
In all of the UFraw steps, I have added a --silent option. If you desire verbose output, delete it in the configuration.
I hope that this fixes everyone's RAW issues!
It still needs testing though!
(And one of these days, I'll get around to more sophisticated color profiling. See Misty's excellent thread about how that can be accomplished)
-
- Posts: 79
- Joined: 15 Sep 2010, 15:33
- Number of books owned: 2000
- Country: USA
- Location: Ohio
Re: Scantailor preprocess script
It seems like UFRaw does something strange with tif files...I get error messages from imagemagick when it opens each tif. It still gets the job done though, so it doesn't really matter. If those error messages bother you when using the raw_color method, it's pretty simple to hide those messages. Just add a -quiet option to lines 106 and 122
It's definitely the fastest lossless format to use as the intermediate step, so I'm content with meaningless error messages. Pngs, for example, take at least 15 times longer, which in a long book would be pretty annoying.
I've tested the new version on a few different sets of raw files of my own, and it seems to work properly.
It's definitely the fastest lossless format to use as the intermediate step, so I'm content with meaningless error messages. Pngs, for example, take at least 15 times longer, which in a long book would be pretty annoying.
I've tested the new version on a few different sets of raw files of my own, and it seems to work properly.