Daniel Reetz, the founder of the DIY Book Scanner community, has recently started making videos of prototyping and shop tips. If you are tinkering with a book scanner (or any other project) in your home shop, these tips will come in handy. https://www.youtube.com/channel/UCn0gq8 ... g_8K1nfInQ

Bitonal conversion using Photoshop (Gimp & Imagemagic...)

Share your software workflow. Write up your tips and tricks on how to scan, digitize, OCR, and bind ebooks.
Benedictus
Posts: 15
Joined: 14 Jan 2013, 00:38
E-book readers owned: None
Number of books owned: 2000
Country: Spain

Re: Bitonal conversion using Photoshop (Gimp & Imagemagic...

Post by Benedictus » 14 Jan 2013, 21:45

Hi.

I've been using the script and I find it amazing. Very tweakable.

However, I've found something weird with both the regular and the batch version. It's about file saving.

When I save the files, they look indeed bitonal, but they are actually grayscale images. The look is the same, but the size isn't. The files the script produces are bigger than the same files saved as 1 bit images by the GIMP itself (through Image > Mode > Indexed > Use black and white (1-bit) palette).

I don't know what the "Handle images" option does. B&W is mentioned there, but the option is not present in the batch version anyway.

Am I doing something wrong?

Bets regards.
Plustek OpticBook A300

xorpt
Posts: 42
Joined: 24 Feb 2012, 01:37
E-book readers owned: Sony PRS-T1
Number of books owned: 2000

Re: Bitonal conversion using Photoshop (Gimp & Imagemagic...

Post by xorpt » 15 Jan 2013, 03:39

HI Benedictus,

thanks for your feedback!

You are right, the output mode of my script is grayscale, because after converting RGB to Grayscale I leave it as is. Is the filesize difference very significant? Maybe I can try and reconvert the file to Bitonal before saving. I just did not see the need for it until you asked :) Interesting...

Regarding the "Handle images" option, you can use it on pages that contain both text and images to reduce the size of the images by using dithering (in a similar way that is used in newspapers: the images are transformed in a cloud of dots), the following way:

1. select the images in the page (with the select box tool - press Shift if you select several images)
2. select the color reduction method (for example "Dither to B&W" will transform a RGB or Grayscale image into a cloud of black and white dots)
3. run the script

Since it involves manual selection of the images, I did not put this option in the batch mode. Use the batch mode first, then process the pages containing images one-by-one. I'm planning on adding a Save option to the non-batch mode to make this step easier

"Dither to B&W" does the same as Image > Mode > Indexed > Use black and white (1-bit) palette) with "Color-Dithering Floyd-Steinberg (Normal)". But it does it only on the image (dithered text looks awful!)

xorpt
Posts: 42
Joined: 24 Feb 2012, 01:37
E-book readers owned: Sony PRS-T1
Number of books owned: 2000

Re: Bitonal conversion using Photoshop (Gimp & Imagemagic...

Post by xorpt » 15 Jan 2013, 07:43

I found a way to address these two points (change mode to bitonal when needed and save file in non-batch mode)

I'll upload a new version soon :D

xorpt
Posts: 42
Joined: 24 Feb 2012, 01:37
E-book readers owned: Sony PRS-T1
Number of books owned: 2000

Re: Bitonal conversion using Photoshop (Gimp & Imagemagic...

Post by xorpt » 15 Jan 2013, 20:54

Here it is:

Bitonal Converter for GIMP v.0.4

; change log:
; - Save image to real bitonal (not Grayscale) in Batch mode and when relevant in non-batch mode
; - Allow saving images after processing in non-batch mode

Enjoy. As usual, any feedback is welcome!

How to install the script:

Unzip the .scm file to "<GIMP install directory>\share\gimp\2.0\scripts". You should see a new submenu "DIY Book Scanning" appear under the "Filters" menu. If Gimp is already opened, use "Filters > Script-Fu > Refresh Scripts".
Attachments
bitonal-converter-v.0.4.zip
Bitonal Converter v.0.4
(1.91 KiB) Downloaded 363 times

Benedictus
Posts: 15
Joined: 14 Jan 2013, 00:38
E-book readers owned: None
Number of books owned: 2000
Country: Spain

Re: Bitonal conversion using Photoshop (Gimp & Imagemagic...

Post by Benedictus » 17 Jan 2013, 14:33

Very nice. Thanks.

Seeing how the filters work I guess this is a limitation in the GIMP itself, but do you think it will be possible to add a "preview" feature in the script?. Maybe not for all the settings (because some of them involve a lot of processing) but just for the levels one. That's basically the main setting I'm using.

I'm asking this because right now I play with the levels with a live preview in either Photosop or FineReader and, when I have numbers that I like, I move them to the script for fine tuning.

Best regards.
Plustek OpticBook A300

xorpt
Posts: 42
Joined: 24 Feb 2012, 01:37
E-book readers owned: Sony PRS-T1
Number of books owned: 2000

Re: Bitonal conversion using Photoshop (Gimp & Imagemagic...

Post by xorpt » 17 Jan 2013, 23:37

Unfortunately a Preview function cannot be implemented with script-fu. I would have to write a real plugin for that, not a simple script

Maybe in the future...

Benedictus
Posts: 15
Joined: 14 Jan 2013, 00:38
E-book readers owned: None
Number of books owned: 2000
Country: Spain

Re: Bitonal conversion using Photoshop (Gimp & Imagemagic...

Post by Benedictus » 10 Jun 2013, 19:24

Hi.

I've been using this script for bitonal conversion for some months and I haven't and won't look back.

It is extremely tweakable and it makes fine tuning as easy as it can get. But it has one drawback: speed. This doesn't matter when converting a few images, but it is a bit of a problem when working over large batches of files. The last book I scanned had 830 pages and it took GIMP 3 hours to process it. A bit too much.

Lately I've started using ImageMagick to merge separate sets of files (extracted pictures in one and GIMP-bitonalized files in the other) and I find it extremely powerful and fast.

I'm wondering if the steps in the bitonal conversion GIMP script can be translated into IM commands to achieve the exact same results. The aim of this will be having GIMP as a live preview and fine tuning tool and then let IM handle the hard word that currently is being done in GIMP with the Batch bitonal converter script.

I guess that levels, DoG and Gaussian blur won't be a problem, but I'm a bit lost when it comes to layers and thresholds.

So, it is possible to get an 1:1 translation between GIMP Bitonal converter and ImageMagick commands? Will the potential gain in speed be as big as I'm expecting?

Best regards.
Plustek OpticBook A300

xorpt
Posts: 42
Joined: 24 Feb 2012, 01:37
E-book readers owned: Sony PRS-T1
Number of books owned: 2000

Re: Bitonal conversion using Photoshop (Gimp & Imagemagic...

Post by xorpt » 11 Jun 2013, 06:35

Hi Benedictus, I agree that the Gimp script is not so fast, but since it's a batch, I personally don't mind the slowness... I just let it run in the background, it's just using one CPU thread anyway

As for Imagemagick I proposed some (Windows-based) scripts in the first page of this thread, that do exactly the same, but using IM commands.

Please see this post in particular:
xorpt wrote:Thanks!

I have experimented with potrace, which gives very good results, provided the input is clean, with clearly delimited lines.

The last book I converted was done using the following workflow:

1. Rename files with Ant Renamer
2. Merge the files in one directory
3. Use Scan Tailor to deskew, select contents, add margins. Export to Color, White margins, Equalize illumination.
4. In the out directory add a .bat file with the following (* note: this script will work with the latest versions of ImageMagick only)

Code: Select all

FOR %%1 in (*.tif) DO (
convert %%1 -colorspace Gray -level 20%%,80%%,.67 ^( +clone -bias 50%% -morphology convolve DoG:1,0,1 -level 0%%,50%%,.45 ^) -compose dissolve -define compose:args=70 -composite -threshold 80%% "%%~ndp1_dog.pnm"
potrace -3 -b pgm "%%~ndp1_dog.pnm"
del "%%~ndp1_dog.pnm"
)
5. After conversion, combine files in a DJVU file with Djvu Solo.

The -morphology convolve DoG is very good a tracing edges. I will try and experiment more with other methods...
You can take this as a basis and change the output format, filename, etc.

Also the results are not going to be exactly the same. To be honest I do not think the results using IM are as good as those I get using Gimp. But they are not bad at all, and maybe by fine-tuning a little more than I did can you get something to your likening.

Also I don't know about speed, I don't remember IM being super-fast. but I suppose it will be faster than GIMP (no display...)

Benedictus
Posts: 15
Joined: 14 Jan 2013, 00:38
E-book readers owned: None
Number of books owned: 2000
Country: Spain

Re: Bitonal conversion using Photoshop (Gimp & Imagemagic...

Post by Benedictus » 11 Jun 2013, 08:49

Hi, xorpt.
xorpt wrote:As for Imagemagick I proposed some (Windows-based) scripts in the first page of this thread, that do exactly the same, but using IM commands.
I've seen that post, but posted anyways because I was only able to understand this part of the command:

Code: Select all

convert %%1 -colorspace Gray -level 20%%,80%%,.67
Everything else is hard to understand to me. That's why I was asking if it is possible to mimic the exact steps performed by the GIMP script in IM.
xorpt wrote:Also the results are not going to be exactly the same. To be honest I do not think the results using IM are as good as those I get using Gimp. But they are not bad at all, and maybe by fine-tuning a little more than I did can you get something to your likening.
Sad to read this. Is there any technical reason for it? If you ask IM to perform the exact same steps being done by GIMP script, will the result be different?

Besides, if that's true, then tuning the IM command would also mean moving away from GIMP, which I'm happy with, and it will also make impossible to have GIMP as a "live preview and fine tuning tool", as I said earlier.
xorpt wrote:Also I don't know about speed, I don't remember IM being super-fast. but I suppose it will be faster than GIMP (no display...)
Well, yes. It is not that fast, but it won't take that much to outperform GIMP. Any gain in speed will be substantial I think.
Plustek OpticBook A300

xorpt
Posts: 42
Joined: 24 Feb 2012, 01:37
E-book readers owned: Sony PRS-T1
Number of books owned: 2000

Re: Bitonal conversion using Photoshop (Gimp & Imagemagic...

Post by xorpt » 11 Jun 2013, 10:37

Yes this unique command replicates as much as possible the steps of the Gimp script
convert %%1 -colorspace Gray -level 20%%,80%%,.67 ^( +clone -bias 50%% -morphology convolve DoG:1,0,1 -level 0%%,50%%,.45 ^) -compose dissolve -define compose:args=70 -composite -threshold 80%% "%%~ndp1_dog.png"
-colorspace Gray : changes the colorspace to Grayscale
-level 20%%,80%%,.67 : applies level (black point 20%, white point 80%, gamma 0.67)
( +clone ) : duplicates the current layer
-bias 50%% -morphology convolve DoG:1,0,1 -level 0%%,50%%,.45 : applies Difference of Gaussians on the duplicate layer
-compose dissolve -define compose:args=70 -composite : merges the two layers together
-threshold 80%% : applies a threshold on the resulting image to keep only two colors

The results cannot be exactly the same because the implementation of similar functions in Gimp and Imagemagick is not exactly the same: they are different programs :) but still, it's close, so I think you should test both. So I don't think that you can use Gimp as a preview tool for IM...

Maybe what you could try is to use a Gimp script-fu without the GUI interface, only the command line. I haven't tried it, but I suppose it's possible. Probably the speed would be better, since it would not have to display the result in real-time

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest