Daniel Reetz, the founder of the DIY Book Scanner community, has recently started making videos of prototyping and shop tips. If you are tinkering with a book scanner (or any other project) in your home shop, these tips will come in handy. https://www.youtube.com/channel/UCn0gq8 ... g_8K1nfInQ

Scan Tailor

Scan Tailor specific announcements, releases, workflows, tips, etc. NO FEATURE REQUESTS IN THIS FORUM, please.
Locked
spamsickle
Posts: 596
Joined: 06 Jun 2009, 23:57

Re: Scan Tailor

Post by spamsickle » 30 Mar 2010, 17:49

The "select content" step seems to be a bottleneck for many. You say you don't need to select anything except the entire page. Is the automatic selection clipping out a lot of content you want to retain, or would you just prefer to save time by skipping what you regard as an unnecessary step?

If the skew is failing most of the time, it may be that ST is picking up on the bars that connect notes and trying to make those lines horizontal. If that is the case, it may be that this is not the right tool for the job you want to do, and something like Photoshop batch commands would be better suited.

daphnis

Re: Scan Tailor

Post by daphnis » 30 Mar 2010, 18:34

Thanks for your reply. It isn't necessarily the case where the "select content" bounding box is removing active data, just that when these images are scanned, they're scanned on a profile that is based on static measurements, ie a profile is created that scans a set area of the glass. I want these images to retain their frame size and to only manipulate the data within. So again, if possible, I'd simply either like to deactive the "select content" option, or perhaps I can simply set the "content" box on the first image, and apply that to all other images (since all images I'd process are the exact same dimensions).

I wouldn't say skew is failing all the time, but some of the time. How difficult would it be to give this program (which is terrific, by the way) the option to select the type of source material? And regarding your photoshop option, I have used droplets in the past to automate this process but dislike the shear rotation Photoshop et al uses to perform the image deskewing. Part of the reason that I'm seeking a new work flow is primarily to 1.) Decrease amount of time spent post-processing scanned images and 2.) Increase quality of post-processed image, especially after an automated deskewing process.

spamsickle
Posts: 596
Joined: 06 Jun 2009, 23:57

Re: Scan Tailor

Post by spamsickle » 31 Mar 2010, 08:33

I have succeeded in setting the content box for the first image and applying that selection to all the images in a batch, but I did it outside of Scan Tailor itself.

In Scan Tailor's File menu, there is an option to "Save Project As", which writes an XML file of all the current settings. Within that file, the content selections are tagged with "content-rect", which provides the location and dimensions of the selection for that image. Using an editor which can operate on regular expressions (I used UltraEdit), it is a simple matter to set all of these to the same value. It sounds like for what you want to do, the "location" would always be x="0" and y="0", and the width and height would be the dimensions of the image itself. If you would be interested in trying such a scheme yourself, and need some assistance, I'd be happy to help.

Tulon doesn't seem too interested in implementing such a function within Scan Tailor. I suspect it's because he realizes that the function would not be as useful as those who are requesting it imagine it would be, but perhaps it's just because he has higher priorities for enhancements and will get to this eventually. I hope to implement it myself, but so far I haven't had the time (or perhaps the talent) to actually dive in and get it done.

If you want to turn off skew correction, I think it might be possible to do that also by manipulating the XML file, but the regular expression would be more complicated. What you'll be changing will be a multi-line parameter called "rotated-page-outline".

Amjad
Posts: 8
Joined: 31 Mar 2010, 01:49

Re: Scan Tailor

Post by Amjad » 31 Mar 2010, 14:30

Tulon wrote:This allowed people to write scripts that would separate Scan Tailor's Mixed TIFFs into pairs of "text only" and "pictures only" images and feed them to command-line JB2 and IW44 encoders.
That's definitely the bestest and only smart way of doing it.
Can Scan Tailor output those separately? Or can you provide/point to a script/patch that would make it do this?

Turning debug on in Scan Tailor, I can see the pieces are already there, but I find no option to save those intermediary stages somewhere.. but, they are already there, so its only a matter of offering an option to dump them on disk.. :)

By the way, Scan Tailor rocks! Thank you so much for such a wonderful software!

daphnis

Re: Scan Tailor

Post by daphnis » 31 Mar 2010, 23:32

Thanks for the suggestion. Am I allowed to put in a feature request for an optimization (sheet music)? If so, let this serve as a big ol' fat one. Aside from that, I'd like to see quicker process times, especially on multi-core machines.

Mandor
Posts: 24
Joined: 28 Jul 2009, 01:27
E-book readers owned: lBook V8, lBook V3
Number of books owned: 0
Location: Sofia, Bulgaria

Re: Scan Tailor

Post by Mandor » 01 Apr 2010, 01:06

Amjad wrote:Can Scan Tailor output those separately?
No.
Amjad wrote:Or can you provide/point to a script/patch that would make it do this?
I tested two programs - LayerTailor (didn't work for me, I don't know why) and ST Separator (works fine in my projects).

Amjad
Posts: 8
Joined: 31 Mar 2010, 01:49

Re: Scan Tailor

Post by Amjad » 01 Apr 2010, 09:21

Thanks Mandor for the pointers.
Layer Tailor just outputs white pages! Broken.
ST Separator speaks Russian only! Niet !

But. Separator (before you edited your post) actually does something useful: it has the same output as Scan Tailor + the text layer. So, technically speaking, it doesn't separate the image into a {graphics,text} couple images.
I realized the text layer (foreground, DjVu speaking) in both Scan Tailor & Separator goes through a B/W stage (hence the "Mixed" denomination ST uses, I guess). Well, that's bad too. I'm scanning a book right now, and the text (font, size, thickness) is too "dim". I'd like to make it thicker, but working from the b/w version is no good, when I can achieve awesome results from the greyscale original..
Let me show you :
Scan Tailor :
Image
Gaussian blur + levels on the grayscale original :
Image

So, finally, I'm not only asking for both layers to be separated, but also for them to be untouched.
I understand ST tries to come up with a nice final result in one go, which is great. I also understand I can't ask Tulon to replace the binarization with a blur + level adjust : these have to be tuned depending on the "darkness" of the scanned original - darkness is good for "gray images", but keeps more speckles, and if the paper is thin, details from the other side of it.. I know first hand. Binarization is much safer. And what makes text great (like above) destroys details in graphics, so text and graphics *have to* be processed separately (especially that the both binarizing or adjusting levels actually have the awseome side effect of despeckling/cleaning up the image - level adjust is more subtle though).

Conclusion : I will have to do it myself (I hate when that happens! :mrgreen: ).
This also means : all I need is the mask : a completely black b/w image, with ST "zones" in white. That's all.
This thing is already computed by ST for its zones identification. If you turn debug on, it will be the bw_mask tab.
I think Tulon can add a couple lines to write this on disk in 30 seconds or so, but he seems a little busy nowadays, and the task seems easy enough for me to try a stab right in the dark middle of the code..
After that, it will be easy to apply the mask with ImageMagic's composite tool (mathematical Multiply) to get separate graphics from text into two images, untouched.

Let me have a sneak peek at ST's source to see if I can actually dump the computed graphics mask on disk at some point.
I'm afraid my C++ skills are too rusty now (too much Unix/Linux scripting :roll: ), so it'll be more like 30 minutes for me. With coffee.

Amjad
Posts: 8
Joined: 31 Mar 2010, 01:49

Re: Scan Tailor

Post by Amjad » 01 Apr 2010, 10:43

Well, I have good news and bad news.
The good news is the mask is already dumped to disk, even if no debug! (Thanks Tulon for reading my mind!).
I realized a subdirectory "automask" is created in the cache, and the mask is written there, always. Great!
(by the way, I found it right away in Task.cpp, so my C++ is not that rusty, I guess (even though all the Qt stuff makes my eyes pop)).

So problem solved?
Well, almost. Playing with it, I discovered the mask doesn't retain manual zones! ST misses some of the graphics sometime, so I add a zone manually, but that zone is not in the mask when batch processing is finished!

There is hope though.
Manual zones are basically polygons (ST code calls them splines), which are just a series of points. and those points series (for manual zones) are saved to the project file.
So the info is there, one could add those manual zones to the mask, but this means modifying an image: no more sweet ImageMagic command line tools, graphics library needed! Need to apply the mask + the polygons (reconstructed from ST's project file).

I'll try to look around the web for a graphics lib to apply the mask first, if I can (I couldn't possibly do it with Qt, too much to learn for this simple task).
It would be much easier though, if ST could retain manual zones in the mask dumped in cache/automask, instead of the project file! Or dumping a other mask with manual zones included. Tulon?

Tulon
Posts: 687
Joined: 03 Oct 2009, 06:13
Number of books owned: 0
Location: London, UK
Contact:

Re: Scan Tailor

Post by Tulon » 01 Apr 2010, 15:19

Amjad, what you are trying to do is not the way to go. You've already found one reason you can't use automask cache files, and there are more - all of them technical and boring. I would think the easiest thing to do would be to ask the author of ST Separator to make an English version of it. It's open-source BTW, so we could do it even without the help of its author.

Regarding binarization vs blurring:
Binarized output only looks bad when zoomed in. When you are actually reading a digitized book, zoom level will be less than one. Such a zoom level implies downsampling and makes antialiasing possible. Any good PDF or DJVU reader will do antialiasing for you, so you'll get your smooth edges anyway.

BTW, I am going to release Scan Tailor 0.9.8 this weekend.
Scan Tailor experimental doesn't output 96 DPI images. It's just what your software shows when DPI information is missing. Usually what you get is input DPI times the resolution enhancement factor.

Amjad
Posts: 8
Joined: 31 Mar 2010, 01:49

Re: Scan Tailor

Post by Amjad » 02 Apr 2010, 00:31

Tulon wrote:Amjad, what you are trying to do is not the way to go. You've already found one reason you can't use automask cache files, and there are more - all of them technical and boring.
Would it really be that hard to add manual zones to the mask?
Didn't think so. Please ignore my request in that case. Scan Tailor is already great as it is anyway!
And, besides, I actually achieve it myself fairly easily, so don't bother.
Tulon wrote:I would think the easiest thing to do would be to ask the author of ST Separator to make an English version of it. It's open-source BTW, so we could do it even without the help of its author.
I agree. But I like Scan Tailor more :) and several things raised alarms about his project: 1) svn not working 2) ST Separator's code not readily available (for an open source project..) 3) ST Separator looks like a side-project 4) Language is C# (which I don't know at all) ...
Tulon wrote:Regarding binarization vs blurring:
Binarized output only looks bad when zoomed in.. Any good PDF or DJVU reader will do antialiasing for you, so you'll get your smooth edges anyway.
Oh really ?! I didn't see it that way, but I think I can safely assume you know more about the subject than me.

The thing is, here I wanted to modify the text, namely, to make it bolder. The original is in tiny thin characters, and I'm afraid my eyes are not that good!
That is why I still think having the text and pictures (un-processed) in two separates sets is still useful, for different reasons (not necessarily for blurring/smoothing the text). It's just more flexible, and gives users more options.
Now, I understand that 98% of the users would rather have 1 button to press and just enjoy the result. I understand your wisdom.

For the other 2%, its actually not that hard to update the mask from cache/automask with manual zones (from project file) and apply it to the original to get text/pictures in two sets of files. I did it fairly easily using a graphics lib (CxImage, the first I tried), so, if anyone is interested, I'm willing to share.

Can't wait to get my hands on ST 0.9.8! Thanks for the hard work.

Locked