Daniel Reetz, the founder of the DIY Book Scanner community, has recently started making videos of prototyping and shop tips. If you are tinkering with a book scanner (or any other project) in your home shop, these tips will come in handy. https://www.youtube.com/channel/UCn0gq8 ... g_8K1nfInQ

Can the lights be too bright?

All about lighting. LED, CFL, Halogen, Other? Questions and info about lighting go here.
cday
Posts: 224
Joined: 19 Mar 2013, 14:55
Number of books owned: 0
Country: UK

Re: Can the lights be too bright?

Post by cday » 23 Jun 2015, 13:51

wmalcolmk wrote:Questions/comments from duerig and cday

cday "...wouldn't the evenness of the overall illumination be better represented by plots at different intensity values?"

I think the width of the histogram is the best measure of evenness of illumination, since perfect illumination would result in every pixel having the same brightness - giving a single vertical line on the histogram.
Yes, the width of the histogram is a simple measure of the evenness of illumination, and ideally the histogram would be a single vertical line, but plots at different intensity levels (ideally calibrated levels... ) would indicate how the intensity varies at different locations on the reference sheet, which could be useful information...

cday
Posts: 224
Joined: 19 Mar 2013, 14:55
Number of books owned: 0
Country: UK

Re: Can the lights be too bright?

Post by cday » 23 Jun 2015, 14:21

Further...

Trying to think of a way of directly indicating the evenness of illumination, I downloaded the reference image and then applied a Levels adjustment to it, placing the white and black points at the approximate limits of the intensities in the histogram; I don't know how well reasoned that is, but the result is an image that intuitively seems to provide a useful indication of the evenness of illumination:
Daylight1A.JPG
The reference image as downloaded.
Daylight1A.JPG (19.5 KiB) Viewed 3294 times
Levels adjustment (XnViewMP).png
The White and Black points moved to the edges of the histogram.
Levels adjustment (XnViewMP).png (14.09 KiB) Viewed 3294 times
Daylight1A_Levels.jpg
The reference image after a levels adjustment.
Daylight1A_Levels.jpg (26.76 KiB) Viewed 3294 times
I then converted the original camera image to grayscale:
Daylight1A_Grayscale.jpg
The reference image converted to grayscale.
Daylight1A_Grayscale.jpg (10.82 KiB) Viewed 3289 times
And then applied a Levels adjustment as above:
Daylight1A_Grayscale_Levels.jpg
The reference image converted to grayscale after a levels adjustment.
Daylight1A_Grayscale_Levels.jpg (27.51 KiB) Viewed 3294 times
The results seem intuitively useful in illustrating the variation in illumination, although different levels settings could probably as well be used to exagerate the variation intensity, if needed. The potential use would be to assist in optimising the evenness of the lighting, which in practice determines the degree of enhancement that can be applied to colour and grayscale images, and hence the quality of the final images.

dpc
Posts: 270
Joined: 01 Apr 2011, 18:05
Number of books owned: 0
Location: Issaquah, WA

Re: Can the lights be too bright?

Post by dpc » 23 Jun 2015, 16:27

Then you just use the midle part of the lens and therefore get more even light in the corner.
Sure, but if you do this you'll be throwing away a portion of the capability your camera's sensor and decreasing the DPI of your image.

How much falloff are you seeing toward the corners of your shot and can you correct for in a post-processing step?

If that's not possible then could you take multiple shots, varying the exposure, the use a blend mask based on the light values across a blank page to determine the contribution of each shot during the blend operation?

wmalcolmk
Posts: 24
Joined: 12 May 2015, 04:39
Number of books owned: 500
Country: England

Re: Can the lights be too bright?

Post by wmalcolmk » 24 Jun 2015, 10:03

Footsie Platen Lighting

I need to decide if more work is needed on my lighting system, since there are other aspects of the design that need attention.
Here are the results of photographing a white test sheet.
Footsie24jun1.JPG
Footsie24jun1.JPG (27.35 KiB) Viewed 3264 times
It was taken using a Canon Powershot A810, file size 2.9MB, 4608x3456 pixels, 1/50 at f/3.5, iso 400, focal length 9.16mm, white balance set for tungsten, light metering - evaluative.
Illumination is from warm white led strips.
The test sheet was placed on top of the glass platen, to avoid any influence from reflections. The photo above is at reduced resolution, but the following tests were done on the unretouched image, including the strip at the bottom of the picture.

Histogram using Gimp 2
Footsie24jun2.JPG
Footsie24jun2.JPG (31.67 KiB) Viewed 3264 times
Threshold image at histogram peak
Footsie24jun3.JPG
Footsie24jun3.JPG (157.08 KiB) Viewed 3264 times
There are a number of artefacts showing up on the threshold image, probably due to stray light entering the scanner, or internal reflections, or possibly a consequence of my lighting method. I could spend time investigating these, but will press on with other work and only come back if they cause poor quality scans. The thresholding technique should be a powerful way of pinning down these artefacts.

Malcolm

rolling29
Posts: 10
Joined: 18 Oct 2015, 10:57
E-book readers owned: nook, nook simple
Number of books owned: 600
Country: USA

Re: Can the lights be too bright?

Post by rolling29 » 24 Oct 2015, 00:10

Yes! I very much enjoyed reading this thread. (Sorry that I just found it and it is already several months old...)

Some memories from my past:
1. yes, in general the more narrow the histogram the better
2. re-scaling the width from [whatever range you get] to 0-255 is the best way to emphasize the (un)evenness of the lighting you have and show you where the improvements need to be made
3. don't just overexpose so it looks better! take your test images with the exact same equipment and parameters you will scan with (lens, ISO, f-stop, shutter, white balance, etc)
4. don't target 100% levels, it only means you can fool yourself easily - shoot for 200-225 (0-255) and a width as narrow as possible for the most useful image to correct by

My answer to the original question of the thread - yes the lights can be too bright, and that can hide other problems. It depends on your camera, lens, and parameters, but if you are washing everything out to white there is too much light and at least one parameter needs to change. Having enough light is usually pretty easy. Making it really nice and even (with no reflections) is the fun bit.

When you get that light spread even enough from corner to corner so that the re-scaled threshold image is showing the clumps in the fiber distribution in the paper or stray diffraction patterns, you may stop tweaking and be satisfied. :D

I'd like to add that these histogram transformations are useful for post processing much like dark frames are for astro-photography image stacking. (Maybe it already has come up, I haven't read all the years of posts yet...) You can apply the light calibration image as a mask on your captured images. As long as the lighting is consistent you can correct for evenness in post process and have an extremely even looking output image by adding brightness proportionally based on the darker areas of the calibration image.

I sometimes feel like I've woken from a coma. What is github and why do I not understand anything anymore? Sorry, my point was that I read about you all using GIMP etc. to do this testing, when that is a very time consuming way to get it done. I would like to finish and share my imageMagick script that tries to make these re-scaled brightness maps. Maybe someday it works with a lighting calibration correction plugin that can be applied in spreadpi as it does the page rotation. More fun things to think and do...

rolling29
Posts: 10
Joined: 18 Oct 2015, 10:57
E-book readers owned: nook, nook simple
Number of books owned: 600
Country: USA

Re: Can the lights be too bright?

Post by rolling29 » 24 Oct 2015, 00:55

Ok, I just checked up on imageMagick and I feel pretty dumb. A lot has changed since I last used it...

(using wmalcomk's daylight test image)
Daylight1A.JPG
Daylight1A.JPG (19.5 KiB) Viewed 3034 times
All you need for re-scaled visual feed back is this:

Code: Select all

convert Daylight1A.JPG -colorspace Gray -auto-level out-autolevel-1shot.jpg
out-autolevel-1shot.jpg
out-autolevel-1shot.jpg (35.96 KiB) Viewed 3034 times
Then another "script"...

Code: Select all

convert Daylight1A.JPG -colorspace Gray -auto-level -negate out-autolevel-invert-1shot.jpg
out-autolevel-invert-1shot.jpg
out-autolevel-invert-1shot.jpg (35.96 KiB) Viewed 3034 times
And this would be an example of the "calibration" image that could be applied to the scans in spreadpi. It would still take some doing to evaluate the calibration and determine the scale of how much brightness to apply, but the map makes it pretty clear where and in what proportion for any given scanner with whatever lighting situation it currently has. (assuming a repeatable "archivist" style setup with platen and fixed camera position / lighting)

I'd also like to see the imagemagick generated image contain some quantifications. What value does the minimum represent? The range, maximum, mean, etc? It's already clear that the bottom right needs more light than the top left, but how much more? Looking at the original, all my eyes see is a nice even gray box...

cday
Posts: 224
Joined: 19 Mar 2013, 14:55
Number of books owned: 0
Country: UK

Re: Can the lights be too bright?

Post by cday » 24 Oct 2015, 16:12

Successful implementation of this process, whether using standard software, a standalone application or integrated into spreadpi, could be of considerable benefit in two situations where uneven illumination sets a limit on the enhancement that can be successfully applied to the page image.

When pages need to be imaged in colour or grayscale due to the content of the page (because they contain colour content or half-tone illustrations, for example) the ability to apply additional enhancement will result in an image that is both a better representation of the original page and more pleasing to view.

When black and white pages are imaged at resolutions that are insufficient to render text to best effect in black and white, which typically requires up to 600DPI, due to camera or other limitations, text on a page that can be enhanced to an attractive black and white image and is then saved in grayscale or colour benefits from anti-aliasing, smoothing of the outline of the text characters using subtle shades of gray, which can render text more attractively as if it were imaged at a substantially higher DPI value.

rolling29
Posts: 10
Joined: 18 Oct 2015, 10:57
E-book readers owned: nook, nook simple
Number of books owned: 600
Country: USA

Re: Can the lights be too bright?

Post by rolling29 » 27 Oct 2015, 00:07

So here goes my attempt at quantifying and correcting lighting...

Still using the Daylight1A.JPG source file for testing, let's assume it's a plain white paper test image taken by whatever camera and lighting you have available. (I think it was, outdoors?) This command will convert that file into a grayscale image and dump out some useful numbers that can be hoovered up into automation later:

Code: Select all

convert Daylight1A.JPG -colorspace Gray -verbose info: | grep -E 'min|max|mean'
      min: 110 (0.43209)
      max: 148 (0.579721)
      mean: 134.673 (0.528131)
So now I know that this test image has a brightness range from 43-58%. For now I'm not interested in evaluating the evenness quality of the lighting, so I'll skip generating the autolevel scaled contrast images for now. I just want to make a compositing mask that will even out the scanned image somehow, and after banging my head on the keyboard a while I came up with this command:

Code: Select all

convert Daylight1A.JPG -colorspace Gray -level 58%,43% +level 0%,15% correction-mask.jpg
If I understand correctly this will invert the range, re-scale the range, and shift things around magically to whatever they need to be. (Hey, now I know why they call it imagemagick...) We end up with a very dark image like this:
correction-mask.jpg
correction-mask.jpg (16.49 KiB) Viewed 2992 times
If you run a similar grep command on this new dark image it tells you that the brightness ranges from 0-15%. That's the same brightness range we started with, so it looks promising. We want it shifted down to the dark side because the best way to do a composite brightness correction is by the "screen" method. (I think.) Black is transparent in this process and never darkens, only lighter areas lighten, your other image. In fact if your scanner had perfect lighting the correction mask would come out perfectly black and the scanned image wouldn't be changed by this process because it wouldn't need to be. (Sanity check - that sounds like circular logic, but it isn't... right?)

Ok, so now we assume we have this dark correction image, our optical junk is rock solid so the lighting doesn't really change much between scans, and we've now got a bunch of pages with darkness around the edges because we live in the real world and we don't have perfect lighting. Well now we do that "screen" compositing thingy...

Code: Select all

composite -compose Screen -gravity center correction-mask.jpg Daylight1A.JPG Daylight1A-corrected_brightness.jpg
And it spits out an image like this:
Daylight1A-corrected_brightness.jpg
Daylight1A-corrected_brightness.jpg (18.47 KiB) Viewed 2992 times
And the brightness profile comes out like:

Code: Select all

convert Daylight1A-corrected_brightness.jpg -colorspace Gray -verbose info: | grep -E 'min|max|mean'
      min: 131 (0.513573)
      max: 149 (0.583368)
      mean: 141.149 (0.553527)
Honestly, it still looks like a flat gray box to my eyes. I can't tell the difference unless I load up an image viewer and swap them back and forth quickly.

So a little better but still kind of a failure. Good experiments always fail, don't they? I took it from about a 15% variation in brightness to about 7%, but at least I didn't change the maximum brightness level hardly at all. (less than 0.5%) Maybe a grayscale alpha correction on a color image isn't the best approach. Maybe my math is wrong somewhere or I fat-fingered a command. There's always more to learn and test. Anyway, I need to go try this on a bunch of real pages. I hope anybody following my mental bread crumbs can figure out what I'm trying to explain. I hope a bunch of you out there will show me what I did wrong, so I don't have to figure it out all on my own. ;) With more experimentation some better target values for max brightness, range, and correction limits should become apparent. How this affects visual contrast or clarity could be a deal breaker. Combined with further post-processing I have hope that this isn't all wasted effort.

Regarding work flow integration, theoretically you could do this lighting calibration process once on your scanner and just save the correction mask image to be applied as often as you want later. Or at least until something in the optical tool chain changes somehow. Maybe start a book session with a pair of blank white pages so that set of images can be calibrated to the particular age of the light bulbs, or whatever else might have an effect that day. Two extra images out of hundreds isn't that much trouble compared to the processing required of astro-photography image stacking.

rolling29
Posts: 10
Joined: 18 Oct 2015, 10:57
E-book readers owned: nook, nook simple
Number of books owned: 600
Country: USA

Re: Can the lights be too bright?

Post by rolling29 » 27 Oct 2015, 01:57

wmalcolmk wrote:Footsie Platen Lighting

I need to decide if more work is needed on my lighting system, since there are other aspects of the design that need attention.
...
Malcolm
Looks like the "-auto-level" doesn't like it when the scan area isn't all white / needs cropping. The minimum level is already quite dark.
Footsie24jun1-gray-autolevel.jpg
Footsie24jun1-gray-autolevel.jpg (24.92 KiB) Viewed 2987 times
Now that I figured out how to find the min|max|mean levels easily, I run the original through and get 1% | 61% | 54% after convert to grayscale. So instead of using min and max like auto-level does, I used (max-(max-mean)*2) and max and got not bad results using 47%,61%. Do you have a window with mini-blinds near the scanner? That was my first impression of those light and dark horizontal bands. Oh wait, didn't you say you were using LED strip lighting? Is that what I'm seeing?
Footsie24jun1-gray-manulevel.jpg
Footsie24jun1-gray-manulevel.jpg (42.66 KiB) Viewed 2987 times
Generated by:

Code: Select all

convert Footsie24jun1.JPG -colorspace Gray -level 47%,61% -verbose Footsie24jun1-gray-manulevel.jpg
Anyway...

Code: Select all

convert Footsie24jun1.JPG -colorspace Gray -level 62%,46% +level 0%,16% Footsie-correction-mask.jpg
Making the correction mask from the max and mean formula instead of max and min gives this:
Footsie-correction-mask.jpg
Footsie-correction-mask.jpg (19.64 KiB) Viewed 2987 times

Code: Select all

composite -compose Screen -gravity center Footsie-correction-mask.jpg Footsie24jun1.JPG Footsie24jun1-corrected_brightness.jpg
And then applying it to the original image with the screen composite gives this:
Footsie24jun1-corrected_brightness.jpg
Footsie24jun1-corrected_brightness.jpg (23.5 KiB) Viewed 2987 times
And for easy comparison here's the original:
Footsie24jun1.JPG
Footsie24jun1.JPG (27.35 KiB) Viewed 2987 times
To me it seems the faint bright/dim horizontal bands of the original are diminished after the correction mask is applied, but still there, and that lower left corner is getting closer to match the rest of the paper too. My conclusion so far is that this method can help even up the page appearance, but it still needs plenty of experimenting and a little more "becoming a useful tool" type work.

dpc
Posts: 270
Joined: 01 Apr 2011, 18:05
Number of books owned: 0
Location: Issaquah, WA

Re: Can the lights be too bright?

Post by dpc » 27 Oct 2015, 16:27

I've been using this technique for a few years now. I wrote my own post-processing routines and the first stage of the pipeline corrects for uneven lighting using this method.

Before each scan I take a photo of a 18% 8x10 gray card placed under each half of the platen. The resulting images are then used to determine how much I need to darken or lighten subsequent scanned pages on a per-pixel basis. The gray card represents a baseline that is midway between white and black. Values in the image of the gray card that vary from the midpoint tell us how far to push/pull the values in the scanned pages in post-processing.

In the simple case, the calculation to adjust the scanned page looks like this:
output(x, y) = input(x, y) + (0.5 - gray(x, y))

This can be done using a simple pixel shader that executes entirely on the GPU. In my current system this happens as the individual images are pulled from the camera after each shot. I try to do as much work as possible in parallel during the acquisition phase to keep the apparent post-processing time to a minimum. It also allows viewing the post-processed images in real time.

I've also been toying with the idea of using the gray card image to detect a dirty platen. The values in the gray card image shouldn't change radically between adjacent pixels even with uneven lighting, but they can change that quickly if there are specks/dirt/dust on the glass. Taking a second series of shots of the gray card after completion of the scanned book and comparing with the first set could also tell you if something got on the glass during the scan. Knowing the location of those specks would help with eliminating them in an automated manner in the scanned images.

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest