Daniel Reetz, the founder of the DIY Book Scanner community, has recently started making videos of prototyping and shop tips. If you are tinkering with a book scanner (or any other project) in your home shop, these tips will come in handy. https://www.youtube.com/channel/UCn0gq8 ... g_8K1nfInQ

Spreads Problems and Solutions

Johannes Baiter's Spreads and SpreadPi are the latest control systems and postprocessors for DIY scanning. http://spreads.readthedocs.org
duerig
Posts: 371
Joined: 01 Jun 2014, 17:04
Number of books owned: 1000
Country: United States of America

Spreads Problems and Solutions

Post by duerig » 13 Jun 2014, 10:10

I've now got Spreads installed and working. It allows for an amazing and efficient workflow. But in order to get here, I had to spend a large amount of time figuring out how to install it and its dependencies because unexpected problems kept coming up. This thread is for people with problems installing/running Spreads and what the solutions to those problems are.

I began with a clean Debian installation.

Dependency notes:

Each build of CHDK only works for a particular firmware version on your camera. My two cameras were identical in every respect except for the firmware. Check the firmware version of each camera independently before trying to install CHDK on them.

The CHDK wiki/manual says that the default button to get into Alt mode on an A2200 is the smiley face button. On mine, it was the 'play' button.

There is no scantailor Debian package, in spite of what the documentation suggests. You have to install it yourself. First, scantailor dependencies are:

Code: Select all

sudo apt-get install build-essential cmake libqt4-dev libjpeg-dev zlib1g-dev libpng-dev libtiff4-dev libboost-all-dev libxrender-dev
Then to install scantailor, you have to download the latest source of scantailor-enhanced, enter the source directory and run:

Code: Select all

    cmake .
    make
    sudo make install
pdfbeads is tricky to install because the gem code doesn't automatically bring in dependencies. But after installing it, I haven't managed to get it to work inside of Spreads, so skip it to begin with and come back once other stuff is working.

I think it is easier to try to install chdkptp from source rather than trying to get the binary to work. The reason is that the binary requires the IUP/CD libraries which are themselves quite difficult to install. But if you install from source, you can turn off GUI support and don't have to worry about them.

Once you have built or downloaded chdkptp, you will want to copy both the binary and lua directory to /usr/local/lib. So you will have /usr/local/lib/lua/*.lua and /usr/local/lib/chdkptp after. This is necessary to get chdkptp to run properly with spreads later. But it won't let you run chdkptp from the command line yourself. So I would copy all of the lua files into /usr/local/lib/lua/5.2 as well (assuming you are using lua 5.2). With two complete copies of the lua files, you will be able to use chdkptp manually and this is useful later.

Since you want to use the built-in autorotate plugin for Spreads, you will need to install the jpegtran module. Installing the libjpeg-dev apt package isn't sufficient because that doesn't export python bindings. After much searching, I found that there are a bunch of dependencies to install:

Code: Select all

sudo apt-get install python-dev libffi-dev libjpeg-dev
sudo pip install pycparser
sudo pip install cffi
sudo pip install jpegtran-cffi
These were the main trouble spots. When you try to run spread configure, if you are missing a dependency, it will throw an exception after the first few steps. So you may have to go a few extra rounds of dependency installation, especially for the web interface because its dependencies are not documented.

Spreads notes:

The current pip package of Spreads is broken, claiming that you are missing 'plugins'. So you want to install Spreads from the repository. Here is the command for doing that (make sure git is installed already):

Code: Select all

sudo pip install git+git://github.com/DIYBookScanner/spreads.git
You will need to run spread configure multiple times. The first time, only set up plugins and say no to the odd/even setup and the focus setup. Those won't work yet. After the first time, a configuration file will be generated. You need to hand-edit the configuration file to point at the chdkptp directory. The configuration file is in ~/.config/spreads/config.yaml and you can look at an example configuration file here: http://spreads.readthedocs.org/en/lates ... uring.html

You need to add a chdkptp_path as shown in the example configuration. But the path should be /usr/local/lib/ instead of what is in the example because the sample is incorrect. You will also want to add in scantailor preferences to your file by hand because those are not automatically added by the configuration. Any other device preferences except focus need to be added by hand to this file as well.

Another problem I ran into is that when I first plugged in the cameras, Spreads kept failing to connect to them. I found out that the connect primitive was returning a device busy error and Spreads was failing to parse that because it was expecting a normal return value. That failure caused an exception so Spreads never retried. A workaround I found is to run chdkptp in another console. Whenever I plug in or turn on a camera, I run 'list' in the chdkptp console and it triggers the busy message and then I can use the camera in spreads.

Once you have chdkptp_path added, you can run spread configure again to assign odd/even cameras. Make sure only one camera is on and connected to the computer at a time. If you have busy problems like I did before, just make sure to run list in chdkptp whenever you plug in a camera. You can also run the focus configuration option. Note that Spreads does not have a separate focus for each camera. It assumes that they are both approximately the same distance from the opposing page.

To take some test pictures, just run 'spread capture testdir' and after taking a couple of shots, look at the results. If you need to adjust configuration settings, just Ctrl-C and re-run. Spreads will not overwrite your pictures, just keep adding to the end. So you can compare results.

If your pages are well-lit, then the default settings will mean that your first test pages are all white. Spreads does not allow you to adjust aperture size (the F number), so you will want to dim the light by making the shutter speed faster. I found that 1/800th of a second was a good exposure time for my A2200 cameras and lighting.

Open issues:

I've installed pdfbeads and selected the pdfbeads module. But it doesn't appear in the allowed items for listing the order of plugins, and the postprocess step does not yield any pdfs.

User avatar
jbaiter
Posts: 98
Joined: 17 Jun 2013, 16:42
E-book readers owned: 2
Number of books owned: 0
Country: Germany
Location: Munich, Germany
Contact:

Re: Spreads Problems and Solutions

Post by jbaiter » 13 Jun 2014, 13:42

Hey duerig,

thank you so much for the detailed feedback!
I'm painfully aware that the software is in a pretty rough state at the moment, especially on the documentation and installation front. I'm working together with someone to rework the documentation, but I can't say yet when it will be done.

Just some comments on a few of your observations:
There is no scantailor Debian package, in spite of what the documentation suggests. You have to install it yourself. First, scantailor dependencies are:
There is, though it is only available in testing and unstable: https://packages.debian.org/de/jessie/scantailor
It should probably say so in the docs :-/
But it won't let you run chdkptp from the command line yourself. So I would copy all of the lua files into /usr/local/lib/lua/5.2 as well (assuming you are using lua 5.2). With two complete copies of the lua files, you will be able to use chdkptp manually and this is useful later.
You can use it manually if you specify the LUA_PATH environment variable:

Code: Select all

$ LUA_PATH=/usr/local/lib/chdkptp/lua/?.lua /usr/local/lib/chdkptp/chdkptp
You need to add a chdkptp_path as shown in the example configuration. But the path should be /usr/local/lib/ instead of what is in the example because the sample is incorrect. You will also want to add in scantailor preferences to your file by hand because those are not automatically added by the configuration. Any other device preferences except focus need to be added by hand to this file as well.
The chdkptp path is not "wrong", it just assumes that you installed the binary and lua scripts to /usr/local/lib/chdkptp.
As for having to add all those settings manually, you don't have to if you are fine with the defaults. Leaving them out of the configuration file will not raise an error.
If your pages are well-lit, then the default settings will mean that your first test pages are all white. Spreads does not allow you to adjust aperture size (the F number), so you will want to dim the light by making the shutter speed faster. I found that 1/800th of a second was a good exposure time for my A2200 cameras and lighting.
Thanks for pointing out the lacking aperture size setting, this can easily be added since chdkptp supports it!
I've installed pdfbeads and selected the pdfbeads module. But it doesn't appear in the allowed items for listing the order of plugins, and the postprocess step does not yield any pdfs.
Can you open an issue on GitHub for that? Preferably with a debug log attached.

Sorry again that the software is a b**** to install at the moment, I'm very much focused on the web interface and the SpreadPi distribution currently, so I've neglected some other things a bit.
spreads: Command-line workflow assistant

duerig
Posts: 371
Joined: 01 Jun 2014, 17:04
Number of books owned: 1000
Country: United States of America

Re: Spreads Problems and Solutions

Post by duerig » 27 Jun 2014, 00:26

The pdfbeads issue is a bit confusing, but I ended up just using another tool that was almost as automated to make the pdf files. I did run a scan postprocess (and configure) in debug mode and nothing even mentioning pdfbeads appeared. I've given up on it for now and might look into it again if/when I install the spreads apt package that you just set up.

I managed to get my hands on a Raspberry Pi today and installed spreadpi on it. This was a much smoother experience, and I think I will just use spreadpi instead of giving up my laptop to spreads from now. There were a few problems, but I've updated the spreadpi wiki to address the install challenges and created Github issues for the rest. All in all, using a Pi for this project seems to be a major win. WIth a standard piece of hardware and a standard whole-disk image, all the local eccentricities I had to deal with seem to be gone.

Thanks much for this great piece of software! When my lasers arrive, I may have to try my hand at adding a laser module to spreads.

aladar
Posts: 4
Joined: 07 Sep 2012, 05:11
E-book readers owned: BeBook, Samsung Galaxy Tab 10.1
Number of books owned: 300
Country: Netherlands

Re: Spreads Problems and Solutions

Post by aladar » 24 Nov 2014, 12:54

Is there a good guide on how to manually compile the chdkptp?
I'm continuously running into issues with it, the latest is that ir cannot find the shared lua libraries... and there are none, so I must be missing something here.

Thanks,
Aladar

duerig
Posts: 371
Joined: 01 Jun 2014, 17:04
Number of books owned: 1000
Country: United States of America

Re: Spreads Problems and Solutions

Post by duerig » 24 Nov 2014, 13:05

Make sure you have lua installed on your system. If you are using Ubuntu, then you might need to install lua-dev or a similarly named project with the -dev suffix.

I will try to recompile it from scratch and take detailed notes to see if I can reproduce and help you get through it. I know that I have successfully installed it at least once. :-)

aladar
Posts: 4
Joined: 07 Sep 2012, 05:11
E-book readers owned: BeBook, Samsung Galaxy Tab 10.1
Number of books owned: 300
Country: Netherlands

Re: Spreads Problems and Solutions

Post by aladar » 26 Nov 2014, 04:39

Thanks, I figured out. the config.mk had some issues.

barth
Posts: 28
Joined: 28 Sep 2014, 09:42
Number of books owned: 0
Country: Belgium

Re: Spreads Problems and Solutions

Post by barth » 08 Feb 2015, 17:04

Hello,
I have the problem that the camera triggers go off twice at a regular interval of ca. 7 seconds every time I use the keypedal. That means almost every intended capture is followed by an unintended one, which creates a lot of useless images.
I have posted this problem in the Camera dual triggering thread before, because I had wrongly attributed it to the cameras. As jbaiter pointed out, and if I recall correctly, the excess capture is caused by the default setting of the camera trigger interval on the Raspberry Pi.

Jbaiter advised me to remove the 'intervaltrigger' from the 'plugins' entry in the spreads configuration file on the Raspberry Pi ('/home/spreads/.config/spreads/config.yaml').
I don't know how to do this: I can't find the configuration file.

I run SpreadPi with the GUI/webinterface on a Windows 8 laptop using Chrome. How do I get access to the configuration file?

I can't find a file with the extension yaml on the SDHC-card so guess that's not where to look. And I don't know how to look on the Pi itself because I only access it through the graphic web interface, via a LAN router. I greatly appreciate any help with this.

User avatar
jbaiter
Posts: 98
Joined: 17 Jun 2013, 16:42
E-book readers owned: 2
Number of books owned: 0
Country: Germany
Location: Munich, Germany
Contact:

Re: Spreads Problems and Solutions

Post by jbaiter » 08 Feb 2015, 17:19

Is the keypedal attached to the computer with the web interface or the Pi?
If it is attached to the Pi, are there any input devices attached to it besides the keypedal?
As for the configuration file, you can find it on the Pi. SSH into it (using e.g. PuTTY on Windows) and modify it with a texteditor like 'nano'.
spreads: Command-line workflow assistant

barth
Posts: 28
Joined: 28 Sep 2014, 09:42
Number of books owned: 0
Country: Belgium

Re: Spreads Problems and Solutions

Post by barth » 08 Feb 2015, 18:20

Thanks for your swift and helpful reply.
The keypedal is attached to the Pi, either directly or via the PiHub USB-hub. There are no other input devices attached to it.
A router is connected to its ethernet port to create a LAN through which the computer accesses the webinterface.
I use the webinterface to create and edit workflows and save images to the computer, but it is rarely, if ever, used to trigger the cameras directly. I prefer to use the pedal for that because I need both hands to manipulate the cradle and the book.

User avatar
jbaiter
Posts: 98
Joined: 17 Jun 2013, 16:42
E-book readers owned: 2
Number of books owned: 0
Country: Germany
Location: Munich, Germany
Contact:

Re: Spreads Problems and Solutions

Post by jbaiter » 08 Feb 2015, 18:31

That sounds pretty much identical to my current setup.
Can you share a debugging log from one of you workflows? You can get it from the web interface.
Also, which version of the SpreadPi image are you using? I completely rewrote the CHDK driver two weeks ago, maybe that has something to do with it?
spreads: Command-line workflow assistant

Post Reply