Daniel Reetz, the founder of the DIY Book Scanner community, has recently started making videos of prototyping and shop tips. If you are tinkering with a book scanner (or any other project) in your home shop, these tips will come in handy. https://www.youtube.com/channel/UCn0gq8 ... g_8K1nfInQ

Introducing spreads: command-line workflow tool

General discussion about software packages and releases, new software you've found, and threads by programmers and script writers.
spomwii
Posts: 69
Joined: 17 Jun 2011, 07:57

Re: Introducing spreads: command-line workflow tool

Post by spomwii » 25 Dec 2013, 14:51

Thanks! Looking forward to test this. Is it possible to upgrade from the old version? Or should I make a "clean" install?

User avatar
jbaiter
Posts: 98
Joined: 17 Jun 2013, 16:42
E-book readers owned: 2
Number of books owned: 0
Country: Germany
Location: Munich, Germany
Contact:

Re: Introducing spreads: command-line workflow tool

Post by jbaiter » 25 Dec 2013, 15:02

spomwii wrote:Thanks! Looking forward to test this. Is it possible to upgrade from the old version? Or should I make a "clean" install?
If you've installed via pip, it should be enough to run "pip install -U spreads". Be aware of the changed external dependencies, though (chdkptp) :-)
spreads: Command-line workflow assistant

spomwii
Posts: 69
Joined: 17 Jun 2011, 07:57

Re: Introducing spreads: command-line workflow tool

Post by spomwii » 25 Dec 2013, 18:05

Hi,

Thanks again.

I tried a new install on a new Ubuntu 12.04 and ran into som problems.
After installing all the dependencies in you guide I tried to run the command spread configure and I got this error message.

Code: Select all

tommy@tommy-scanner:~$ virtualenv ~/.spreads
New python executable in /home/tommy/.spreads/bin/python
Installing distribute.............................................................................................................................................................................................done.
Installing pip...............done.
tommy@tommy-scanner:~$ source ~/.spreads/bin/activate
(.spreads)tommy@tommy-scanner:~$ pip install spreads
Downloading/unpacking spreads
  Downloading spreads-0.4.tar.gz
  Running setup.py egg_info for package spreads
    
Downloading/unpacking colorama>=0.2.5 (from spreads)
  Downloading colorama-0.2.7.tar.gz
  Running setup.py egg_info for package colorama
    
Downloading/unpacking pyusb>=1.0.0a3 (from spreads)
  Downloading pyusb-1.0.0b1.tar.gz (44Kb): 44Kb downloaded
  Running setup.py egg_info for package pyusb
    
Downloading/unpacking PyYAML>=3.10 (from spreads)
  Downloading PyYAML-3.10.tar.gz (241Kb): 241Kb downloaded
  Running setup.py egg_info for package PyYAML
    
Downloading/unpacking Wand>=0.3.1 (from spreads)
  Downloading Wand-0.3.5.tar.gz (41Kb): 41Kb downloaded
  Running setup.py egg_info for package Wand
    
Downloading/unpacking stevedore>=0.9.1 (from spreads)
  Downloading stevedore-0.13.tar.gz (760Kb): 760Kb downloaded
  Running setup.py egg_info for package stevedore
    
    Installed /home/tommy/.spreads/build/stevedore/pbr-0.5.23-py2.7.egg
    [pbr] Processing SOURCES.txt
    warning: LocalManifestMaker: standard file '-c' not found
    
    warning: no previously-included files found matching '.gitignore'
    warning: no previously-included files found matching '.gitreview'
    warning: no previously-included files matching '*.pyc' found anywhere in distribution
    warning: no files found matching '*.py' under directory 'tests'
Downloading/unpacking futures>=2.1.4 (from spreads)
  Downloading futures-2.1.5.tar.gz
  Running setup.py egg_info for package futures
    
Installing collected packages: spreads, colorama, pyusb, PyYAML, Wand, stevedore, futures
  Running setup.py install for spreads
    
    changing mode of build/scripts-2.7/spread from 664 to 775
    changing mode of /home/tommy/.spreads/bin/spread to 775
  Running setup.py install for colorama
    
  Running setup.py install for pyusb
    
  Running setup.py install for PyYAML
    checking if libyaml is compilable
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c build/temp.linux-i686-2.7/check_libyaml.c -o build/temp.linux-i686-2.7/check_libyaml.o
    build/temp.linux-i686-2.7/check_libyaml.c:2:18: fatal error: yaml.h: No such file or directory
    compilation terminated.
    
    libyaml is not found or a compiler error: forcing --without-libyaml
    (if libyaml is installed correctly, you may need to
     specify the option --include-dirs or uncomment and
     modify the parameter include_dirs in setup.cfg)
    
  Running setup.py install for Wand
    
  Running setup.py install for stevedore
    [pbr] Reusing existing SOURCES.txt
  Running setup.py install for futures
    
Successfully installed spreads colorama pyusb PyYAML Wand stevedore futures
Cleaning up...
(.spreads)tommy@tommy-scanner:~$ spread configure
Traceback (most recent call last):
  File "/home/tommy/.spreads/bin/spread", line 31, in <module>
    import spreads.cli as cli
  File "/home/tommy/.spreads/local/lib/python2.7/site-packages/spreads/cli.py", line 36, in <module>
    import spreads.vendor.confit as confit
ImportError: No module named vendor.confit
Do you have any idea whats wrong here?

User avatar
jbaiter
Posts: 98
Joined: 17 Jun 2013, 16:42
E-book readers owned: 2
Number of books owned: 0
Country: Germany
Location: Munich, Germany
Contact:

Re: Introducing spreads: command-line workflow tool

Post by jbaiter » 25 Dec 2013, 18:13

A stupid mistake of mine... The distribution package was missing some modules...
I already had them installed on my machine, that's why I didn't catch it. I just pushed 0.4.1 to GitHub and PyPi, please run 'pip install -U spreads' and try again.
Sorry!
spreads: Command-line workflow assistant

spomwii
Posts: 69
Joined: 17 Jun 2011, 07:57

Re: Introducing spreads: command-line workflow tool

Post by spomwii » 25 Dec 2013, 18:16

Thanks! Now I got to the configuraton menu :)

I will report back when I have done some testing :)

spomwii
Posts: 69
Joined: 17 Jun 2011, 07:57

Re: Introducing spreads: command-line workflow tool

Post by spomwii » 25 Dec 2013, 18:18

Hmm. Another problem after selecting plugins.

Code: Select all

(.spreads)tommy@tommy-scanner:~$ spread configure
Please select a device driver from the following list:
  [1]: chdkcamera
  [2]: a2200
  [3]: dummy
Select a driver: 2
Selected "a2200" as device driver
Please select your desired plugins from the following list:
    1: scantailor
    2: djvubind
    3: gui
    4: colorcorrect
    5: autorotate
    6: tesseract
    7: pdfbeads
Select a plugin (or hit enter to finish): 1
  ✔ 1: scantailor
    2: djvubind
    3: gui
    4: colorcorrect
    5: autorotate
    6: tesseract
    7: pdfbeads
Select a plugin (or hit enter to finish): 2
  ✔ 1: scantailor
  ✔ 2: djvubind
    3: gui
    4: colorcorrect
    5: autorotate
    6: tesseract
    7: pdfbeads
Select a plugin (or hit enter to finish): 5
  ✔ 1: scantailor
  ✔ 2: djvubind
    3: gui
    4: colorcorrect
  ✔ 5: autorotate
    6: tesseract
    7: pdfbeads
Select a plugin (or hit enter to finish): 6
  ✔ 1: scantailor
  ✔ 2: djvubind
    3: gui
    4: colorcorrect
  ✔ 5: autorotate
  ✔ 6: tesseract
    7: pdfbeads
Select a plugin (or hit enter to finish): 7
  ✔ 1: scantailor
  ✔ 2: djvubind
    3: gui
    4: colorcorrect
  ✔ 5: autorotate
  ✔ 6: tesseract
  ✔ 7: pdfbeads
Select a plugin (or hit enter to finish): 
You are missing a dependency for one of your enabled plugins:
Could not find executable `exiftool` in $PATH. Please install the appropriate package(s)!

User avatar
jbaiter
Posts: 98
Joined: 17 Jun 2013, 16:42
E-book readers owned: 2
Number of books owned: 0
Country: Germany
Location: Munich, Germany
Contact:

Re: Introducing spreads: command-line workflow tool

Post by jbaiter » 25 Dec 2013, 18:23

'apt-get install exiftool' ;-)
See here: https://github.com/jbaiter/spreads#requirements
spreads: Command-line workflow assistant

spomwii
Posts: 69
Joined: 17 Jun 2011, 07:57

Re: Introducing spreads: command-line workflow tool

Post by spomwii » 25 Dec 2013, 18:32

Think I got exiftool installed now. But now this:

Code: Select all

(.spreads)tommy@tommy-scanner:~$ exiftool
Syntax:  exiftool [OPTIONS] FILE

Consult the exiftool documentation for a full list of options.
(.spreads)tommy@tommy-scanner:~$ spread configure
Please select a device driver from the following list:
  [1]: chdkcamera
  [2]: a2200
  [3]: dummy
Select a driver: 2
Selected "a2200" as device driver
Please select your desired plugins from the following list:
    1: scantailor
    2: djvubind
    3: gui
    4: colorcorrect
    5: autorotate
    6: tesseract
    7: pdfbeads
Select a plugin (or hit enter to finish): 1
  ✔ 1: scantailor
    2: djvubind
    3: gui
    4: colorcorrect
    5: autorotate
    6: tesseract
    7: pdfbeads
Select a plugin (or hit enter to finish): 2
  ✔ 1: scantailor
  ✔ 2: djvubind
    3: gui
    4: colorcorrect
    5: autorotate
    6: tesseract
    7: pdfbeads
Select a plugin (or hit enter to finish): 5
  ✔ 1: scantailor
  ✔ 2: djvubind
    3: gui
    4: colorcorrect
  ✔ 5: autorotate
    6: tesseract
    7: pdfbeads
Select a plugin (or hit enter to finish): 6
  ✔ 1: scantailor
  ✔ 2: djvubind
    3: gui
    4: colorcorrect
  ✔ 5: autorotate
  ✔ 6: tesseract
    7: pdfbeads
Select a plugin (or hit enter to finish): 7
  ✔ 1: scantailor
  ✔ 2: djvubind
    3: gui
    4: colorcorrect
  ✔ 5: autorotate
  ✔ 6: tesseract
  ✔ 7: pdfbeads
Select a plugin (or hit enter to finish): 
spreads encountered an error:
Traceback (most recent call last):
  File "/home/tommy/.spreads/bin/spread", line 39, in <module>
    cli.main()
  File "/home/tommy/.spreads/local/lib/python2.7/site-packages/spreads/cli.py", line 437, in main
    args.subcommand(config)
  File "/home/tommy/.spreads/local/lib/python2.7/site-packages/spreads/cli.py", line 155, in configure
    setup_plugin_config(config)
  File "/home/tommy/.spreads/local/lib/python2.7/site-packages/spreads/plugin.py", line 344, in setup_plugin_config
    pluginmanager = get_pluginmanager(config)
  File "/home/tommy/.spreads/local/lib/python2.7/site-packages/spreads/plugin.py", line 316, in get_pluginmanager
    name_order=True)
  File "/home/tommy/.spreads/local/lib/python2.7/site-packages/stevedore/named.py", line 43, in __init__
    invoke_kwds)
  File "/home/tommy/.spreads/local/lib/python2.7/site-packages/spreads/plugin.py", line 81, in _load_plugins
    invoke_kwds,
  File "/home/tommy/.spreads/local/lib/python2.7/site-packages/stevedore/named.py", line 95, in _load_one_plugin
    ep, invoke_on_load, invoke_args, invoke_kwds,
  File "/home/tommy/.spreads/local/lib/python2.7/site-packages/stevedore/extension.py", line 146, in _load_one_plugin
    plugin = ep.load()
  File "/home/tommy/.spreads/local/lib/python2.7/site-packages/distribute-0.6.24-py2.7.egg/pkg_resources.py", line 1989, in load
    entry = __import__(self.module_name, globals(),globals(), ['__name__'])
  File "/home/tommy/.spreads/local/lib/python2.7/site-packages/spreadsplug/__init__.py", line 31, in <module>
    __import__("spreadsplug.dev." + module[:-3], locals(), globals())
  File "/home/tommy/.spreads/local/lib/python2.7/site-packages/spreadsplug/dev/dummy.py", line 5, in <module>
    from pathlib import Path
ImportError: No module named pathlib

mera461
Posts: 7
Joined: 27 Dec 2013, 07:08
Number of books owned: 0
Country: Denmark

Re: Introducing spreads: command-line workflow tool

Post by mera461 » 27 Dec 2013, 21:45

I don't know if you are interesting in supporting windows? Being a developer I know that it is really difficult/annoying to support a platform you are not using yourself.

But I anyhow managed to install the different dependencies to at least get started, but got the same error with the exiftool, and I think it is because of the util.find_in_path function that doesn't work on windows (partly because of the .exe extension and partly because of the path separator). After a bit of search I found: http://stackoverflow.com/questions/3770 ... -in-python where it is suggested either to use the function distutils.spawn.find_executable (I seems only to work with .exe files, but that should be ok for at least the exiftool) or something like this code for a more general solution, if you want to include also .bat, .cmd, and other executable extensions:

Code: Select all

def which(program):
    def is_exe(fpath):
        return os.path.exists(fpath) and os.access(fpath, os.X_OK)

    def ext_candidates(fpath):
        yield fpath
        for ext in os.environ.get("PATHEXT", "").split(os.pathsep):
            yield fpath + ext

    fpath, fname = os.path.split(program)
    if fpath:
        if is_exe(program):
            return program
    else:
        for path in os.environ["PATH"].split(os.pathsep):
            exe_file = os.path.join(path, program)
            for candidate in ext_candidates(exe_file):
                if is_exe(candidate):
                    return candidate

    return None
Frank

User avatar
jbaiter
Posts: 98
Joined: 17 Jun 2013, 16:42
E-book readers owned: 2
Number of books owned: 0
Country: Germany
Location: Munich, Germany
Contact:

Re: Introducing spreads: command-line workflow tool

Post by jbaiter » 28 Dec 2013, 07:02

@spomwii:
That's a known problem, I unfortunately messed up the packaging for 0.4.1. Once I get back from 30c3, I will prepare a 0.4.2 that should resolve these issues.

@mera451:
My plan for windows-compatibility would actually be to ship a fully bundled version of the Python runtime, the required libraries, spreads+plugins, all external dependencies. That would mean that all external tools would be in a known location, making "util.find_in_path" a lot simpler on Windows systems.
I've been playing around with modifying a WinPython installation to include all of the dependencies and spreads itself, unfortunately my current kernel doesn't like VirtualBox very much and I get kernel panics all the time, which makes the work even more unpleasant ;-)
If you want to give it a shot, contact me and I'll help as much as I can!
spreads: Command-line workflow assistant

Post Reply