Debian Post-Install Script Work Thread

General discussion about software packages and releases, new software you've found, and threads by programmers and script writers.

Moderator: peterZ

Post Reply
abmartin
Posts: 79
Joined: 15 Sep 2010, 15:33
Number of books owned: 2000
Country: USA
Location: Ohio

Debian Post-Install Script Work Thread

Post by abmartin »

Since there seem to be a number of folks heading out onto the Linux side of the tracks, I want to do something to help out. I think a post-install script that can be run to install necessary packages and dependencies, download appropriate source code, compile, and install a standard set of applications could be very helpful. I'm going to try to hack out one for Debian and, hopefully, the derivatives like Ubuntu and Mint. (But they do things some things differently, so no guarantees, but I expect this to be easily modifiable, since I'll put in noob-appropriate documentation)

As I see it, the basic needs are:
Spreads
Scantailor-enhanced
Djvubind
Tesseract
Ppmunwarp
and some general system packages like minidjvu, etc...

If anyone has other ideas about what software you would like to have installed, please add some info here.

I'm going to start making a list of packages that we would need in a post install script.
User avatar
scann
Posts: 79
Joined: 31 Jul 2011, 01:23
Number of books owned: 0
Country: Argentina

Re: Debian Post-Install Script Work Thread

Post by scann »

Actually, what we really need is a live-cd that you can put on a USB stick or on a CD, without having Linux installed in any machine, just to use it when you want to scan a book.
abmartin
Posts: 79
Joined: 15 Sep 2010, 15:33
Number of books owned: 2000
Country: USA
Location: Ohio

Re: Debian Post-Install Script Work Thread

Post by abmartin »

I agree that this is something that will be great to have in the future. It's certainly beyond my abilities to manage. Unfortunately, it would be pretty complicated, since we'd probably need 32 bit, 64 bit, and RaspberryPi versions too. I'd be concerned about how we did persistent storage of scanned files too, unless it were installed on a very large usb stick. (It wouldn't be as fast as a native install though. Using a CD would be painful) I also really wouldn't want to deal with the extra tech support with the core os that would likely arise.

It's a shame that windows messes with cameras too much to just do a virtual machine install...
abmartin
Posts: 79
Joined: 15 Sep 2010, 15:33
Number of books owned: 2000
Country: USA
Location: Ohio

Re: Debian Post-Install Script Work Thread

Post by abmartin »

Last weekend, I tried out a few live images to see how viable using a USB stick might be. To make a long story short, I found that unless it is a very fast flash drive, it's a horrible experience. Using a quick drive is definitely okay. (Still not as good as a full install) There are sometimes reasons why some flash drives are 8 bucks and others are 45 other than branding and packaging. (Sandisk has a model that they claim can write at 200 MB a second, with USB 3, and if that upper-case B isn't a mistake, that's pretty impressive. My brain is stuck in the USB 2 paradigm with a max of 480Mb)

Unfortunately, making a live usb stick with perstistence for debian takes a little work. (Not a huge amount, but since it involves creating partitions, it may be a bit too intimidating) However, using unetbootin, Ubuntu and derivatives like Mint are very easy to make with persistence. (And is a cross-platform application) I'm going to work on a Mint script some tonight and tomorrow. My hope is that very little needs to be changed to make it work on Debian.

Is a guide needed to install a live distro with persistence using Unetbootin necessary? Or is the guide on their homepage good enough? (I'm not really sure what I would write differently) Basically, just follow the directions, and just max out the space for pesistent storage.
http://unetbootin.sourceforge.net/
abmartin
Posts: 79
Joined: 15 Sep 2010, 15:33
Number of books owned: 2000
Country: USA
Location: Ohio

Re: Debian Post-Install Script Work Thread

Post by abmartin »

I've got a start. I'm currently testing it on my old netbook, which is definitely on the slow slide of things when running off an old usb stick. I'm testing it with Mint 15. (It really makes me miss my Openbox Crunchbang when system resources are this tight) It's still early on in the process and is compiling scantailor.

At the moment, the sequence is:
1. Update Repos
2. Run a full dist-upgrade to get the latest packages (Maybe more of a Debian thing than Ubuntu, if running an LTS Ubuntu, it should probably be changed to just upgrade)
3. Install Scantailor build dependencies (quite a lot of them)
4. Download source for Scantailor-enhanced
5. Untar the source
6. cmake, make, make install Scantailor-enhanced
7. Install Djvubind dependencies
8. Download djvubind repository
9. Compile and install Djvubind
10. Install dependencies for Pdfbeads
11. Install Pdfbeads
12. Install dependencies for Spreads
13. Install Spreads
14. Downloads ppmunwarp sources
15, Compile and install ppmunwarp
16. Then install a series of optional packages that I find useful

I thought about adding the gui option to the Spreads plugin via the script, but decided it's not a good idea, since at some point, that process might change, so I just throw a line of text out there about it.

The optional packages are currently commented out and can be chosen by the user if so desired. I currently have djvusmooth, pdftk, djview4, ocrodjvu, unpaper, and cuneiform as options. Any other suggestions for useful packages?

I'll post what I have done whenever it's done testing, which might be about an hour from now. (And then I'll have to read the log file, which will almost certainly be 5 or 6 thousand lines at least, since apt-get and make are not exactly silent in the stdout. (I was tempted to put a "> /dev/null" on the end of each apt-get line, which makes it silent and would still show errors)


I hope that this will be useful. Even if you don't want to run it as a script, this can be used as just a list of commands to install the various applications.

Source files are stored in a directory called software in the home folder.


Loving Mint for Windows folks. It really is very intuitive for those familiar with that paradigm. I haven't seriously sat down with it since they forked their own desktops. It really could be the next Windows XP in terms of its mass adoption and acceptance. Too bad it won't be.
abmartin
Posts: 79
Joined: 15 Sep 2010, 15:33
Number of books owned: 2000
Country: USA
Location: Ohio

Re: Debian Post-Install Script Work Thread

Post by abmartin »

Okay, it seems to install things properly. I've only tested it in Mint, but the package names would also work in Debian. (And, I assume that Ubuntu in the middle would be fine too, but that's just speaking out of the proverbial) On a normal system with a good internet connection, it won't take very long.

Another option would be for me to install all of the dependecies in the initial step. I've decided to keep it that way for those who want to just install one thing.

When I get a chance, I may write a brief tutorial about how to put your /tmp in a ramdisk which will speed things up and preserve the life of a usb drive. (esp. for djvubind, which is designed for that) Also, I can explain briefly about how to mount a system drive into your filesystem so you can put the book project on a system harddisk in the event you choose to boot off of a stick.

A slightly better option than the persistent live stick model is to actually install the distro onto the stick itself just as you would a hard disk. When doing that, you'll have to manually deal with the partitioner so you can put the bootloader on the stick and leave your harddisk intact. (Not hard, just format the stick to ext4 and put the root partition (/) on it, and manually put the bootloader on that device)

And finally, a strong suggestion to install onto a local disk...nothing is going to beat the speed of a modern SSD. (And a normal HD is going to be faster than all but the best usb3 sticks)

Code: Select all

#!/bin/bash
(

echo -e "\nPreparing to install tools for bookscanning..."

echo -e "\nUpdating repositories..."
sudo apt-get update

echo -e "\nUpdating system..."
sudo apt-get dist-upgrade


##############
##Scantailor##
##############

#Installing Dependencies
echo -e "\nInstalling dependecies for Scantailor"
sudo apt-get install build-essential cmake libqt4-dev libjpeg-dev zlib1g-dev libpng-dev libtiff4-dev libboost-all-dev libxrender-dev

#Downloading Source
echo -e "\nDownloading Scantailor-enhanced source"
mkdir ~/software
cd ~/software

#Untarring Source
echo -e "\nUntarring source"
wget "http://downloads.sourceforge.net/project/scantailor/scantailor-devel/enhanced/scantailor-enhanced-20120914-pre.tar.bz2?r=http%3A%2F%2Fsourceforge.net%2Fprojects%2Fscantailor%2Ffiles%2Fscantailor-devel%2Fenhanced%2F&ts=1380929005&use_mirror=softlayer-dal" -O "scantailor-enhanced.tar.bz2"
tar xvjf "scantailor-enhanced.tar.bz2"

#Compiling Source
cd scantailor-enhanced
cmake .
make
sudo make install


############
##Djvubind##
############

#Installing Dependencies
echo -e "\nInstalling dependencies for Djvubind"
sudo apt-get install python3 djvulibre-bin imagemagick tesseract-ocr minidjvu mercurial

#Downloading Source
cd ~/software
echo -e "\nDownloading Djvubind source"
hg clone https://code.google.com/p/djvubind/

#Installing
cd djvubind
sudo python3 ./setup.py install


############
##Pdfbeads##
############

#Installing Dependencies
echo -e "\nInstalling dependencies for Pdfbeads"
sudo apt-get install ruby ruby-rmagick ruby-hpricot 

#Installing 
echo -e "\nInstalling Pdfbeads"
sudo gem install pdfbeads


###########
##Spreads##
###########

#Installing Dependencies
echo -e "\nInstalling Dependencies for Spreads"
sudo apt-get install libusb-dev python-pip python-pyside

#Installing Spreads
sudo pip install spreads

#Adding entry for gui in the spreads config file
echo -e "\nTo use the Spreads gui, don't forget to add gui as a plugin in your ~/.config/spreads/config.yaml"


#############
##ppmunwarp##
#############

#Downloading Source
echo -e "\nDownloading ppmunwarp"
cd ~/software
wget "http://diybookscanner.org/forum/download/file.php?id=4400" -O "ppm_rose_tools_v1.6.zip"
unzip ppm_rose_tools_v1.6.zip

#Compiling and Installing
echo -e "\nCompiling and installing ppmunwarp"
cd ppm_rose_tools_v1.6
mkdir ~/bin
make
cp ppmunwarp ~/bin
cp ppmwhitebalance ~/bin
cp ppmrectlist ~/bin


######################
##Other Useful Tools##
######################

##Uncomment those you want

##Djvusmooth -- Gui Editor for Djvu files: useful for metadata and TOC
#sudo apt-get install djvusmooth

##PdfTk -- Swiss army knife for splitting, merging, and a number of other tasks for PDF files
#sudo apt-get install pdftk

##Djview4 -- Simple, light DjVu viewer
#sudo apt-get install djview4

##Ocrodjvu -- Uses ocropus and tesseract to perform OCR on existing djvu files
#sudo apt-get install ocrodjvu

##Unpaper -- Simple post-processing tool for scanned images
#sudo apt-get install unpaper

##Cuneiform -- OCR system
#sudo apt-get install cuneiform


exit 0
) 2>&1 | tee postinstall.log



abmartin
Posts: 79
Joined: 15 Sep 2010, 15:33
Number of books owned: 2000
Country: USA
Location: Ohio

Re: Debian Post-Install Script Work Thread

Post by abmartin »

Fixed a stupid mistake. I put the binaries for ppmunwarp in a directory that mint doesn't use for binaries. I've changed it to just put it in /usr/bin.

Code: Select all

    #!/bin/bash
    (

    echo -e "\nPreparing to install tools for bookscanning..."

    echo -e "\nUpdating repositories..."
    sudo apt-get update

    echo -e "\nUpdating system..."
    sudo apt-get dist-upgrade


    ##############
    ##Scantailor##
    ##############

    #Installing Dependencies
    echo -e "\nInstalling dependecies for Scantailor"
    sudo apt-get install build-essential cmake libqt4-dev libjpeg-dev zlib1g-dev libpng-dev libtiff4-dev libboost-all-dev libxrender-dev

    #Downloading Source
    echo -e "\nDownloading Scantailor-enhanced source"
    mkdir ~/software
    cd ~/software

    #Untarring Source
    echo -e "\nUntarring source"
    wget "http://downloads.sourceforge.net/project/scantailor/scantailor-devel/enhanced/scantailor-enhanced-20120914-pre.tar.bz2?r=http%3A%2F%2Fsourceforge.net%2Fprojects%2Fscantailor%2Ffiles%2Fscantailor-devel%2Fenhanced%2F&ts=1380929005&use_mirror=softlayer-dal" -O "scantailor-enhanced.tar.bz2"
    tar xvjf "scantailor-enhanced.tar.bz2"

    #Compiling Source
    cd scantailor-enhanced
    cmake .
    make
    sudo make install


    ############
    ##Djvubind##
    ############

    #Installing Dependencies
    echo -e "\nInstalling dependencies for Djvubind"
    sudo apt-get install python3 djvulibre-bin imagemagick tesseract-ocr minidjvu mercurial

    #Downloading Source
    cd ~/software
    echo -e "\nDownloading Djvubind source"
    hg clone https://code.google.com/p/djvubind/

    #Installing
    cd djvubind
    sudo python3 ./setup.py install


    ############
    ##Pdfbeads##
    ############

    #Installing Dependencies
    echo -e "\nInstalling dependencies for Pdfbeads"
    sudo apt-get install ruby ruby-rmagick ruby-hpricot

    #Installing
    echo -e "\nInstalling Pdfbeads"
    sudo gem install pdfbeads


    ###########
    ##Spreads##
    ###########

    #Installing Dependencies
    echo -e "\nInstalling Dependencies for Spreads"
    sudo apt-get install libusb-dev python-pip python-pyside

    #Installing Spreads
    sudo pip install spreads

    #Adding entry for gui in the spreads config file
    echo -e "\nTo use the Spreads gui, don't forget to add gui as a plugin in your ~/.config/spreads/config.yaml"


    #############
    ##ppmunwarp##
    #############

    #Downloading Source
    echo -e "\nDownloading ppmunwarp"
    cd ~/software
    wget "http://diybookscanner.org/forum/download/file.php?id=4400" -O "ppm_rose_tools_v1.6.zip"
    unzip ppm_rose_tools_v1.6.zip

    #Compiling and Installing
    echo -e "\nCompiling and installing ppmunwarp"
    cd ppm_rose_tools_v1.6
    mkdir ~/bin
    make
    sudo cp ppmunwarp /usr/bin
    sudo cp ppmwhitebalance /usr/bin
    sudo cp ppmrectlist /usr/bin


    ######################
    ##Other Useful Tools##
    ######################

    ##Uncomment those you want

    ##Djvusmooth -- Gui Editor for Djvu files: useful for metadata and TOC
    #sudo apt-get install djvusmooth

    ##PdfTk -- Swiss army knife for splitting, merging, and a number of other tasks for PDF files
    #sudo apt-get install pdftk

    ##Djview4 -- Simple, light DjVu viewer
    #sudo apt-get install djview4

    ##Ocrodjvu -- Uses ocropus and tesseract to perform OCR on existing djvu files
    #sudo apt-get install ocrodjvu

    ##Unpaper -- Simple post-processing tool for scanned images
    #sudo apt-get install unpaper

    ##Cuneiform -- OCR system
    #sudo apt-get install cuneiform


    exit 0
    ) 2>&1 | tee postinstall.log




Post Reply