Set Up an Ubuntu VM for Standard Ebooks Tools on a Mac
Update: This Guide is Deprecated Because the Tools Now Support the Mac Natively
(Deprecated -- just use the tools directly on your Mac now!)
This guide will walk you through the exact steps to get all dependencies set up for Standard Ebooks (SE) book production work on a Mac.
This guide assumes your Mac is around 8 years old or newer, so that it is 64-bit. If you are running anything newer than Snow Leopard then that is probably the case. (This was tested on MacOS 10.12 Sierra.)
Convenient, cross-platform development
With Vagrant's shared folder (synced folder) feature, the files are hosted in a normal folder on your Mac, and mounted to /vagrant/ inside the VM. You can use all of your Mac tools (iBooks for viewing .epub files, your favorite text editor(s), visual git tools such as Xcode's FileMerge or Atlasian SourceTree, etc.) to work on the files (and manipulate the Git repo) and still run the SE command line tools (build-images, build, typogrify, etc.) from inside the Ubuntu VM. It is kind of magical.
First, install VirtualBox and Vagrant
- Download and install the Mac version of VirtualBox (look for the OS X hosts link)
- Download and install Vagrant
Vagrant Ubuntu VMs
For this guide, we will be using Xenial Xerus, or Ubuntu 16.04 LTS.
For more info:
For more Vagrant Ubuntu VMs:
According to Stack Overflow the VM downloaded for us by Vagrant will be hidden in your user folder here:
~/.vagrant.d/boxes
Open Terminal and decide where to put your working directory and Vagrantfile on your Mac
For this guide we will assume you are putting it in ~/Documents/setools/
In a Terminal window (do not type the $ belowl; it is just an indicator of what commands are typed into the shell.
$ cd ~/Documents
At this point you may see a prompt like this:
my-mac:Documents my-username$
but for simplicity I will just write $
Now make the folder and cd into it
$ mkdir setools && cd setools
Have Vagrant install the VM for us
$ vagrant init ubuntu/xenial64
This created a Vagrantfile for us.
IMPORTANT: increase the VM's RAM to 1024 MiB so GCC does not crash!
Before we proceed, we edit the Vagrantfile to get the VM to have enough RAM for purposes. Find the section like the following, and uncomment a few lines so it looks like this:
config.vm.provider "virtualbox" do |vb| # Display the VirtualBox GUI when booting the machine #vb.gui = true # Customize the amount of memory on the VM: vb.memory = "1024" end
Ready to download and boot
Now we wait for it to download and install (bandwidth usage of about 450 MiB? + another 1,500 MiB after dependencies install -- but on-disk is smaller when we are done, around 290 MiB) and boot the VM (still on the Mac side of the Terminal, in the ~/setools/ folder):
$ vagrant up --provider virtualbox
This Mac folder (~/Documents/setools) and the Vagrant VM share a folder which stays there on the Mac side when the VM is turned off. It is mounted in the Ubuntu VM's filesystem at /vagrant/
Note that the Vagrant VM / Virtualbox .box file lives in ~/.vagrant.d/boxes and not in this mostly empty folder, with just the Vagrantfile
Time passes...
Let Vagrant do its download business.
Ready to log in finally
Inside Terminal, from our (Mac) working folder (where the Vagrantfile lives), we can log in with:
$ vagrant ssh
Ubuntu prints this friendly welcome screen each time we login:
Welcome to Ubuntu 16.04.2 LTS (GNU/Linux 4.4.0-81-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
Get cloud support with Ubuntu Advantage Cloud Guest:
http://www.ubuntu.com/business/services/cloud
0 packages can be updated.
0 updates are security updates.
ubuntu@ubuntu-xenial:~$
The user is vagrant and vagrant ssh will log us in without having to type a password.
Congratulations, you are over the hump.
Inside the Ubuntu VM
Commands shown below are assumed to be entered into the VM's command line, once you are logged in (after vagrant ssh. You may see something like:
ubuntu@ubuntu-xenial:~$
But again I will just write $ for the prompt, where you type the stuff besides the $
Verify RAM settings
$ top
Top is like Activity Monitor on the Mac. The fourth line should say something like
KiB Mem : 1015968 total, 640844 free, 51932 used, 323192 buff/cache
As long as we have >= 1,000,000 KiB (>= 1 GiB) we should be fine.
(press q to quit)
Get all the dependencies, in the right order.
First we make sure our package sources are up to date
$ sudo apt update
And wait a little bit.
Now we get Debian package dependencies, using APT
$ sudo apt install python3-pip xsltproc libxml2-utils xmlstarlet libxml-xpath-perl recode html-xml-utils librsvg2-bin libimage-exiftool-perl zip epubcheck calibre default-jre
It will print a bunch of package names, then at the bottom:
After this operation, 1,044 MB of additional disk space will be used.
Press y to coninue and then wait for a while as the packages are downloaded and installed for you.
More dependencies
To help with the Python image library stuff (pacakge Pillow) for Ubuntu 16 (see https://pillow.readthedocs.io/en/latest/installation.html#building-on-linux.
$ sudo apt install libtiff5-dev libjpeg8-dev zlib1g-dev libfreetype6-dev liblcms2-dev libwebp-dev tcl8.6-dev tk8.6-dev python-tk
(May have to press y to continue)
In order to get Python lxml package to install, and gitdb, which needs realpath (and to use build-images) we need a few more packages (and their dependencies):
$ sudo apt install libxml2-dev libxslt-dev realpath inkscape
Panoply of Python Packages
Now we install the SETools' Python dependencies, which should have their dependencies satisfied by the above package installs.
$ sudo pip3 install pyhyphen roman titlecase beautifulsoup4 smartypants pillow gitpython cssselect regex lxml gitdb termcolor terminaltables
Install the fonts for the book covers / title pages
mkdir -p ~/.fonts/ wget -O ~/.fonts/LeagueSpartan-Bold.otf "https://github.com/theleagueof/league-spartan/blob/master/LeagueSpartan-Bold.otf?raw=true" wget -O ~/.fonts/OFLGoudyStM.otf "https://github.com/theleagueof/sorts-mill-goudy/blob/master/OFLGoudyStM.otf?raw=true" wget -O ~/.fonts/OFLGoudyStM-Italic.otf "https://github.com/theleagueof/sorts-mill-goudy/blob/master/OFLGoudyStM-Italic.otf?raw=true" sudo fc-cache -fv
Continuing with Step 1 of the SE Producing an Ebook guide
Install hyphenation dictionaries for the pyhyphen library:
$ sudo python3 -c "exec(\"from hyphen import dictools\\ndictools.install('en_GB')\\ndictools.install('en_US')\")"
And of course: install Git for the Ubuntu command line
$ sudo apt install git
Now for the 'easy' part of step 1 of the guide!
$ cd ~/ $ git clone https://github.com/standardebooks/tools.git
We make a working folder that is shared between the Mac and the Ubuntu side:
$ cd /vagrant/ $ mkdir books $ cd books
Test an existing book
Let's test that the tools work on an existing repo:
$ cd /vagrant/books/ $ git clone https://github.com/standardebooks/henry-david-thoreau_walden.git $ cd henry-david-thoreau_walden/ $ ~/tools/build-images .
You can open the vectorized SVG files on your Mac, in the Finder.
Navigate to ~/Documents/setools/books/henry-david-thoreau_walden/images and press the space bar to use MacOS's built-in Quick Look preview ability to see the SVG file, complete with backtground art and vector text.
Cool tip: Install the same fonts (below) on your Mac
Then you can use Finder's space bar quick look feature to preview .SVG source files (not the ones generated by the build-images script) and you can see how the book cover will look, before you even run the build-images script.
Test the epub build process
Continuing in the Thoreau / Walden folder:
$ ~/tools/build --output-dir="dist" --kindle --check .
The End
On your Mac, use the Finder to navigate to the Documents/setools/books folder. Drill down to the books/whoever-whatever/dist folder and double-click the .epub file to open it in iBooks.
Congratulations: you have built an .epub file (~1 MiB) with the Standard Ebooks tools (290 MiB VM on disk).
Now you can make your own skeleton(s) for your own book(s):
$ cd /vagrant/books/ $ ~/tools/create-draft --author="John Doe" --title="Anonymous Cowards"
Have fun with steps 2–21!
Next steps:
You are done with step 1 of the Producing an Ebook Step by Step guide.
(Don't worry: it is a lot more fun and more interesting from here on out. And hopefully this tutorial has been explicit enough that you will not have had to do any distracting, undocumented Side Quests!)
Small tip
If you are proofreading an ebook and making changes to it, then syncing and pulling in the file from Dropbox or iCloud Drive, and if your iPad iBooks does not show the updated version, try this:
- Delete the old version from iBooks on your iPad.
- Before you sync it in your Dropbox, or iCloud Drive, rename it with an increasing version number: h-d-t-walden-42.epub, then after another change, h-d-t-walden-43.epub.
- Now grab the newest file, h-d-t-walden-43.epub from Dropbox (for example) and Export > Open in ... > Send to iBooks
- When you tap to open, it should have the up-to-date draft you are working on!
(Obviously you don't have to release a weirdly-numbered .epub, this is just a hack to get around iPad/iBooks inability to use the newer file, each time you make changes. iBooks on the Mac seems well behaved if we just delete and then double click the .epub in the Finder to re-import.)
Optional, Useful Tidbits
Vagrant Tips and Tricks
If you change your Vagrantfile (use less RAM -- untested) and want to restart the VM:
(inside of the Ubuntu VM, opposite of vagrant ssh)
$ exit
In Mac Terminal, in your ~/Documents/setools/ folder
$ vagrant reload
Once it finishes booting:
$ vagrant ssh
To shutdown:
In the Terminal, in the (Mac side) folder where your Vagrant file lives (e.g. ~/Documents/setools)
$ vagrant suspend
To boot it from suspended state:
In the Terminal, in the (Mac side) folder where your Vagrant file lives (e.g. ~/Documents/setools)
$ vagrant up
And to log back in:
$ vagrant ssh
While you are editing Ebooks, you may want to know...
How to type real, typographic, “smart,” curved quotes on your Mac
| Option + [ | “ (double left quote) |
| Shift + Option + [ | ” (double right quote) |
| Option + ] | ‘ (single left quote) |
| Shift + Option + ] | ’ (single right quote, and real typographic curved apostrophe, too) |
Optionally: to install emacs (command-line no-X version) if you do not like the nano text editor:
$ sudo apt install emacs24-nox
but you can just use whatever you have on your Mac, in the Terminal (or any App / Finder, etc.) on the ~/Documents/setools side
Using command-line git to see word-by-word or character-by-character diffs:
Works on the Mac or Linux side:
$ git diff --word-diff
Assuming you have Xcode installed, run from the Mac side, in Terminal:
$ git difftool -t opendiff -y
or
$ git difftool -t opendiff
Go into Preferences and make sure Wrap Text is checked. Also, you may need to resize the window to have the wrap text feature work properly.