Note: it is easier to build from a tarball instead.
CCP4-JHBuild is JHBuild customized for building CCP4.
JHBuild is a Python program that can pull a number of modules from a variety of sources (Git, Subversion, Bazaar, tarballs...) and build them in the correct order. JHBuild was developed to build GNOME, but now is also used for a few other projects.
Modules (packages) are downloaded, patched if necessary, configured, built and installed, one by one.
For example, when one asks to build Refmac:
./cj build refmac
first its dependencies are downloaded and built (Lapack is downloaded from Netlib, CCP4 libraries from Bazaar) and finally Refmac is fetched and built. The user can skip compilation of Lapack if it is already installed.
New packages that use popular build systems like autotools, cmake or distutils can be easily added by specifying the download location and dependencies in the moduleset file.
CCP4-JHBuild is intended for developers, but can also be used by users who are familiar with compiling software and want to test new features.
What prerequisites you need depend on what you want to build. To build complete CCP4 suite you need all that is needed when building from tarball (read that first).
Additionally, JLigand requires Java SDK (alternatively, put JLigand.jar into checkout/jligand).
On Windows MinGW/MSYS is recommended, see the Windows section below.
If you need to use a proxy, check if your terminal has all the corresponding environment variables set.
Get "devtools" from bzr:
bzr checkout bzr+http://oisin.rc-harwell.ac.uk/bzr/devtools/trunk devtools cd devtools/
If you want to build stable (release) branch, which is used for official release and updates, use bzr+http://oisin.rc-harwell.ac.uk/bzr/series-65/devtools/ instead.
Check if jhbuild works (it prints prerequisites of clipper):
./cj list clipper
Or, if python 2.7 or 2.6 is not default on your system:
/path/to/python2.7 ./cj list clipper
It is possible to work in a different directory: just call cj using full path.
Before you start building CCP4 you may want to change some options in the conf.py configuration file. Some options can be also set through environmental variables. Read the conf.py file for details. In particular, if you have multiple compilers installed, choose which one should be used.
If you can't install cmake 3.1+ system-wide, build it locally:
./cj build cmake
The Tcl library and headers are needed for building ccp4mapwish and (optionally) diff-image. Although any Tcl version (8.4 - 8.6) would do, we use 8.4, because that is the only version iMosflm works with. Since iMosflm and ccp4i require several Tcl/Tk libraries, we've prepared a bundle with all of them. It can be built on all supported platforms, but not cross-compiled:
# on Unix Xlib headers must be installed (e.g. libX11-dev on Debian) curl -O http://devtools.fg.oisin.rc-harwell.ac.uk/files/tcltk++_8.4.19.3.tar.gz tar xzf tcltk++_8.4.19.3.tar.gz # (90MB unpacked) tcltk++_8.4.19.3/build.sh TclTk84 # the argument is dir for installation
If you have some of the popular libraries used by CCP4 (zlib, libxml2, libjpeg, LAPACK) already installed, add them to the skip list in conf.py to not waste time on building them from sources.
Then you may build selected module(s) with dependencies:
./cj build refmac
or default set of modules (packages that don't require Tcl, Qt, Java, X11):
./cj build
or everything:
./cj build ccp4-linux # on Linux, or ./cj build ccp4-osx # on OSX (both are almost the same), or ./cj build ccp4-windows # on Windows
The list command shows what packages would be built:
./cj list refmac ./cj list default ./cj list ccp4-osx
and the info command shows additional info:
./cj info refmac
The build command first checks for updates in the repository and then builds the program. You can separate the two steps:
./cj update [program] ./cj build -n [program] # -n == --no-network
Other useful build options are -f (forces build even if the module has not been updated) and -c (runs make clean before make).
There are also equivalents of make clean and make uninstall:
./cj clean [program] ./cj uninstall [program]
To build a module without dependencies, replace build with buildone. The same with update/updateone and clean/cleanone.
The build command creates three directories (checkout, builddir and install) in the current directory. They are used for, respectively, downloads, building and installing. These locations can be changed in conf.py (variables checkoutroot, buildroot and prefix).
Other settings in conf.py include compiler and linker flags, number of parallel jobs (jobs=N) and options for individual programs (like Tcl version or OpenMP support). These settings can be changed either in conf.py or in $HOME/.cjrc.
On OSX dylib paths are absolute after the build. Make them relative:
./cj run misc/osx-postinst.sh
Similarly on Linux, RPATH in some binaries needs to be changed (if you don't have patchelf installed do ./build patchelf first):
./cj run misc/linux-rpath.sh
All JHBuild commands (some of them are not relevant when building CCP4) can be listed with:
./cj help
Option --help shows details about each command:
./cj build --help
Make sure that older CCP4 is not used ($CCP4, etc. are not set). Run the BINARY.setup script (it's part of the ccp4-core module):
./install/BINARY.setup
Source one of the two setup files (they are created by BINARY.setup):
. install/bin/ccp4.setup-sh # in bash, dash, zsh source install/bin/ccp4.setup-csh # in csh, tcsh
and the just compiled suite (or selected programs) are ready to use.
Update devtools, then update and compile modules:
bzr update ./cj build
By default, the repositories are branched using anonymous access to bazaar. bzr commit is a local operation. To push your commits, specify location:
bzr push --remember bzr+ssh://USER@fg.oisin.rc-harwell.ac.uk/scmrepos/bzr/PROJECT/trunk/
A few projects use so-called lightweight checkout by default (because of a problem with large files in the current configuration of bzr). In this case bzr push will not work.
Alternatively, add in your local config:
repos['oisin'] = 'bzr+ssh://USER@fg.oisin.rc-harwell.ac.uk/scmrepos/bzr/'
To use branches other than trunk for some projects, specify corresponding URLs in local config (and remove checkout/PROJECT if it's already there):
branches['mmdb'] = 'bzr+ssh://fg.oisin.rc-harwell.ac.uk/scmrepos/bzr/mmdb/dev/'
It is possible to compile the suite on Windows or to cross-compile it for Windows on other system. In both cases we use MinGW (port of GCC).
C library used by MinGW has no regex. Regex is needed for the ccif library. mingw-libgnurx is a possible replacement.
Before compiling cctbx python extensions with MinGW make sure you have so-called import library for pythonNN.dll. Programs that use cctbx from C++ (like pointless) don't need Python extensions. To skip building the extensions (and reduce compilation time) uncomment --build-boost-python-extensions=False in conf.py.
Bash or similar shell with basic Unix tools is needed to run autoconf-generated configure scripts. Below we describe three setups.
One way to install MinGW with MSYS is using mingw-get-inst.
Install necessary packages:
mingw-get install mingw32-gcc mingw32-gcc-fortran mingw32-gcc-g++ \
msys-make msys-wget mingw32-libz
mingw-libgnurx needs to be installed manually. After unpacking .h file goes to include under C:\MinGW, .a to lib and .dll to bin.
If Qt4.8 is compiled from sources, adding some -no options makes compilation faster. For example:
./configure.exe -release -opensource -no-3dnow -no-sse2 \ -system-zlib -system-libpng -system-libjpeg \ -no-sql-sqlite -no-qt3support -no-libmng -no-libtiff \ -no-openssl -no-dbus -no-phonon -no-phonon-backend -no-multimedia \ -no-audio-backend -webkit -no-declarative -no-declarative-debug \ -no-script -no-scripttools -no-style-motif -no-style-cde \ -no-s60 -nomake examples -nomake demos -nomake tools
Outside of MSYS install CMake, Python 2.7 and Bazaar (Python-based installer will do, but for 64-bit Python see recent comments here). Make sure all three are in PATH, e.g. add to ~/.profile:
PATH=$PATH:/c/Python27:/c/Python27/Scripts:"/c/Program Files (x86)/CMake/bin" export PATH
Note
MSYS shell allows to specify paths in different ways. For example /home, /c/MinGW/msys/1.0/home, C:/MinGW/msys/1.0/home and C:\MinGW\msys\1.0\home point to the same directory. MSYS is converting paths that are passed between MSYS and non-MSYS programs, using heuristics to decide which strings are paths. Since paths are passed a lot between shell (MSYS), Python (non-MSYS), CMake (non-MSYS), make (MSYS), compilers (non-MSYS) and other programs during the build, the whole thing is fragile.
In default MinGW installation user's home directory is in C:\MinGW\msys\1.0\home\username. In our setup all directories used by the build system are in our home directory. Other configurations have not been tested much. It was spotted that make/libtool fail if checkoutroot is outside of the MinGW directory (C:/MinGW by default).
If wget has problems with https certificate, you may add check_certificate = off to ~/wgetrc.
When environment variables go through Python they are converted to uppercase, and wget ignores uppercase *_PROXY variables, so if you are behind proxy you need to additionally configure it in ~/.wgetrc.
MinGW doesn't come with pkg-config, but much better replacement (pkgconf) can be easily built:
./cj build pkgconf
Thanks to automatic path conversion in MSYS, it is even possible to run unix tests.
Cygwin provides more tools and is more convenient for debugging. Cctbx does not compile under cygwin (I don't have time to work on it). All other modules can be compiled.
Native Windows programs can be built under Cygwin using MinGW (cross-)compiler, in a similar way as on Linux (see the next section):
TARGET=i686-w64-mingw32 ./cj build clipper-progs
Most of Linux distribution have MinGW already packaged (note that g++ and gfortran are also needed).
To easily cross-compile Tcl extensions, download Tcl/Tk 8.4 compiled on Windows (the same that is distribued with CCP4).
To build ccif, install mingw-libgnurx library (on Fedora and OpenSuse in mingw32-libgnurx* RPM).
Either install mingw32-pkg-config or build module pkgconf.
Use the same commands as for native compilation, but set environmental variable $TARGET to the prefix of cross-compiler, e.g.:
TARGET=i686-pc-mingw32 ./cj build
By default, directories for building and installing are build-$TARGET and install-$TARGET.
missing bits
Cross-compilation of cctbx python extensions is not supported, so we automatically set --build-boost-python-extensions=False flag when cross-compiling (see conf.py). Fortunately, these extensions are not needed to build pointless, aimless and phaser.
Unless you have configured wine and binfmt_misc, ccif build script will not generate automatically cif_mmdic.lib. It can be generated later on Windows:
cifdic_to_symtab %CCP4%\share\ccif\cif_mm.dic %CLIB%\ccp4\cif_mmdic.lib 199
If multiple compilers are installed, autoconf and cmake may have different preferences. Mixing compilers may cause errors. In such case choose compilers explicitly in the compilers section in conf.py. (Example: gfortran has ver. 4.7, f95 is gfortran-4.2. LAPACK uses the latter, other programs the former and the error says ilaenv_': undefined reference to `_gfortran_copy_string').
If phaser compilation fails (e.g. when compiling src/ListEditing.cc) with internal compiler error, either use newer compiler or comment out the --enable-openmp-if-possible=True option.
If cmake gives a warning about libxml2.so.2 version, probably you have built libxml2 (rapper dependency) and cmake is indirectly (via libarchive) linked to another libxml2 version. While this warning should not cause any harm, it can be avoided by using libxml2 from the system instead of building it. Set the skip option in conf.py and do ./cj uninstall libxml2.
Each change in a library triggers make install, which reinstalls also not modified headers, changing their timestamps and triggering not necessary recompilations. To avoid it, jhbuild has a small wrapper around the install program. This wrapper, called install-check, uses cmp to compare headers before calling install. It works only on Unix. To enable it, compile the wrapper in devtools:
cc -O2 install-check.c -o install-check