How to install Hedge from Git

You're looking at the generic instructions for installing hedge. You may be able to save a fair bit of work by using these distribution- or machine-specific instructions:

This tutorial will walk you through the process of building hedge. To follow, you need:

• A UNIX-like machine with web access.

• A C++ compiler, a version 4.x gcc is preferred. (Debian/Ubuntu package g++. The installation has been tested with gcc 4.3 and newer.)

• A working Python installation, Version 2.4 or newer, with development headers (Debian/Ubuntu package python-dev or newer. Python 3 is only supported by some packages for now, but definitely in the cards for the future.)

• A version of Numpy. (Debian/Ubuntu package python-numpy)

• The git Version control system. (Debian/Ubuntu package git)

Hedge can take advantage of some extra packages:

• BLAS (will result in some speedup at high order, see below)

• libsilo (write Silo visualization files, see below)

• MPI (compute in parallel), via mpi4py.

• CUDA (compute on graphics cards) using PyCuda.

To build, follow the instructions below: (Note that bold face indicates the directory for each command.)

1. Install Boost C++ (version 1.35 or newer)

2. git clone http://git.tiker.net/trees/repotool.git src

3. cd src

4. src ./repotool clone http://git.tiker.net/trees/ .git

5. cd ..

6. Place this text in $HOME/.aksetup-defaults.py:

   BOOST_INC_DIR = ['${HOME}/pool/include/boost-1_38/']
   BOOST_LIB_DIR = ['${HOME}/pool/lib']
   BOOST_BINDINGS_INC_DIR = ['${HOME}/pool/include/boost-numeric-bindings/']
   BOOST_PYTHON_LIBNAME=["boost_python"]
   BOOST_THREAD_LIBNAME=["boost_thread"]
   USE_SHIPPED_BOOST = False
   # WITH_SPARSE_WRAPPERS = True

   Note that the appropriate paths and version of gcc on your machine may be different. In .aksetup-defaults.py you can also tell hedge about additional packages such as MPI. Continuing with our installation...

7. git clone http://git.tiker.net/trees/boost-numeric-bindings.git

8. boost-numeric-bindings ./configure --prefix=$HOME/pool

9. boost-numeric-bindings make install

10. export LD_LIBRARY_PATH=$HOME/pool/lib:$LD_LIBRARY_PATH (spelled DYLD_LIBRARY_PATH on the Mac)

11. Create excluded-subprojects in src and insert

    pycuda
    pyublasext
    pyopencl

12. On Snow Leopard Macs, do export ARCH_FLAGS="-arch x86_64".

13. src ./repotool start-over

Using hedge

To actually use the Virtualenv that hedge was built in, do the following

1. src source env/bin/activate

(As above, the bold-face 'src' means that's the directory you have to be in.)

If you haven't already added the LD_LIBRARY_PATH thing above to your bash init script, you might need to redo that step as well.

Compiling with BLAS and LAPACK

This is optional.

Make sure you have a usable installation of BLAS and LAPACK. Use of ATLAS for accelerated BLAS operation is highly recommended. Then add adaptations of the following lines to your $HOME/.aksetup-defaults.py:

HAVE_BLAS = True
HAVE_LAPACK = True
BLAS_LIBNAME = ['f77blas','atlas','gfortran']
LAPACK_LIBNAME = ['lapack']

# if necessary:
# BLAS_LIB_DIR = ['/some/where']
# LAPACK_LIB_DIR = ['/some/where']

Note that these modifications should be performed before the final install step. (Otherwise reubuild.) You are using Ubuntu? To install BLAS and LAPACK try:

sudo aptitude install libatlas-dev liblapack-dev

Rebuilding from Scratch

./repotool start-over

does the trick.

Visualizing hedge's Results

Use one of

• VisIt (recommended)

• Paraview

Don't want to litter your system-wide Python installation?

Take a look at virtualenv.

Exploring the Source Code

If you like Vi/Vim

See SciComp/VimTricks.

If you like Emacs

./repotool etags

Then follow the Directions on Emacs and Tags.

Compiling with libsilo

This is optional.

Download and compile libsilo from here: https://wci.llnl.gov/codes/silo/downloads.html

1. silo-4.7 ./configure --prefix=$HOME/pool --without-readline --without-python --without-hdf5 --disable-silex

   • If the final step above results in the error message 'couldn't find Silo.h' try rerunning this step as: ./configure --prefix=$HOME/pool --disable-silex

2. silo-4.7 make install

3. Add the following lines to $HOME/.aksetup-defaults.py:

    SILO_INC_DIR = ['${HOME}/pool/include']
    SILO_LIB_DIR = ['${HOME}/pool/lib']

Do this before the final step above, and do not insert "pylo" into "src/excluded-subprojects". Then everything should get built as it's supposed to.

Problems and Solutions

Using the right Editor

As an ex-Windows user, I edited $HOME/.aksetup-defaults.py with openoffice. This can be a bad idea : http://www.python.org/dev/peps/pep-0263/

So it would be best to stick to Emacs, Vi/Vim, or IDLE.

Setuptools vs Distribute

Building a Distribute egg in /home/schwede/PIC_Hedge/pytools
/home/schwede/PIC_Hedge/pytools/setuptools-0.6c9-py2.6.egg-info already exists
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
   or: setup.py --help [cmd1 cmd2 ...]
   or: setup.py --help-commands
   or: setup.py cmd --help

error: invalid command 'develop'
------------------------------------------------------------------------
COMMAND develop ON pytools FAILED!
------------------------------------------------------------------------

If you get this error with the last step, please look at The Story about Distribute and Setuptools. This might occur if you are using Python 2.6.3.

Attribute '__doc__' is read-only

If you got the following error during developing HEDGE you probably are using Pyton 2.6.4. and a BOOST version 1.39 or 1.40.

AttributeError: 'Boost.Python.StaticProperty' object attribute '__doc__' is read-only

To solve this problem you have to install the newest version of Boost from SVN. In the following section you can find a detailed instruction how to do that. (see Python bug)

Installing with the latest SVN version of Boost

First, see the instructions on deleting an old version of Boost and gettting Boost from svn. As the SVN version has no number you have to change your '$HOME/.aksetup-defaults.py':

BOOST_INC_DIR = ['$HOME/pool/include/']
BOOST_LIB_DIR = ['$HOME/pool/lib']
BOOST_BINDINGS_INC_DIR = ['$HOME/pool/include/boost-numeric-bindings/']
BOOST_COMPILER = 'gcc43'
# WITH_SPARSE_WRAPPERS = True
BOOST_PYTHON_LIBNAME=["boost_python"]
BOOST_THREAD_LIBNAME=["boost_thread"]

to your '$HOME/.aksetup-defaults.py' as well. Now build everything again by

./repotool start-over

/usr/bin/ld: cannot find -lgfortran

If you get the following error:

/usr/bin/ld: cannot find -lgfortran

You might try to remove gfortran from your BLAS LIBNAMES:

BLAS_LIBNAME = ['f77blas','atlas']

Permission problems in built files

If you get permission errors from prior installation as root, this command will fix your problems:

src sudo chown YOUR_USER_NAME . -R

Alternatively, you may safely delete these files:

sudo rm */build -Rf
sudo rm -Rf env
sudo rm */*.egg-info -R

(BE CAREFUL when copying and pasting these commands, they are potentially dangerous!)

Cannot find numpy

./repotool start-over gives errors:

File "/home/lyyang/build/repotool/hedge/aksetup_helper.py", line 70, in get_numpy_incpath
    file, pathname, descr = find_module("numpy")
ImportError: No module named numpy

Fixed in repotool revision ff83519.

• Same problem as above ('ImportError: No module named numpy') when compiling on Ubuntu 10.04 Lucid from the git repo

  • What do you mean 'Fixed in repotool revision ff83519."? Where can I find this revision?

    • Here, for example: https://github.com/inducer/repotool/commits/master and http://git.tiker.net/repotool.git

    • As of right now, it's the newest revision of repotool. Note that you need to do a 'start-over', see above.
• Thanks for the update of the git repo. The build is now ok! (provided "pyrticle" is removed from the subproject, otherwise the build fails)

• Now, although executing the "python test_basics" works fine, all examples (such as "python waves.py") fail with the following msg: "TypeError: argument of type 'int' is not iterable". (on ubuntu 10.04 lucid, python 2.6, libboost 1.40).

• Thank you for your help.
  • I find discussing through the wiki awkward--can you please sign up for the mailingl list (see below)?

Getting help with problems

Please sign up for the Hedge/MailingList and ask there.