Installation Experience of pyCuda on SuSE_11.0 x86_64 Architecture

First, refer to the general Linux Installation Page for general procedure and prerequisites. This page deals particularly with SuSE pecularities I encountered. Maybe it helps with other systems as well. Please consider this page as a crude stub. Try yourself, find out better ways and don't hesitate to share your experience by updating this page.

Since this page is based only on a one-shot installation experience, it is not yet written as a tutorial, but may help you not to get stuck at the same points I got. If successful, please add your experience here, in the hope we end up in a usable tutorial.

Partly you'll find further details I've discussed with Andreas Klöckner in the mail list archive:
http://article.gmane.org/gmane.comp.python.cuda/977
http://article.gmane.org/gmane.comp.python.cuda/972


Disclaimer


System reference

For reference: some details of my system:

Distribution: openSUSE 11.0 (X86-64)

Kernel: Linux version 2.6.25.18-0.2-debug (geeko@buildhost) (gcc version 4.3.1 20080507 (prerelease) [gcc-4_3-branch revision 135036] (SUSE Linux) ) #1 SMP 2008-10-21 16:30:26 +0200

Python 2.5.2
Some python extensions were previously installed with YaST.
Some of those may be required by pyCuda, most I think will not.
I didn't make a systematic scrutiny of dependencies, so your experience may vary. Following list is just for reference, don't pay too much attention to it until you ran into dependency trouble.

:~> rpm -qa | grep -i python
libxml2-python-2.6.32-14.1
python-pam-0.5.0-35.1
python-gtksourceview-2.2.0-15.1
python-twisted-names-8.0.0-14.1
python-orbit-devel-2.14.3-93.1
python-gtk-doc-2.12.1-18.1
wxPython-common-gtk2-unicode-2.8.8.1-4.4
python-twisted-doc-8.0.1-14.1
python-curl-7.16.4-80.1
python-gtkglext-1.1.0-61.1
python-pygame-1.8.0-22.1
python-twisted-words-8.0.0-13.1
python-mysql-1.2.2-50.1
python-gnome-extras-doc-2.19.1-92.1
python-doc-pdf-2.5.1-75.1
python-visual-3.2.9-2.34
python-numeric-24.2-162.1
python-httplib2-0.2.0-136.1
python-libproxy-0.2.3-17.1
python-pyx-0.10-15.1
python-kde3-examples-3.16.0-124.1
python-cmdln-1.1.1-2.1
gnome-python-desktop-2.22.0-29.1
apache2-mod_python-3.3.1-123.1
python-zopeinterface-3.3.0-77.1
python-openssl-0.7a2-17.1
python-dateutil-1.1-136.1
python-qt-3.17.4-49.1
python-twisted-web-8.0.0-13.1
python-gnome-devel-2.22.0-21.1
python-xlib-0.14-52.1
python-twisted-conch-0.8.0-79.1
python-magic-4.24-15.1
python-idle-2.5.2-26.1
python-eyeD3-0.6.14-81.1
python-aqbanking-2.3.3-43.1
python-rrdtool-1.2.27-22.1
python-numpy-devel-1.3.0-10.2
python-scipy-0.7.1-2.6
python-gmpy-1.03-2.1
python-cairo-1.4.12-19.1
python-urlgrabber-3.1.0-145.1
python-opengl-2.0.1.09-224.1
python-curses-2.5.2-26.1
python-lxml-2.0.4-14.1
python-qscintilla-1.73-103.1
python-wxGTK-examples-2.8.7.1-12.1
python-vorbis-1.5a-109.1
python-sqlite2-2.4.1-16.1
python-mutagen-1.13-37.1
python-imaging-sane-1.1.6-165.1
python-gnome-bluetooth-0.11.0-52.1
python-avahi-0.6.22-68.1
python-logilab-common-0.22.2-79.1
libboost_python1_39_0-1.39.0-6.1
mapnik-python-0.6.0-1.20
python-setuptools-0.6c6-75.1
libbeagle-python-0.3.5.1-2.1
python-tz-2006p-1.1
python-compizconfig-0.7.4-28.1
python-orbit-2.14.3-93.1
python-gdbm-2.5.2-26.1
dbus-1-python-0.82.4-49.1
python-tk-2.5.2-26.1
python-gtk-vnc-0.3.3-34.1
python-cairo-devel-1.4.12-19.1
python-wxGTK-2.8.7.1-23.1
python-yui-0.1.0-3.1
python-twisted-mail-8.0.0-13.1
python-pexpect-2.1-115.1
python-libbtctl-0.10.0-20.1
python-gtkglext-devel-1.1.0-61.1
python-dialog-2.7-165.1
python-GnuPG-Interface-0.3.2-82.1
python-logilab-astng-0.17.1-84.1
postgresql-plpython-8.3.1-4.1
python-sqlalchemy-doc-0.5.6-2.2
python-tables-2.1.2-1.3
python-2.5.2-26.1
gimp-plugins-python-2.4.5-41.1
python-opensync-0.22-115.1
python-fcgi-2000.09.21-309.1
python-qt4-4.4.2-6.1
python-devel-2.5.2-26.1
python-gtk-devel-2.12.1-18.1
python-xmldiff-0.6.8-146.1
python-twisted-lore-8.0.0-13.1
python-ply-2.3-21.1
python-lxml-doc-2.0.4-14.1
python-gtksourceview-devel-2.2.0-15.1
python-dnspython-1.3.3-164.1
python-ao-0.82-305.1
python-ply-doc-2.3-21.1
python-numpy-1.3.0-10.2
python-sqlalchemy-0.5.6-2.2
python-serial-2.2-162.1
python-gpgme-0.1-44.1
python-imaging-1.1.6-165.1
python-twisted-news-8.0.0-13.1
python-pywbem-0.6.20080221.1-20.1
python-ogg-1.3-326.1
python-kde4-4.0.4-14.1
python-gstreamer-0_10-0.10.11-22.1
python-crypto-2.0.1-164.1
python-pivy-0.5.0svn765-0.pm.1.1
python-configobj-4.6.0-0.pm.2.1
python-gobject2-devel-2.14.1-15.1
python-twisted-8.0.1-14.1
python-gnome-extras-2.19.1-92.1
pythoncad-DS1_R36-148.1
python-pygame-doc-1.8.0-22.1
python-kid-0.9.6-75.1
python-demo-2.5.2-26.1
python-gammu-0.25-24.1
python-sexy-0.1.9-0.pm.3.1
python-xml-2.5.2-26.1
python-gobject2-2.14.1-15.1
python-ldap-2.3.4-16.1
python-doc-2.5.1-75.1
python-wxGTK-doc-2.8.7.1-12.1
python-scgi-1.12-89.1
python-kde3-doc-3.16.0-124.1
python-bibtex-1.2.4-85.1
python-goocanvas-0.9.0-56.1
python-gtk-2.12.1-18.1
python-gdata-1.0.11.1-16.1
python-sip-4.7.6-5.1
python-gnome-2.22.0-21.1
python-kde3-3.16.0-124.1
python-twisted-runner-8.0.0-13.1
python-obexftp-0.21-163.1
python-gobject2-doc-2.14.1-15.1
wxPython2.8-gtk2-unicode-2.8.8.1-4.4
wxPython2.8-devel-gtk2-unicode-2.8.8.1-4.4

Install CUDA

My GPU is a GeForce 9800 GT. I got CUDA from nvidia page and installed

The CUDA-examples in ~/NVIDIA_CUDA_SDK/bin/linux/release/ work in a more or less satisfying way.

Trials on pyCuda

:~>tar -xvzf pycuda-release-0.93.tar.gz
:~>cd pycuda/
:~>./configure.py

This will give you a siteconf.py which will probably not work on a standard SuSE-box:

:~>python setup.py build
will probably fail, complaining something like

/usr/lib64/gcc/x86_64-suse-linux/4.3/../../../../x86_64-suse-linux/bin/ld:  cannot find -llibboost_python-mt collect2: ld returned 1 exit status error: command 'g++' failed with exit status 1

It took me a whole night and a hard kick from Andreas to get it.

Result in short - this is the siteconf.py that works for me:

BOOST_INC_DIR = ['/usr/include'] 
BOOST_LIB_DIR = ['/usr/lib64'] 
BOOST_COMPILER = 'gcc43' 
BOOST_PYTHON_LIBNAME = ['boost_python-mt'] 
BOOST_THREAD_LIBNAME = ['boost_thread-mt'] 
CUDA_TRACE = False 
CUDA_ENABLE_GL = True 
CUDADRV_LIB_DIR = ['/usr/lib64'] 
CUDADRV_LIBNAME = ['cuda'] 
CXXFLAGS = [] 
LDFLAGS = []

According to Andreas, you may not even need to run ./configure.py. Simply create a siteconf.py with above content using your favourite editor, which is vi, of course ;-). If it works, please update here.

Issues I encountered up to this solution:

libcuda comes from cuda installation.

The boost libraries come from boost-devel-1.39.0-6.1 which was already on my box.
Check with rpm -q boost-devel.
If missing, install it at best using YaST.
My Version 1.39... is from repository opensuse_Geo, which was configured at my system from another project.
Genuine to the openSuSE-11.0 distribution is 1.34.1... Maybe this works as well. If so, please notify here as it makes installation easier.

Having this in place, give it a new try:

python setup.py build

Hope that it will compile now without serious complaints and produce a build/lib.linux-x86_64-2.5/pycuda/_driver.so

Do a su -c "make install". Maybe it were more consitant with python usage to do python setup.py install instead. Try, learn, share experience here. This will put things in a place supposed to be a proper one. In my case this is /usr/local/lib64/python2.5/site-packages/pycuda.

Following SuSE convention, the even more proper place might be /usr/lib64/python2.5/site-packages/pycuda.
RTFM in pycuda root dir to learn about configuring this.
Try out and share your learnings here.

Check out the examples

Feel happy for a little while but not too much:

cd examples/
python hello_gpu.py 
.... pytools not found

... gooogle .... pytools is a piece of Andreas' software which his applications always seem to rely upon. Here http://article.gmane.org/gmane.comp.python.cuda/976 he said that it should get installed automagically with pycuda. For me, it didn't, but this may be due to side effects of some earlier undocumented fiddling. Try, learn, share. If you get the same error than I did, get pytools from http://pypi.python.org/pypi/pytools.

Version in my trial:
pytools-9.tar.gz
python setup.py build
su -c "python setup.py install"
... as far as I can remember...

Having gone so far, I happily could run those examples:
hello_gpu.py demo.py demo_struct.py dump_properties.py demo_elementwise.py.
Hope you can share my happyness till here.

Python-Open-GL-Version

Before getting drowned in happiness, run gl_interop.py.

I got No module named vertex_buffer_object. Obviously OpenGL.GL.ARB.vertex_buffer_object  is new in python-opengl ~ Version 3. There are only rpms for Version 2.0.something for SuSE 11.0. Think it might be available for SuSE 11.2 upwards.

I have bad experience with forcefully mixing rpm packages from different distribution versions. So, better, I got new source from http://pyopengl.sourceforge.net . Build and install, obviously to /usr/local/lib64/python2.5/site-packages/OpenGL/

However, gl_interop.py did not run until I did export PYTHONPATH=/usr/local/lib64/python2.5/site-packages/ (was PYTHONPATH="" before)

So it might be preferrable to install it according to SuSE scheme to /usr/lib64/python2.5/site-packages/OpenGL/

The PYTHONPATH issue is not yet really clear for me. I'm still in discussing it with Andreas on the list. In case of trouble, stay tunded. In case of a solution, please share. I don't know whether it is advisable to upgrade systemwide pyopengl-2.X to ...3.X without screwing other applications.

Anyway - run gl_interop.py again and enjoy - so I hope.

Other dependencies

demo_meta_codepy.py ImportError: No module named codepy.cgen

codepy refers also to another project of Andreas: http://documen.tician.de/codepy/
I did not find any download, just a versioned repository:
http://git.tiker.net/codepy.git/tree
Might get it with git. Try man git if it is on your box. Admitted, I skipped this.

demo_meta_template.py import - Error: No module named jinja2
Maybe relevant pointer: http://jinja.pocoo.org/2/documentation/
I skipped this as well.

python plot_random_data.py No module named matplotlib.pylab
No suitable matplotlib to be found in YaST, so I went to
http://matplotlib.sourceforge.net/ :
download, untar, read but not understand INSTALL - dozens of dependencies
python setup.py build yields many warnings ...deprecated or antiquated header - ignore
... install ... Writing /usr/local/lib64/python2.5/site-packages/matplotlib-0.99.1.1-py2.5.egg-info

python plot_random_data.py
.... seems to run but not to show anything???
ah, it saves its output to a file: plot-random-data.png

Other tests

Rest in example dir seems to run as intended.
cd ../test scripts there seem to run but not to produce any exaggerating effect.
cd undistributed scripts there seem to run until GPU memory error. Maybe this is intended behaviour to crudly check available memory. If courious, check source and let us know

Ways to go from here