Python: Installing PIP and Virtualenv without root

Using virtualenv in combination with pip is a great way to run isolated python environments with all kinds of packages. However if the system you are using does not have pip, easy_install or virtualenv installed system-wide, it’s a little bit tricky to set up.

With some experimentation and some googling (stackoverflow), here’s a quick guide to do it. I’m assuming you have python already on the system.

First, we need ez_setup. The following commands grab the scripts and install them to ~/.local/bin:

wget https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py
python ez_setup.py –-user

Now we can install pip in the same way:

curl -O https://bootstrap.pypa.io/get-pip.py
python get-pip.py –-user

Now we have pip, and just need to add it to the path variable (add it to your .bashrc):

export PATH=~/.local/bin:$PATH

Finally, we can install packages. Get virtualenv:

pip install --user virtualenv

Now you can set up virtual environments for python, which is well-documented here.

Neat Python Debugging Trick

Debugging python is a pretty painless experience using pdb, the built in standard debugger. The basic usage is to insert breakpoints in the code:


import pdb

print('hello world')
print('entering breakpoint')
pdb.set_trace()
sum = 0
for x in range(0, 10):
    sum += x

When the interpreter hits ‘pdb.set_trace()’ it’ll launch the interactive debugging interface, which lets you investigate the different variables, print things and run calculations. A tutorial on using pdb can be found here.

A really useful trick using this snippet allows for quick debugging anywhere in your code. Place the following lines at the top of your python script:

import signal
    def int_handler(signal, frame):
       import pdb
       pdb.set_trace(frame)
signal.signal(signal.SIGINT, int_handler)
Then run the script, and wherever you want to start a debugger press ‘ctrl’+’c’. Press ‘c’ to carry on with execution. Really useful for unpredictable bugs!

Compiling OpenMM as a 64-bit library on Windows

A problem with developing scientific software on Windows is the difficulty in installing scientific packages, which typically provide only very minimal support. To one who isn’t well-versed in CMake, cross-compilation or cygwin, it can be quite daunting. Fortunately, OpenMM, an open source GPU-accelerated molecular dynamics package supporting both CUDA and OpenCL breaks the mould and at the time of writing provides precompiled 32-bit binaries built under Visual Studio 2010 as well as decent instructions on how to compile it yourself. It’s a blazingly fast tool that has run well on every device I’ve tried, while also being extremely flexible (custom forcefield terms automatically turn into optimised OpenCL kernels!). Using it, I’ve been able to develop real-time interactive protein simulations:

For running simulations, downloading the precompiled library and using the python interface should suffice. However, if you want to use OpenMM as a library with an existing C++ application and use a different version of Visual Studio – or your application is 64-bit – you’ll probably need to compile it yourself.

What follows is a short guide for compiling and installing OpenMM 6.0.1 as a 64-bit library for Visual Studio 2012.

Assuming you’ve downloaded OpenMM, and have opened up the user guide (here), follow Chapter 9 ‘Compiling OpenMM From Source’ in that document, but add in the following tweaks for 64-bit.

Once you get to Section 9.2.2.2, when you click Configure and are prompted to select a Generator, choose ‘Visual Studio 12 Win64’, or whatever version you have. Make sure to read through all the options and tick anything you need.

Carry on following the user guide, and try to build it.

You’ll probably get an error like the following:

 error LNK2019: unresolved external symbol __imp_pthread_create referenced in function "public: __cdecl OpenMM::ThreadPool::ThreadPool(int)" (??0ThreadPool@OpenMM@@QEAA@H@Z) C:\Code\Libs\Source\OpenMM\build\ThreadPool.obj OpenMM 

The problem is that OpenMM ships with a 32-bit distribution of pthreads, but is trying to link with a 64-bit version. Thankfully, SimTK have a version available here. Download  ‘pthreadVC2_x64.dll’ and ‘pthreadVC2_x64.lib’. These should live in the directory ‘src\libraries\pthreads\lib\’, where ‘src’ is where you downloaded the OpenMM source. You also need to put them in ‘openmm_build\Release\’.

Now open the Visual Studio solution and change the dependencies for the projects OpenMM, OpenMMCUDA and OpenMMOpenCL (if you’re building these). Do this by right clicking on the project, selecting ‘Project Properties’. In the window that appears, click Link->Input, and in the ‘Additional Dependencies’ field change the ‘pthreadVC2.dll’ to ‘pthreadVC2_x64.dll’. Do that for each project, then try to build again. With a bit of luck, it will work. If anybody alters the CMake configuration to automatically do this, I’d like a copy, as this is a bit of a dirty hack.

Run the unit tests, then build the ‘INSTALL’ project. Once that’s done, navigate to the installation lib directory (‘C:\Program Files (x86)\OpenMM\lib’ by default), and paste the pthread files there too. Make sure you set the environment variables as described in the User Guide, and you should be able to run some MD!