I’ve been wanting to get into Python-C bindings for a while.

I started experimenting with ctypes this week, so I thought I’d share what I’ve learned.

In this post I demonstrate very basic usage of ctypes to call C functions from Python. This basic demo works for me on OS X and Linux, using clang++ 3.5 and Python 2.7.

The C Module

Here’s a basic C module with 3 functions I’d like to use from Python:

// Demoing shared library callable from Python

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

// Actual functions in extern "C" block to control the symbol names in the shared library.
// Without it, C++ mangles symbol names.

extern "C" {

int add(int a, int b) {
  return a + b;
}

char * say_hi(const char * to_whom) {
  size_t msg_len = strlen(to_whom) + 5;
  char * hi_message = static_cast<char *>(malloc((msg_len) * sizeof(char)));
  printf("buffer at %p\n", hi_message);
  snprintf(hi_message, msg_len, "hi %s!", to_whom);
  return hi_message;
}

void say_my_address(const char * buff) {
  printf("buffer at %p\n", buff);
}

}

To expose the functions to Python via ctypes, I need to build a shared library.

In OS X:

> clang++ -o mything.os -c -std=c++11 -Wall -fvectorize -fslp-vectorize -fcolor-diagnostics -O2 -fPIC mything.cc
> clang++ -o libmything.dylib -dynamiclib mything.os

In Linux:

> clang++ -o mything.os -c -std=c++11 -Wall -fvectorize -fslp-vectorize -fcolor-diagnostics -O2 -fPIC mything.cc
> clang++ -o libmything.so -shared mything.os

The Python program

# #!/usr/bin/python2.7
# -*- coding: utf-8 -*-

"""Basic ctypes demo."""

import ctypes
import ctypes.util

if '__main__' == __name__:
  # Try loading Linux and OS X variants, using whichever works
  try:
    mylib = ctypes.cdll.LoadLibrary('libmything.so')
  except OSError:
    mylib = ctypes.cdll.LoadLibrary('libmything.dylib')

  libc = ctypes.cdll.LoadLibrary(ctypes.util.find_library('c'))
  print '1 + 2 =', mylib.add(1, 2)
  # Configure return type of say_hi to "pointer to char"
  # (setting it to ctypes.c_char_p will also work for getting the return string,
  # but that way I couldn't figure out how to release the string later...)
  mylib.say_hi.restype = ctypes.POINTER(ctypes.c_char)
  hi_msg_p = mylib.say_hi('Oogi')
  # Get the string from the char-pointer

  hi_msg = ctypes.string_at(hi_msg_p)
  print hi_msg
  mylib.say_my_address(hi_msg_p)
  # cleanup
  libc.free(hi_msg_p)

In OS X this runs immediately:

> python ctypes-demo.py
1 + 2 = 3
buffer at 0x7fd3f9530370
hi Oogi!
buffer at 0x7fd3f9530370

In Linux I need to add the current directory to LD_LIBRARY_PATH:

> python ctypes-demo.py
Traceback (most recent call last):
File "ctypes-demo.py", line 13, in <module>
mylib = ctypes.cdll.LoadLibrary('libmything.dylib')
File "/usr/lib/python2.7/ctypes/__init__.py", line 443, in LoadLibrary
return self._dlltype(name)
File "/usr/lib/python2.7/ctypes/__init__.py", line 365, in __init__
self._handle = _dlopen(self._name, mode)
OSError: libmything.dylib: cannot open shared object file: No such file or directory
> export LD_LIBRARY_PATH=.
> python ctypes-demo.py
1 + 2 = 3
buffer at 0x1607790
hi Oogi!
buffer at 0x1607790

Summary

So that was my “Hello world” experience with C-Python interface using ctypes.

If you figured a more elegant way to return a C allocated string, use it in Python, and free it – please let me know! If you can show me how to use smart-pointer-based data types, even better!

In addition, I hoped that ctypes.util.find_library('mything') could save me from the try-except dance, like with libc. I assumed it will search for shared libraries in LD_LIBRARY_PATH paths. This wasn’t the case in my attempts. If you know of a more elegant way for Linux-OS X portability here, I’d love to know about it!

Should be interesting to see how this fits in my SCons environment.