Supporting External Libraries In My SCons Shortcuts

By Thursday, February 26, 2015 0 , , Permalink 0

This is the fifteenth post in my SCons series. This post introduces a small enhancement to my SCons shortcuts system – nicer support for external libraries via the with_libs keyword argument.

In recent episodes, I needed to link with the protobuf library to use the Protocol-Buffers-based AddressBook library. I did this by adding the LIBS=['protobuf'] argument to the Program target, which works just fine.

If this works “just fine”, why mess with it? Well, I already mentioned my OCD, haven’t I? I already have a nicer way to refer to libraries I use, so why can’t I write with_libs=['AddressBook::addressbook', 'protobuf']? It looks a bit cleaner.

The reason this would not work as is, is because I lookup the with_libs entries in a shared dictionary of project-specific libraries (more no that in the post that introduced the shortcuts system), and “protobuf” is not a project library.

This post extends the shortcuts system to support also external libraries. In addition to improved aesthetics, I add a couple of useful features:

  • Support configuration-based list of “supported external libraries”. This allows for centralized control of external libraries used across the project, which can be very useful in projects that want to enforce policies about library-usage (e.g. licensing requirements etc.).
  • Simpler support for libraries that are not installed system-wide, taking care of icky details, like CPPPATH and LIBPATH crap.
  • Protection against potentially difficult troubleshooting due to library name typo’s.
  • External library aliases and groups.

This episode picks up where the previous episode left off. Read on for the full details, or check out the final result on my GitHub scons-series repository.

Weekly Review, February 21

By Saturday, February 21, 2015 0 Permalink 0

In case you missed it, Lenovo preinstalled an adware called Superfish on Windows laptops. Not only that this adware obnoxiously injects ads to your search results, turns out that the Superfish adware implements man-in-the-middle to do its thing!

If you think that’s bad enough, the certificate they use to do this was easy to crack… This means that anyone can use this to implement man-in-the-middle attacks against machines with the Superfish certificate installed… :-( In case you’re curious, the password is “komodia”.

Scared? Rightly so! Go ahead and check yourself, and clean it up if you’ve got it!

Interesting to see claims that the adware poses no risk… O_o

The Weekly Review is a recurring (sort-of-)weekly summary, reviewing highlights from the last week.

Adding SCons Proto Builder Shortcut

This is the fourteenth post in my SCons series. The topic of this post is adding a shortcut for the custom SCons Protoc builder from the previous episodes.

The shortcut is in line with the SConscript simplification approach described in an earlier episode. In this installment, I add a new Proto shortcut to the collection, so the address book SConscript can look like this:

"""AddressBook proto-based library SConscript script"""


AbProtos = ['person.proto', 'addressbook.proto']

Lib('addressbook', protos=AbProtos)

The final result is available on my GitHub scons-series repository.

Don’t Use Python Lists Where Generators Work Better!

By Sunday, February 15, 2015 0 Permalink 0

I recently learned about a subtle difference between lists and generators in Python.

The gist: when passing iterables, always prefer generators over lists!

Also: when returning iterables, always prefer generators over lists!

Most of the time, the intuitive way does the Right Thing. I noticed the subtlety in some cases where my intuition didn’t do the Right Thing.

In this post I explain the difference using several examples. The last example is an important one, demonstrating how fnmatch.filter() can misbehave!

Fixing the Protoc SCons Builder

This is the thirteenth post in my SCons series.

In the previous episode I integrated Protoc, an existing custom builder, in my SCons project.

This post will demonstrate how this builder fails with non-trivial projects, and suggest some fixes and improvements.

I wanted to share my proposed fix via the SCons wiki page, but I couldn’t create a user… I’d appreciate if someone with access to that wiki could assist :-) .

The final result is available on my GitHub scons-series repository.

Weekly Review, February 7

By Saturday, February 7, 2015 0 Permalink 1

The twins had their first birthday yesterday! I’m glad to let you know that we all survived their first year of existence! I wonder if this surviving thing was harder for them or for us. ^_^

I finished “audio-reading” I Will Teach You To Be Rich by Ramit Sethi last week. I realize that the title sounds like crappy self-help, but I thought I’d give this book a chance. I got several positive recommendations, and Ramit’s personal finance blog is pretty good.

Unfortunately, after reading it, I can’t recommend it to any non-US readers…

I agree with the principles that are at the essence of the book:

  1. Invest aggressively, invest early.
  2. Spend responsibly on what brings you the most value.
  3. Be as frugal as possible with everything else.
  4. Automate financial processes whenever you can.
  5. Don’t be stupid with debt, fees, tax advantage opportunities, etc.

These are great messages, worth passing on. My problem with the book is that 95% of the content delves into how these principles apply to the average 20-something american – which I’m not. Three examples from the top of my mind:

  • Going into the details of credit score at length.
  • Detailing the different types of US bank accounts.
  • Reading out phone numbers of US banks?! Seriously?!

Bottom line, if you live in the US, this book may be valuable to you. Otherwise, save your time.

The Weekly Review is a recurring (sort-of-)weekly summary, reviewing highlights from the last weeks.

The Protoc Builder: Compiling Protocol Buffers With SCons

This is the twelfth post in my SCons series. This posts continues exploring ways to work with protocol buffers in a SCons project.

In the previous episode I covered the manual approach to using protocol buffers in a SCons project. As mentioned there, SCons does not know how to compile .proto files into C++ and Python code out of the box.

This post will take the integration a step further, by actually using SCons to compile .proto files.

I am definitely not the first one to suggest this SCons extension. I started using the SCons ProtocBuilder by Scott Stafford. It worked fine for my needs, until it didn’t. This post focuses on integrating Scott’s builder as is. Future posts will deal with fixes and improvements.

The final result is available on my GitHub scons-series repository.

Shell Foo: Merging Common Files In a Directory

Say you have two directories with a bunch of text files. How can you create a third directory that will contain all the files that are common to both input directories, such that every output file is a concatenation of both input files?

Shell-Foo is a series of fun ways to take advantage of the powers of the shell. In the series, I highlight shell one-liners that I found useful or interesting. Most of the entries should work on bash on Linux, OS X and other UNIX-variants. Some probably work with other shells as well. Your mileage may vary.

Feel free to suggest your own Shell-Foo one-liners!

How To Use Protocol Buffers In a SCons Project, Take 1

This is the eleventh post in my SCons series. This post starts exploring ways to work with protocol buffer files in a SCons project.

Protocol buffers are a structured-data-serialization mechanism from Google. This is not a tutorial on protocol buffers. I will use the address book project example that appears in the official protocol buffers tutorial.

When using protocol buffers, you write .proto files to describe your structured data. You use the protoc compiler to generate C++ and Python code that allows you to serialize, load, and manipulate your protocol buffers data.

Out of the box, SCons does not know how to compile .proto files into C++ and Python code. The purpose of this post is to start exploring ways to integrate protocol buffers in the build process. The first iteration is based on manual “integration”.

The final result is available on my GitHub scons-series repository.