Tag Archives: Python

Let’s code about bike locks some more

I just got sucked into Peter Norvig‘s Let’s Code About Bike Locks. If you haven’t read that yet, read it first, at least the beginning. Otherwise this will make no sense.

The strategy of starting from the first tumbler and then calculating the subsequent ones seemed like it could be improved on. What if we looked at all the four-letter English words, and chose the most common letter, on any tumbler, fixed that letter on its tumbler, and then continued on to the second most common letter, on any tumbler, and so on? How well could we do?

So I coded it up, and here’s the result, a lock that can make 1,410 words from Norvig’s list at http://norvig.com/ngrams/words4.txt, 170 more words than his best:

Lock: ABCDGLMPRST AEHILNORUY ACEILMNORST ADEKLNORSTY

I believe Norvig’s strategy of improving a lock with random permutations also would be less likely to improve on this lock. Changing any letter would, by definition, be choosing a letter that occurs in that position in fewer words. However, it’s still possible to improve; there might be some better letter choices that, while poorer overall, are still better for the specific other letters already chosen for the other tumblers.

Update 15 Jun 2015: Someone was wrong on the internet and this time it was me! Astute readers will notice that a tiny off-by-one bug in my implementation (see the fifth revision) led it to generate a lock with three tumblers with eleven letters each, and one tumbler with ten letters.

The new best lock from this implementation only generates 1,161 words, leaving Norvig’s solution the best still:

Lock: ABCDLMPRST AEHILNORUY AEILMNORST ADEKLNOSTY

A tour of the differences between JavaScript and Python

Introduction

JavaScript and Python are two very important languages today. Too many programmers, however, work in both languages, but know just one of them well. This means they end up writing code in one language in the same style as the other, unaware of some of the more subtle differences between the two. If you know JavaScript or Python well, and you want to improve your skills and knowledge of the other, I wrote this article for you.

Disclaimer: I know Python (slightly) better than I know JavaScript, and I’ve not done any JavaScript outside of the browser, so I tried to keep the bits about JavaScript agnostic to the host environment, but, fair warning, there may be subtle differences in server-side JavaScript that are not mentioned here because I don’t know them.

Continue reading →

Introducing Fragments

Today I am announcing Fragments, a project I’ve been working on for a few months. It’s on GitHub and PyPI.

Fragments uses concepts from version control to replace many uses of templating languages. Instead of a templating language, it provides diff-based templating; instead of revision control, it provides “fragmentation control”.

Fragments enables a programmer to violate the DRY (Don’t Repeat Yourself) principle; it is a Multiple Source of Truth engine.

What is diff-based templating?

Generating HTML with templating languages is difficult because templating languages often have two semi-incompatible purposes. The first purpose is managing common HTML elements & structure: headers, sidebars, and footers; across multiple templates. This is sometimes called page “inheritance”. The second purpose is to perform idiosyncratic display logic on data coming from another source. When these two purposes can be separated, templates can be much simpler.

Fragments manages this first purpose, common HTML elements and structure, with diff and merge algorithms. The actual display logic is left to your application, or to a templating language whose templates are themselves managed by Fragments.

What is fragmentation control?

The machinery to manage common and different code fragments across multiple versions of a single file already exists in modern version control systems. Fragments adapts these tools to manage common and different versions of several different files.

Each file is in effect its own “branch”, and whenever you modify a file (“branch”) you can apply (“merge”) that change into whichever other files (“branches”) you choose. In this sense Fragments is a different kind of “source control”–rather than controlling versions/revisions over time, it controls fragments across many files that all exist simultaneously. Hence the term “fragmentation control”.

As I am a linguist, I have to point out that the distinction between Synchronic and Diachronic Linguistics gave me this idea in the first place.

How does it work?

The merge algorithm is a version of Precise Codeville Merge modified to support cherry-picking. Precise Codeville Merge was chosen because it supports accidental clean merges and convergence. That is, if two files are independently modified in the same way, they merge together cleanly. This makes adding new files easy; use Fragment’s fork command to create a new file based on other files (or just cp one of your files), change it as desired, and commit it. Subsequent changes to any un-modified, common sections, in that file or in its siblings, will be applicable across the rest of the repository.

Like version control, you run Fragments on the command line each time you make a change to your HTML, not before each page render.

What is it good for?

Fragments was designed with the task of simplifying large collections of HTML or HTML templates. It could replace simpler CMS-managed websites with pure static HTML. It could also handle several different translations of an HTML website, ensuring that the same HTML structure was wrapped around each translation of the content.

But Fragments is also not HTML specific. If it’s got newlines, Fragments can manage it. That means XML, CSS, JSON, YAML, or source code from any programming language where newlines are common (sorry, Perl). cFragments is even smart enough to know not to merge totally different files together. You could use it to manage a large set of configuration files for different servers and deployment configurations, for example. Or you could use it to manage bug fixes to that mess of duplicated source files on that legacy project you wish you didn’t have to maintain.

In short, Fragments can be used anyplace where you have thought to yourself “this group of files really is violating DRY”.

Use it

Fragments is released under the BSD License. You can read more about it and get the code on GitHub and PyPI. And you can find me on Twitter as @glyphobet.

Special thanks to Ross Cohen (@carnieross) for his thoughts on the idea, and for preparing Precise Codeville Merge for use in Fragments.

Python else in loops: survey results

As promised, here are the results of two totally unscientific surveys, one conducted at PyCon 2011 and the other over Twitter just a few days ago, about the behavior of else in Python loops. The results show that only 25% of respondents know what else in a Python loop actually does, and 55% think they know but are wrong.

Grasping the nuclear fourth rail of Python syntax with both hands and holding on for dear life

In Python vs. Ruby: A Battle to The Death, Gary Bernhardt wishes for Ruby-style blocks in Python.

The BDFL has already weighed in on anonymous blocks in Python:

If you want anonymous blocks, by all means use Ruby. Python has first-class callables,1 Ruby has anonymous blocks. Pick your favorite, but don’t whine that neither language has both. It ain’t gonna happen.

This seems to imply that first-class callables and anonymous blocks are mutually exclusive language features, but that’s wrong: JavaScript has the ability to pass callables around like anything else, and it has anonymous functions, which can be used just like Ruby’s anonymous blocks. Does that mean JavaScript is better than Python or Ruby? My feelings about Ruby are indifferent with a chance of rain, but I love Python, so I’ve got to ask: are you going to take this lying down, Python?

I’m not sure Python needs full-blown, Ruby-style anonymous blocks. But it might be good enough to be able to use function definitions as r-values, like JavaScript can. (If you’re not already wearing your tinfoil hat, now might be a good time to put it on.)

This would allow asynchronous code to be written in conceptual order (just like in JavaScript):

do_something_asynchronous(param1, param2, (def callback(result):
    do_something_with_result(result)
))

And it would allow encapsulation of lexical scope inside specific iterations of a loop to be used later when an asynchronous call returns (just like in JavaScript):

for item in results:
    (def single_iteration():
        do_something_asynchronous(param1, item, (def callback(result):
            do_something_with_result_and_item(item, result)
        ))
    )(item)

I’ve even had occasion to want that other Python namespace, class, to operate as an r-value:

class MyConverterClass(BaseConverterClass):
    my_converter_type = (class MyConverterType(float):
        def __str__(self): # a custom __str__ method
            return '%0.2f' % self
    )

In these examples I’ve wrapped their inline definitions in Python’s great syntactical equalizer, the parenthesis. It would be even nicer to be able to leave them off, but I’m sure that this syntax would run headlong into Python’s whitespace-based block definitions, and it would be even more of a train-wreck without parentheses.

I’ve also named the defs and classes. It would also be nice to be able to omit the function or class names if they were unneeded (just like in JavaScript). But anonymous functions, a.k.a. lambdas, are the electric third rail of Python syntax, so anonymous classes would be… I dunno, the nuclear fourth rail?

  1. Thanks to Steve for explaining that by “first-class callable,” GvR means functions that can be passed around and assigned to other variables without being executed. He also pointed out that the reason Ruby’s callables aren’t first-class is because optional parentheses in function calls leave no way to refer to a function without calling it, not because of the existence of anonymous blocks. [↩]

Some impossible objects before breakfast

Lately I’ve been playing with rapid prototyping, also known as three-dimensional printing. It’s a (relatively) new fabrication method that allows creation of shapes that would be impossible to create by moulding. It allows for creation of things like interlinked rings, objects trapped inside other objects, or complex voids.

Shapeways, a Netherlands-based website, offers high-quality, relatively cheap rapid prototyping, and a place to host your own selection of models. As a website, Shapeways is no RedBubble — the interface and marketplace tools leave much to be desired. The ratings and sorting can be gamed, it’s trivial to figure out the markup on another user’s model from its price, there’s no way to replace a published model with an improved one, and the site has encoding and markup issues. But what’s a few flaws when you own the category?

The free version of Google SketchUp has been satisfactory for rendering thus far, and its user interface is leaps and bounds ahead of the bizarre interface of Blender. I’m sure Blender has a superior feature set, but what good is power when you can’t figure out how to use it? Meshlab and AccuTrans3d have both come in handy for checking surfaces and converting between formats.

On to the models! Click on the pictures to see more views of each.

Trapped Outside

Trapped Outside is a model of Boy’s Surface with a sphere trapped in the space cut out by the one-sided surface. Boy’s Surface an immersion of the projective plane, which means it is a Möbius strip with a disk glued to its edge. It is a non-orientable surface with no edges and no pinch points.

Trapped Outside

Trapped Outside was fairly easy to create using Google SketchUp. It’s just a few circles extruded along each other here and there.

Hollow Knotted Gear

The Hollow Knotted Gear, inspired by Oskar van Deventer‘s Knotted Gear, consists of two interlinked knots; a trefoil knot (in green) and its dual, a 3,2 torus knot. The green trefoil forms a rectangular cross section and a triangular hole. The blue knot forms a triangular cross section and a rectangular hole. The two knots gear perfectly together, and can move around each other, but only if they are both moved simultaneously.

Hollow Knotted Gear

After many failed attempts at getting various applications to render this complex extrusion properly, I wrote a small Python program to calculate the surface for me and output VRML. Then, after much more trial and error, I used AccuTrans3d via Wine to convert the VRML surface to a DAE file for uploading to Shapeways, and to a 3DS file for examining in Google Sketchup (and to take screenshots).

There are a few more of my designs on Shapeways right now, and even more rattling around in my head just waiting to be prototyped.

Bring on the patches

Zed Shaw’s recent Python Neglect article raises a few points that are unquestionably valid: easy_install and mx.DateTime suck big time, and it sounds like the email tools have some serious problems. But I would like to hear better explanations from him about:

  • What the datetime module (introduced to replace mx.DateTime and fix shortcomings with the time module) is missing? It’s always handled everything I’ve thrown at it.
  • What is wrong with optparse? Usage? Functionality? Implementation?

I also just plain don’t understand his complaint about del versus remove. I think mystuff.remove(mything) and del mystuff[4] do exactly the same thing, so what’s the problem? That del exists?

His promise to provide patches will be an interesting test of the Python community’s ability to judge patches on their objective merit, not by their contributor, since Shaw seems to have a tendency to ruffle feathers1.

And a “WSGI for templating,” and efficient state machines with generators? Sounds exciting.

  1. This is the pot calling the kettle black here. [↩]

What isn’t new in Ruby 1.9.1 (or in Python)

Like Josh Haberman, I was excited to see the changelog for Ruby 1.9, but immediately disappointed by its vagueness and terseness.

This list is meaningless to anyone who isn’t already familiar with the changes that have been happening in Ruby 1.9.x.

For someone like me who tried an older version of Ruby, there’s nothing to read that will tell me whether it’s worth checking out again.

Take this example, from the changelog:

  • IO operations
    • Many methods used to act byte-wise but now some of those act character-wise. You can use alternate byte-wise methods.

That’s terrifying. If I’m switching to a new version, I need to know exactly which methods have changed and which ones haven’t. Saying that “some” have changed is almost less helpful than saying nothing at all.

Here’s hoping that “improved documentation” will make it into a future Ruby 1.9.x release.

In the same blog post, Haberman makes some inaccurate assertions about Python’s encoding support:

Python has taken an “everything is Unicode” approach to character encoding — it doesn’t support any other encodings. Ruby on the other hand supports arbitrary encodings, for both Ruby source files and for data that Ruby programs deal with.

Incorrect. For the last five and a half years, since Python 2.3, source code in any encoding has been supported, and Python 3.0 will expect UTF-8 by default. And of course, Python supports exactly the same wide range of encodings for data. Python’s approach can best be described as “Unicode (via UTF-8) is default.”