rants.org

Breaking news from James Vasile:

Scientists going back to Galileo have secretly abused corpses in the pursuit of knowledge. Researchers studying the Atlantic Ocean garbage patch have been throwing dead pigs into the sea just to see what happens next. Yes, that’s shocking and gross, but just like in the Whitewater scandal, it’s the coverup that is the crime.

National Geographic originally (maybe accidentally?) revealed the dead pig experiment in an article at nationalgeographic.com. You can clearly see references to the dead pigs in Google’s search results (see attached screen shot). The National Geographic url, though, makes no mention of the pigs at all. They scrubbed the record.

Why the coverup, National Geographic? Who are you protecting?!

Naturally, we had the rants.org crack investigative team look into this. James is right — something does smell funny here, and it’s not just dead pigs.

Let’s start with the Google search results. As you can see, at some point in the past the National Geographic article did talk about scientists using dead pigs to measure something about a newly-discovered Atlantic Ocean garbage patch:

The Google search results; red highlights the key words. (Original image here.)

The exact words in the Google preview snippet are: “After dropping dead pigs into the sea and watching via Webcams, researchers were…”.

Uh. What?

But if you visit the article now, it no longer mentions anything about dead pigs — the red arrow points to a failed search for the word “dead” in the page:

The National Geographic article; red arrow highlights the failed search for “dead”. (Original image here.)

The original article is no longer in Google’s cache, unfortunately; it’s been superseded by a revised version of the article. But the cached version does have a message at the top indicating that the word “pigs” appears only in links from other pages, not in the article itself, so obviously other people were on to this too:

Google’s cache of the page has already lost the mention of dead pigs. (Original image here.)

(The word “dead” doesn’t appear in the cached text of the article either; it’s just in the title of an unrelated sidebar news item in the cache.)

I know National Geographic isn’t the New York Times (then again, neither is the New York Times… but I digress). Still, silently removing a substantial fact from an article, leaving no notice of the change? That’s just not kosher.

National Geographic, the people have a right to know: what’s behind the dead pig coverup?

I just read an interesting post on Gabriella Coleman’s blog: The Hacker as Troller and Trickster, in which she points out that the “trickster” character of myth is increasingly embodied — non-mythically, non-allegorically, but in glorious corporeality — by the hackers around us, and by the trolls and all the other anonymity-encouraged occasional pranksters that make the Internet what it is.

In Lewis Hyde’s “Trickster Makes this World”, Coleman says, he asks if there are tricksters in modern industrial societies, and concludes (surprisingly, for 1998) not. The trickster needs a polytheistic system “or lacking that, he needs at least a relationship to other powers, to people, to instructions, and traditions that can manage the odd double attitude of both insisting that boundaries be respected and recognizing that in the long run their liveliness depends on having those boundaries regularly distributed” (p. 13)

Double attitude? Boundary redistribution? Sound anything like people you’ve been met on the Net?

I was kind of surprised by Hyde’s thesis in general (though I haven’t read his book (yet), so you’re now hearing the thesis third-hand, and I could be mangling it). I think there’s sometimes a temptation to assume that modern society is such a sharp break from our tribal, hunter-gatherer past that there are entire categories of human behavior no longer accessible to us. And that may be partly true… it’s just probably not as true as we think it is. Trickster is an impulse deep in human nature; it hasn’t gone away, and if anything, modern industrial society — which was heavily anonymized even before the Net came along — offers more opportunities for people to try out deviant or odd behaviors temporarily, safe in the knowledge that they are not necessarily asserting a permanent identity.

Well, there are my deep thoughts for the day. Biella’s whole post is very much worth reading.

I will always revere the Red Vic movie house because I saw my first Bruce Lee movie there. Now, an event almost as important is about to take place:

Nina Paley’s beautiful animated film Sita Sings the Blues will be playing at the Red Vic for three nights next week:

     Tues, Feb.  9  -  7:15pm, 9:15pm
      Wed, Feb. 10  -  2:00pm, 7:15pm, 9:15pm
    Thurs, Feb. 11  -  7:15pm, 9:15pm

If you haven’t seen it, go see it! If you have seen it, go see it! (Did I leave anyone out?)

You can also buy the DVD or donate to the filmmaker, who released the film under a free license because she wanted people to see it and share it.

Directions to The Red Vic:

The Red Vic Movie House is located on 1727 Haight Street (map),between Cole and Shrader, just a block and a half east from Golden Gate Park. The Red Vic is also served directly by MUNI routes: 7,33,37,43, & 71. MUNI route 6 & N-Judah come within a few blocks.

I meant to post this earlier. But hey, I suck, so I’m posting it now instead, the night before the class:

As part of Ubuntu Developer Week, I’m teaching a class on How to get started hacking Launchpad, at 20.00 UTC on Thursday, 28 Jan. If you’re a Launchpad user, come join us! The session will be held in IRC; details are on the classes page, and you can use either a regular IRC client or Lernid to participate.

No previous Launchpad hacking experience is necessary. You’ll probably want some familiarity with Python, but you don’t have to be an expert. This class will just introduce you to how Launchpad is put together, and the process for making an improvement to Launchpad. The code for Launchpad is fully open source and community hacking is already happening.

Come to the class to find out more!

From the post at QuestionCopyright.org:

Calling all law students (or at least the ones who weren’t planning to work for the RIAA later):

We’re looking for a legal intern — someone interested in learning more about copyright law and using it to promote freedom.

We have several projects right now with legal components, and expect more in the future. The responsibilities of the position will be varied, involving research in U.S. and European copyright law, non-profit law (federal and CA state), tracking legislative developments, some writing, etc. The time commitment is about five hours a week, with more available if you want it. A New York City location is ideal, but not required. There may be some limited travel, at your discretion.

The position is unpaid, but you would be working with an experienced lawyer (our counsel, Karen Sandler), and we’re happy to meet reasonable requirements for law school credit.

Interested? Contact us. We’ll keep the posting open until we get the right candidate — it could be you!

Spread the word!

(Graph from Robert Orenstein, by way of Jim Blandy.)

A recent conversation on an open source mailing list reminded of two fallacies I’d been wanting to write about. (And what is a domain like “rants.org” for, if not the debunking of fallacies?)

The first fallacy is that bugs are bad, or rather, that growth in the number of bug reports in your bug tracker is bad. The second fallacy is thinking of bugs as a form of technical debt.

Taking them one at a time:

Seeing bugs in your tracker is not bad news — bugs, in the aggregate, are good news.

The number of bug reports is proportional to the number of users, not to the number of defects.

This doesn’t mean projects should ignore bug reports, of course. It just means that you shouldn’t be alarmed as the number of bugs in your bug tracker increases. Bug growth is a sign of success: you’re getting users. The bug report rate is a proxy for the user acquisition rate.

The corollary is: you cannot expect to close all the bugs in the tracker. In fact, you shouldn’t even want that, because if you were to succeed, it would mean you’re not getting new users anymore.

This is counter-intuitive. All programmers want to fix every bug they know about; none of us want to ship with known bugs, though we always do. What does it mean if we can’t close every bug in the bug database?

A healthy project is in a constant state of triage between bugs and feature development, and the bugs must not always win. A project can’t let the bug database determine how developers spend all their time, any more than an individual person can let their email inbox determine how they spend all their time. If you’re not expecting to close all the bugs, and you want to acquire more users (so you can get more bugs), then you have to both fix bugs and add or improve features, and you have to be fundamentally comfortable with an ever-growing database of bug reports.

Think of it this way: every unimplemented feature or improvement is also a bug, in a broader sense, whether it’s filed as such in the tracker or not. It is an absence of something that should be present in the software. So if you manage to close out all your bugs, but you ship without that improvement, then you’re still shipping with a bug, you’re just calling it something else (or not calling it by any name at all). Shipping with no bugs at all is therefore impossible for any active project. It might be theoretically possible for some extremely narrowly-scoped project, but by definition such a project would not be “active”: it would achieve its goal and then remain static and perfect forever. Chances are this does not describe most projects you work on.

One of the big fears developers have is that an ever-growing bug database will overwhelm them — that they’ll spend all their time triaging bugs, asking reporters for more information, etc, and not enough time actually developing. This fear is not groundless: if every bug report is seen as a crisis requiring the attention of a core developer, things will grind to a halt as soon as there are too many users. The trick is to have tools that enable the community as a whole to manage the bug database with increasing economies of scale, instead of expecting developer attention to scale. (Apport and Launchpad’s inline dup-finding are two examples of such techniques, and there are many others.) The number of technically-inclined users who can help out with bug management will grow roughly proportionally with the number of users who are inclined to file bugs at all. So if the project provides ways for the first kind of user to participate, the second kind of user will be a help not a hindrance.

It’s true that core developers will need to spend a greater percentage of their time on bug triage as a project matures, but that percentage eventually hits a ceiling and levels off — after all, it has to, since there’s no way a developer can spend more than 100% of her time managing bugs, and yet the rate of incoming bug reports is going to increase with users. So if a developer is going to spend less than 100% of her time on bug management by definition, she might as well choose what percentage it’s going to be, since she’s going to not touch an arbitrary (and increasing) percentage of reports anyway.

Emotionally, this can be difficult for developers, because we are tempted — the second fallacy — to treat bugs as technical debt. In the mailing list conversation I referred to earlier, this equation was made explicitly, and it’s not the first time I’ve seen that happen. But I think it is a category error.

Bugs and technical debt are entirely different things. Fixing a bug may reduce tech debt, leave tech debt unchanged, or even increase tech debt, depending on how the fix is done.

Technical debt is a great concept. As far as I know, it was first introduced by Ward Cunningham (yes, that Ward Cunningham, the person who invented wikis):

Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite… The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation, object-oriented or otherwise. [1]

In other words, all those places in your code where there are comments like "FIXME: this duplicates code in turtles.c; we should really abstract this out" are tech debt. You pay interest on the debt whenever you have to make changes to code that is not as well-factored and maintainable as it could be. You pay back the debt when you finally refactor that code and remove the FIXME comment.

A bug is something completely different: it’s an instance of the software not behaving the way the developers or users expected. You could even increase technical debt by fixing a bug, if you improve the program’s behavior in a way that reduces the overall maintainability or cleanliness of the code (that’s how all those FIXME comments got there in the first place!).

So do not think of that swelling bug tracker as a debt that needs to be paid back. It’s not. It needs to be managed intelligently, and there are many techniques for doing so. But it’s going to grow, and if your project succeeds, it’s going to grow forever. Trying to suppress that growth (for example, by discouraging filings from all but technically qualified reporters) simply squelches a useful information source. Far better to know how many bugs are coming in, how fast, and of what nature, so you can understand how your user base is growing and develop appropriately scaled statistical mechanisms for handling the problems they encounter. Thinking of that information as a debt, rather than a resource, can cripple your project.

This is embarrassing — I meant to post this more than half a year ago, and apparently forgot to click “publish” on the draft. Now it’s old news, but I’ll post it anyway. Maybe some people will go out and buy the book, which is a lot of fun to read.

O’Reilly released a new [well, it was new when I originally wrote this post!] book called Beautiful Teams, edited by Jennifer Greene and Andrew Stellman. The title says it all: what makes a software development team work well?

Considering that programming is traditionally (though not necessarily) a solitary activity, that programmers are not noted for being social animals anyway, and that software architecture does not always lend itself to the divisions of labor that would be ideal from a human perspective, the question is actually rather complicated.

My chapter, Teams and Tools is available online under a free licence. It’s about the tremendous impact that seemingly trivial tweaks to collaboration tool behavior can have on a team. But get the whole book if you can: it’s well-edited, and because each chapter is by a different author, you get the benefit of many minds and many styles. Congratulations to Jennifer and Andrew for taming us all and getting a good book out of it!

Happy New Year!

My New Year’s Resolution: Inbox Zero.

0

(Wish me luck.)

The award-winning (and freely licensed) film Sita Sings the Blues will have a week-long run in New York City’s IFC Film Center, December 25th – 31st!

This is a full theatrical run, with 7-8 screenings a day. The filmmaker, Nina Paley (who is now Artist-in-Residence at QuestionCopyright.org), will be doing Q&A after the 8:25pm show most nights. On Monday, Dec. 28th, I’ll join her to take questions about the film’s free distribution model and the free culture movement.

Tickets are available online. Here’s a show schedule (click on the time to purchase tickets for that show):

• Fri, Dec 25 at: 11:00 AM, 12:35 PM, 2:30 PM, 4:30 PM, 6:25 PM, 8:25 PM, 10:20 PM, 12:10 AM
• Sat, Dec 26 at: 11:00 AM, 12:35 PM, 2:30 PM, 4:30 PM, 6:25 PM, 8:25 PM, 10:20 PM, 12:10 AM
• Sun, Dec 27 at: 11:00 AM, 12:35 PM, 2:30 PM, 4:30 PM, 6:25 PM, 8:25 PM, 10:20 PM
• Mon, Dec 28 at: 11:00 AM, 12:35 PM, 2:30 PM, 4:30 PM, 6:25 PM, 8:25 PM, 10:20 PM
• Tue, Dec 29 at: 11:00 AM, 12:35 PM, 2:30 PM, 4:30 PM, 6:25 PM, 8:25 PM, 10:20 PM

IFC Film Center has beautiful screens and is located at 323 Sixth Avenue at West Third Street in the West Village, right at the W. 4th St. subway station (A, C, E, B, D, F, & V subway lines).

Sita Sings the Blues is a terrific film; it won all those awards for a reason. Please tell all your New York friends — let’s pack the house!

Sometimes I have to trade information with people I’ve never met. I know who they are, in the sense that I know what they’re working on, why we’re collaborating, etc. But I wouldn’t recognize their face or voice. Occasionally, the information we’re trading is sensitive — news of a software security vulnerability, for example, or the address of a mutual collaborator who cares about privacy.

How can you know that the person you’re talking to is the person you think they are? You can send them encrypted messages, but you still need to verify that you’re using the right encryption key. So you call them up or find them in an online chat room and verify the key fingerprint… but how do you know that the person you’re verifying with is the right person and not some man-in-the-middle attacker?

It may all sound a bit spy-vs-spy, but anyone who works on widely-used open source software can find themselves in this situation. It’s happened to me more than once.

The solution I use is something I call the Instant Answer Protocol:

Alice wants to verify that the person she’s talking to in real-time is Bob. She digs up a few random facts about Bob on the Internet, then phones or chats online with Bob and asks him about those facts. In the absence of a very dedicated impersonator, only Bob would be able to instantaneously answer unexpected questions about himself, so his identity would then be established to a high degree of certainty. After that, she can voice-verify Bob’s encryption key fingerprint or do whatever else she needs to do.

Such a check still wouldn’t protect against a determined and well-funded imposter… but then, how can you even be sure it’s you reading this?

I’ve used the protocol several times. I’ve never caught my interlocutor trying to fake someone else’s identity, but it gave me peace of mind anyway.

Obviously, this is a very old protocol. It predates the Internet, and probably literacy itself. Does anyone know if this protocol already has a name in cryptography circles?