2008

Jessica Ferris

I just got this email from my friend Jessica Ferris, and was so delighted that I wanted to share it with everyone:

It’s September 19!

I’ll be 31, which for some reason seems a whole lot more significant than 30. Like, I’m not just in my thirties as a little novelty, I’m actually keeping going straight through.

I won’t be having a birthday party this year — I prefer to keep things quiet for a whole bunch of reasons.

But I’m using my Birthday Girl Immunity to do something that a few of you, my friends, might find to be of questionable taste.

Here’s what it is:

If I’m someone whose birthday party you’d like to attend (assuming I were in your town), you could take those couple hours you’d spend at my birthday party, and go spend them phone-banking for the Obama campaign.

Or maybe you could take the five dollars you’d spend on chips and dip to bring to my hypothetical potluck celebration, and give it to the Obama campaign.

Or if you’re someone who might just click on my e-vite and maaaaybe consider coming to my party, that is, if there were nothing better to do that day, you could just click here and maaaaybe consider supporting the Obama campaign, that is, if you find there’s no one better running for president.

Because honestly, the biggest thing I want for my birthday is to live in a country which isn’t run by McCain/Palin.

Sincerely,
The Birthday Girl

http://my.barackobama.com/page/content/actioncenter

I’m going to give Jessica what she asked for on her birthday. I hope you will too.

We’re looking for a web designer and a systems/db person available soon, for paid work on a project that would last through Election Day. (If it goes well, the project might continue after Nov. 4th, but think of election day as the deadline.)

It hasn’t been green-lighted yet; it partly depends on how quickly we can find people.

Brief summary: it’s about helping college students vote. They often don’t because they can’t figure out where they’re supposed to vote, what the residency requirements are, etc. But it turns out most of them can vote, if they just get the right information. The idea is to have a crowd-sourced, curated site that gathers this information and makes it easy both for students to vote and to pass on information about how to vote to other students they know. Good user interface and comprehensibility will be key.

The funders are aware they had this idea somewhat late in the game, but they’re interested in trying to make it happen if we can put together a good team quickly. This is a non-partisan get-out-the-vote effort; the funding is not coming from any of the campaigns.

By the way, we are aware of NewVotersProject.org, DeclareYourself.com, and the local information at WhyTuesday.com. We’ll use those and similar resources, and try not to duplicate efforts. Please let us know if you’re aware of others.

If interested, please send name, résumé, and rates to me at kfogel@red-bean.com. Just quote your normal corporate rates; we’ll figure out later if we need to ask for non-profit discounts.

Thanks!

-Karl Fogel

How hard would it be to make a distorted map of the United States in which all the state borders are shown, but with the states shrunk or expanded to have areas proportional to their populations? In other words, like this map from FiveThirtyEight.com

FiveThirtyEight.com map

…but with each state re-sized to visually represent its electoral college weight.

Naturally, this would distort the shape of the U.S. as a whole, at least if one wanted to keep borders touching as they do now. But could it be done? And done so that the country is still basically recognizeable?

Getting the areas right could be tricky, since these aren’t rectangles… One would have to be careful to avoid the sorts of accidental data distortion sins Edward Tufte warns of, and which, in a sense, undistorted electoral maps like the one above are inherently guilty of. On the other hand, it doesn’t have to be precise, just close enough for government work (as the saying goes). Any tool that turns the black lines into grabbable, moveable boundaries, and counts the pixels inside to approximate the area of each state, would be fine. Of course, as you’re moving one state’s boundary, that of its neighboring state is changing too, so the interactions get complicated, hmm.

I have no idea if such tools exist anyway, though. Dear lazyweb…

This is for open-source techie friends of mine who read this blog:

The Mozilla Corporation is looking to hire a build engineer. If you’re interested, and you know me personally, please reply privately to me, and I’ll give them your name along with my recommendation (I assume if you and I know each other, then you can gauge whether or not I would recommend you). I’ve heard Mozilla is a great place to work, for what it’s worth.

If you don’t know me, but you’re still interested, I’m happy to pass your name along too, but would do so with a disclaimer that I don’t know you and therefore can’t make any recommendation.

Here’s what they said about the position:

Ideally the ideal candidate would look something like this: Someone with strong Perl and Python skills (as long as they’re strong at least one of the two scripting languages, we’re fine with it). In terms of revision control systems, CVS and Mercurial would be best (we’re fine with someone who has only experience in CVS). Experience with Tinderbox and Buildbot is a bonus. We have a pretty large build infrastructure of about 230 machines (Linux, Mac, and Windows)

The following links might be helpful as well:

http://developer.mozilla.org/en/Build_Documentation

http://developer.mozilla.org/en/How_Mozilla’s_build_system_works

That’s it. Let me know if you’re interested!

This post isn’t really for a general audience; it’s for friends and acquaintances who’ve had trouble keeping up with my peregrinations (and who have let me know this in increasing numbers over the last month 🙂 ).

I moved to from San Francisco New York in January 2008. Sort of. What really happened was that I tried to move to New York, but failed, because I’d planned it very stupidly.

Note to kids thinking of trying this at home: Don’t put all your stuff, including your beloved grand piano, into storage, then drive across the country to New York City expecting to be able to buy an apartment quickly, not if you work as a consultant (which makes mortgage lenders nervous), and especially not if it’s immediately after a huge sub-prime mortgage crisis that’s been all over the news (which makes mortgage lenders especially nervous). And if you do try the above, you might find it wise to listen to the friends who tell you that the New York real estate market is not like other markets, and that you should plan on everything taking longer.

Anyway, after a few months of beating my head against the wall in New York (there were some other stupidities along the way, which in the interests of preserving what little dignity I have left I won’t go into here), I finally gave up and decided to move back to Chicago, a city I’d lived in for a long time, where many of my friends live, where the best choir ever is, and where apartments are much cheaper. “I can always do copyright reform work from Chicago,” I told myself, “I mean, it’s really all online anyway.”

So in April I went back to Chicago. I had a wonderful month there, catching up with family and friends, but… I started realizing from pretty soon after I got there that I wasn’t going to stay. I won’t go into all the reasons why — this post is about logistics, not deep philosophical questions about the importance of place[1]. Suffice it to say that I missed the East Coast (where I’d grown up), and, despite having been thoroughly defeated by New York once, I still more felt at home there than anywhere else.

In May I went back to New York City, armed with a more realistic set of expectations and a much greater willingness to compromise in the apartment department. This time things went better: I found a place pretty quickly, and am now in contract to buy it. It’s in upper Manhattan, utterly lovely, in a musician-friendly building in a quiet neighborhood, gets lots of sunlight because it’s south-facing, and I hope someday to cook you dinner there.

If, that is, I ever actually close on the place. The process of buying a co-op apartment turns out to be like having a whole second job, except you pay them instead of them paying you. The reason it’s so hard is that in addition to the seller, the buyer, and the mortgage lender, and all their lawyers, there’s one more party involved: the co-op board, which has to approve any new residents, and wants everything to be thoroughly documented.

So take yesterday, for example. I got a call from the co-op board’s representative, saying that they needed some more information from me. It turned out to be the sort of thing that requires a signature, and involved some hand-corrections to existing pages in the application (an application that was almost an inch thick when I delivered it, if you count the supporting documentation like bank statements, tax forms, etc — I’m not kidding, I measured it). So I dropped what I was doing and went over to their offices and made the corrections, signed, etc.

Something like this happens several times a week. If it’s not the co-op board, it’s the mortgage lender. I’m expecting it to continue for a while. We still don’t have a firm closing date, but we should have one soon. In the meantime, I do whatever they ask as quickly as I can, because after all I want that closing to happen yesterday. Until then, I’m living in a series of rented rooms in Brooklyn (I have to rent them for short periods of time because I never know what the end date will be, and I don’t want to pay for longer than I need).

So that’s everything: I’m in New York; I don’t have an apartment; I hope to have one soon; in the meantime I live a kind of transient life I haven’t lived for years. I wouldn’t say it’s liberating, exactly, but it is oddly relaxing to not have most of my stuff around. Except for the piano — that I miss like a drug addict misses, uh, drug. Whatever. You know what I mean. I hope it’ll be in its new home soon!


[1]: 2012-01-12 update: This sentiment turned out to be premature; I’m back in Chicago now.

If you’re looking for office space in New York City — as an individual or a company — consider joining forces with New Work City:

New Work City

Tony Bacigalupo, who is putting together NWC, recently wrote this:

We’re looking for an anchor tenant to join forces with. Ideally, this anchor tenant would be a growing company in an existing space that’s looking expand — and who would be interested in being part of a creative work community. If you know someone who you think would be interested, send them our way!

For what it’s worth, I’ve committed to signing up as an anchor tenant myself (assuming reasonable rent and location, but it looks like those conditions will be met). If enough people do the same, then this can actually happen.

Join us!

There’s a particularly insidious kind of comment spam nowadays, one that cannot be defeated by automated measures such as captcha. I’ve been noticing more and more of it on QuestionCopyright.org, and I’m sure other web site admins are experiencing the same wave.

I don’t know if this phenomenon has a name yet; perhaps you can tell me.

Basically, it’s comment spam written by people — real human beings, not robots — who are paid to surf the web. They scan each article as quickly as they can and then leave a “drive-by” comment. The comment is usually on-topic, more or less, but of extremely low quality, and contains a commercial link back to whoever’s paying that person to surf. This is now a business model: there are intermediaries who hook up people willing to leave links for money with companies looking to boost their search engine rankings. The intermediary charges a flat rate per comment (US $0.20 seems to be the going rate), keeping a percentage and paying the rest to the surfer. The customers buy in bulk, of course, and the surfers are paid in bulk; the intermediary’s business model is based on economies of scale and smoothing things out.

Such comments now comprise the vast majority of new comments on QuestionCopyright.org. That is, we still get genuine comments at the same rate we did before — in fact, that rate may even be slowly increasing — but it’s dwarfed by the number of paid-link spams we’re getting now. It’s a total deluge. To a first approximation, all our new comments are spams, and then if you look closely there are a few hams (good comments) scattered randomly in the flood.

No Turing Test can possibly solve this, because actual humans are involved. Making your captcha puzzles harder won’t help: all it does is drive up the price-per-comment a bit for the buyers, while also making it harder for legitimate commenters to leave their remarks. The only way to detect such comments is to have human editors reading and making judgements.

This has profound implications for the user interfaces by which editors filter out spam. But before we get to that, let me show you how insidious the problem is. Here are some examples, all taken recently from the same article on QuestionCopyright.org. (Note that this is just the tip of the iceberg — the same thing is happening on all the articles on the site.)

This article is very
Submitted by Anonymous on Thu, 2008-06-19 12:53.

This article is very useful.I read it carefully and I agree with the main idea of the author.The opportunities that the internet offers to make our life and work simpler should be taken advantage of. It is absolutely necessary in the field of copyright.

Internet marketing

That one was a pretty easy call. Even the link text (“Internet marketing”) practically screams spam.

But how about this next one?

Extremely Informative
Submitted by Sedona on Mon, 2008-06-09 11:32

Thank you for this extremely informative article. I agree I don’t feel its about creativity but the publishing entities sustaining a mood of “go cautiously” and keep a big legal war chest.

Thank you,
Sedona

A little harder to tell, that time. The comment doesn’t exactly say anything, but it’s not immediately clear that it’s nonsense — you have to read it somewhat carefully to figure that out. Sedona is actually a registered user of the site (note that her name is highlighted in the header line), and the link text at the bottom is just the name “Sedona” too. But the link points to www.sedona-spiritual-vacations.com.

It turned out that this comment was also pretty clearly spam: “Sedona” left similar comments elsewhere on the site, always with the same commercial link at the bottom. None of her comments said anything much, let alone responded in a meaningful way to the content of the article.

But it gets worse. Some link spam comments actually say something. The paid surfer reads the article, apparently enjoys it and has some kind of non-trivial thought about it, and leaves a halfway decent (or sometimes even better than that) comment — but still with a paid link. Like this:

Patent and CopyRight
Submitted by Anonymous on Thu, 2008-05-22 15:41

Apart from the middle man and distributors its probably Lawyers who benefit the most from these **laws**.

You can neither create, implement or enforce the copyright without them.

One only has to look at the case of RIM ( Blackberry ) in Canada who was forced to pay 600 Million to what was in essence a group of 30+ lawyers who pro bono backed a patent that was actually overthrown in court ( but not before RIM was told to pay ).

This is not an isolated example where a claim jumper has been given a ridiculous patent by the patent office ( who frequently revokes them after they are challenged )

My uncle, who is somewhat of an economist, likes to say that lawyers are one of the very few professional groups who do not contribute to the gross national product of a country.

I am not against lawyers, they are a useful bunch. But like many government employees ( which they are not ) when allowed, too many of them actively attempt to overvalue their services within the scope or measurement of a country’s forward economic progress.

Signed, A Poker Lover

Not a great comment, I admit, but not completely pointless either. If there had been no commercial link, nothing else about it would have raised my suspicions. It was followed up to some days later by this one:

Authors & Artists
Submitted by Anonymous on Sun, 2008-06-08 14:14

I agree very much with the previous poster. How many lawsuits are brought up about copyright a year. The millions of dollars which are thrown at law companies around the world to up hold a ‘companies’ intellectual rights is pathetic to say the least.

The only person who truely has a right to claim stack is the writer, producer artist. I qould pay my way to anyone who does work for me. If they provide a service like my electric or water company I pay them. But what do these middle men companies do? They look out for themselve and only themselve. It is time the power was taken away from the big corperations and given back to the people who really deserve to be paid. Those who created it in the first place.

Regards,

David of PC Sport Live

Wow, the link spammers are following up to each other’s posts! Actually, it’s possible that the “David” of the second post is the same person who wrote the first post, even though he (or she?) portrays himself as being a different person. I’ve noticed they do that a lot. You can often tell, from a combination of the writing style and the link destination, that supposedly distinct commenters are really the same person.

The next day, someone followed up to “David”‘s comment:

Copyright laws
Submitted by Anonymous on Mon, 2008-06-09 16:34

Hello Everyone,

Today, I note that RedHat Founder Bob Young also weighed in on the copyright issue :

A new open source software group has added its voice to the opposition against the Conservative government’s ( Canada )impending copyright reform bill. Lulu CEO Bob Young likens the legislation to banning screwdrivers because they could be used by burglars.

……

Young said the proposed bill will cater too heavily to the content industry and not to the engineers and software developers that are going to be most severely impacted by the new laws. The proposed anti-circumvention legislation, he said, is similar to making the use and ownership of screw-drivers and pliers illegal because they can be used to commit crimes such as burglary.

Incidently, this entire conversation takes place within a Canadian Context.

Young further says,

“The copyright philosophy behind the U.S. DMCA is that it’s illegal to do what software engineers do every day of the week and what they’ll have to continue to do in order to build better technology for all companies,” Bob Young, spokesperson for the Canadian Software Innovation Alliance (CSIA) and a former founder and CEO at Red Hat Inc., said. “The biggest concern is we’re going to have law substitute for good technology. We’re crafting these laws without having anyone from the technology industry engaged in the process.”

The complete article is here itworldCanada.com

An interested internet marketing guy

Hmmm. It’s clumsily written, and consists mostly of quotes from someone else, but there’s real content there: that quote about the screwdriver is terrific. I have to admit that the comment actually contributes something to the site. I think it probably lies somewhere between typical paid link spam and a real comment: it might be from a person who is actually associated in some permanent way with the business being linked to, and who just makes a habit of always signing his posts with a link back to his business. Or it might be the usual kind of paid link spam. I frankly can’t tell.

I could go on and on; the above is a tiny fraction of what we’ve been getting on the site. There are obvious spams, semi-spams, maybe-not-spams, clearly-not-spams, and every gradation in between. I sometimes have to exercise real judgement when doing comment moderation; it’s not always clear what’s spam and what’s not.

In fact, it is no longer possible to divide comments into “spam” and “not spam” in an unambiguous, binary way. A given comment can now fall into both categories. Paid-link spammers are humans, and may have genuine reactions to the articles they read, even though most of the time they’re reading primarily to get just enough of a sense of the topic to be able to write a drive-by comment. Editors will just have to deal with comments on a case-by-case basis. It may be possible to apply some automatable heuristics, but they will always be imperfect, because the problem of categorization has become arbitrarily complex.

This phenomenon has implications for both site editors and software designers. For the former:

  • Site editors need to get it through their heads that they’re editors. That is, they’re responsible for quality of the site, and that includes the comments. Whether a comment has spam-like commercial links in it or not is not the question. The real question is, Does that comment contribute anything useful to the site? It’s true that there is a strong correlation between commercial links and poor quality, but it’s the poor quality that’s the problem. If a comment is good but has commercial links, you don’t have to throw out the baby with the bathwater — just replace the links with some text like “[commercial link deleted]”. (It’s important to leave some visible sign that the comment has been edited, otherwise the commenter is effectively being misquoted by your site.)

  • Site editors need to educate their readers about the situation. Most readers don’t run web sites with open comment forums, and most also have no idea that this whole paid-link comment spam problem even exists (partly because the editors have been protecting them from it). When you accidentally delete one of their comments, or they see comments disappearing, they’ll wonder why; you should have an explanation at the ready, and should refer to it often. (I’m about to update QuestionCopyright.org’s editorial policy to reflect this, and then will link more prominently to that policy from various places on the site.)

The implications for software designers (particularly of content-management systems such as Drupal, which is what we’re running on QuestionCopyright.org) are equally important:

  • Stop thinking about spam-filtering as the problem of filtering a few spams out of the stream of hams. It’s the other way around: the spams are the stream, and the problem is to pick out the rare hams. Please design interfaces accordingly!

    If it takes two clicks plus a request/response loop with the server just to see the full body of a new comment, and then another click-plus-loop to mark it as spam or ham, and then another click to confirm, then site editors will waste the majority of their time clicking and waiting. If I have to visit comments by visiting the articles to which they are attached, then the interface is mis-tuned: the operative unit should the comment, not the article. Since most new comments are obviously spam, I don’t need to see the original article to mark them as such. They are the common case, and the interface should be optimized toward them.

    The ideal interface would present the editor with a single page showing all as-yet-unmoderated comments, with their full texts (or arrange them in groups of 20, or whatever, if that would make the page too long). They would all be presumed spam, and the editor’s job would simply be to glide down them marking the hams. Each comment would have a link allowing the editor to see the original article, in case that context is necessary (though it rarely is). Each comment should have a flag next to it indicating whether or not there are any links in the comment at all — if a comment has no links, it is much less likely to be paid spam, and therefore much more likely to be high quality. This flag would enable the editor to set her expectations, which is a great help when faced with hundreds of comments.

  • Don’t make the parent->child threading relationship between comments fatal. That is, if comment B is a reply to comment A, but A is later classified as spam and deleted, B should not be deleted along with it. Users often click “reply” just as a way of making a new comment; the fact that the comment they’re replying to is spam has no bearing on the quality of the new comment. B may not be spam, even if A is. (The version of Drupal we’re currently running at QuestionCopyright.org gets this wrong, unfortunately, but newer versions may have fixed it.)

  • Have a setting that allows editors to simply prohibit links. It’s a policy decision that each site must make on its own (some links are useful, as we saw in one of the examples above), but the option should be there.

Any other ideas, folks? It’s a whole new world out there…

It’s over.

I don’t mean the Democratic Primary, I mean the general election. The whole thing. Barack Obama is going to walk over John McCain like a piece of gum on the sidewalk.

John McCain

Why?

Watching John McCain speak tonight, I was reminded of a principle my Go teachers often mentioned: don’t make a move to which your opponent’s best response would be a move he wanted to make anyway. I often did that: I’d put a stone down on the board, and my opponent’s response would simultaneously counter what I had done and serve some other purpose useful to my opponent. Those opponents who were trying to teach me would ask “Why did you make me stronger? I wanted to go there anyway.”

John McCain is making this mistake with Barack Obama. He’s handing Obama exactly the debating points Obama wants. He accuses Obama of thinking that government can provide solutions, but people remember Katrina, and Obama wants a chance to say specifically what he thinks government can do. He accuses Obama of being willing to engage in diplomacy with rogue regimes, but people remember Iraq, and Obama has a whole foreign policy debate he’d just love to get into with John McCain. He accuses Obama of turning to the past for answers; but people look back fondly on the past, because the present is so tarnished. Obama is only too happy to remind people how much better this country used to be run.

When you get right down to it, Obama just understands how this game is played, and McCain apparently does not. Yes, it helps that Obama is smarter, more charismatic, and genuinely has better policies. (That golden baritone voice and his so-ready-for-this wife Jacqueline Michelle don’t hurt either.) But it also helps that John McCain doesn’t seem to realize how easy it is going to be for Obama to turn McCain’s talking points into a real debate, and win it.

Barack Obama

Of course, due to the electoral college mess, it’s hard to predict how close things will be in the fall. But something would have to go seriously wrong for Obama not to pull this one off by a wide margin in the popular vote. John McCain is going to give Obama opportunity after opportunity to show how different he really is, and it’s only going to persuade more people to vote for Obama.

If I were running John McCain’s campaign, I’d start reading some Go books.

My favorite political blog, TalkingPointsMemo, is usually insightful on policy, and even more so on politics. But what really distinguishes it from the competition is the quality of its writing, even in short, ephemeral posts. For example, this is from an item posted today:

“Many of our foreign policy thinkers seem to be developing the kind of character damage suffered by children who can buy the best toy every time their parents go to the mall — the inability to distinguish between necessities, simple wants and the mere desire for kicks which is born of pervasive moral boredom.”

Read the whole post if you have time — it’s excellent, especially given its brevity.

I was never a member of the Boy Scouts myself, so my understanding of what they’re about is limited to what’s seeped in from popular culture, but this looks like a winning idea:

Friday May 9, 2008

The Boy Scouts have joined the Open Source Community.

The Boy Scouts of America National Council in Irving, TX has announced the release of their Open Source Initiative. The OSI Project represents a significant commitment by the Boy Scouts of America to the Open Source Community.

This project represents a “complete embrace of Open Source by the Boy Scouts”, says Greg Edwards, OSI Project Manager. Through the OSS Website (http://opensource.scouting.org) the Boy Scouts are not only committed to becoming users of Open Source Software, but teachers, producers, and advocates as well.

(See Greg Edwards’ open letter for more.)

If this means actual scouts are going to be encouraged to get involved in open source projects — say, it will be considered an official scouting activity that you can (I guess) earn merit badges for — then it seems like a great chance for a lot of kids to experience the open source process.

(Er, I guess that should say “boys”, not “kids”. Why aren’t the Girl Scouts doing the same thing? Why weren’t they doing it first, actually?)

Of course, only a small percentage of scouts will flourish in open source, in the sense of having the temperament and discipline to make useful contributions to the projects they participate in. But that’s okay: that ratio is the norm in open source projects. There’s no reason to expect more or less from Boy Scouts. What’s more important is that all the scouts who participate will be exposed to the cultural norms of the open source community: sharing, respectful technical discussions, taking the time to express oneself clearly in writing, fixing things instead of complaining that they’re broken, etc. For every Boy Scout who gets involved in open source, or who hears his friends talking about their projects, that’s one more person who understands what open source is all about. (I’m using “open source” synonymously with “free software” here.)

I’ll close with this beautiful story from Jim Blandy:

Back in 1993, I was working for the Free Software Foundation, and we were beta-testing version 19 of GNU Emacs. We’d make a beta release every week or so, and people would try it out and send us bug reports. There was this one guy whom none of us had met in person but who did great work: his bug reports were always clear and led us straight to the problem, and when he provided a fix himself, it was almost always right. He was top-notch.

Now, before the FSF can use code written by someone else, we have them do some legal paperwork to assign their copyright interest to that code to the FSF. Just taking code from complete strangers and dropping it in is a recipe for legal disaster.

So I emailed the guy the forms, saying, “Here’s some paperwork we need, here’s what it means, you sign this one, have your employer sign that one, and then we can start putting in your fixes. Thanks very much.”

He sent me back a message saying, “I don’t have an employer.”

So I said, “Okay, that’s fine, just have your university sign it and send it back.”

After a bit, he wrote me back again, and said, “Well, actually… I’m thirteen years old and I live with my parents.”

That is: on the Internet, nobody knows you’re a Boy Scout.