2019

Thanks to user lamayonnaise in this Reddit, I was able to solve the problem described below, which I encountered when upgrading a Debian GNU/Linux box from old stable (9.x, a.k.a. “stretch”) to new stable (10.0, a.k.a. “buster”). I’ve also seen this when upgrading from ‘stable’ to ‘testing’ — presumably the solution below would work there too.

Here’s what the problem looks like — full transcript, out of consideration for search engine indexes:

root# apt-get dist-upgrade
Reading package lists... Done
Building dependency tree       
Reading state information... Done
You might want to run 'apt --fix-broken install' to correct these.
The following packages have unmet dependencies:
 guile-2.2-libs : Depends: libtinfo6 (>= 6) but it is not installed
 libedit2 : Depends: libtinfo6 (>= 6) but it is not installed
 libllvm7 : Depends: libtinfo6 (>= 6) but it is not installed
 libncurses6 : Depends: libtinfo6 (= 6.1+20181013-2) but it is not installed
 libreadline7 : Depends: libtinfo6 (>= 6) but it is not installed
E: Unmet dependencies. Try 'apt --fix-broken install' with no packages (or specify a solution).
root# 

Hmmm, that doesn’t look good. I tried following the advice given there, but it didn’t work:

root# apt --fix-broken install
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Correcting dependencies... Done
The following packages were automatically installed and are no longer required:
  guile-2.2-libs libncurses6 libpython3.7-minimal libsasl2-modules libzstd1
  mariadb-common python3.7-minimal
Use 'sudo apt autoremove' to remove them.
The following additional packages will be installed:
  libtinfo6
The following NEW packages will be installed:
  libtinfo6
0 upgraded, 1 newly installed, 0 to remove and 1326 not upgraded.
47 not fully installed or removed.
Need to get 0 B/325 kB of archives.
After this operation, 534 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
apt-listchanges: Can't set locale; make sure $LC_* and $LANG are correct!
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = (unset),
        LC_ALL = (unset),
        LANG = "en_US.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
locale: Cannot set LC_ALL to default locale: No such file or directory
Setting up libpam0g:amd64 (1.3.1-5) ...
locale: Cannot set LC_ALL to default locale: No such file or directory
Checking for services that may need to be restarted...awk: error while loading shared libraries: libtinfo.so.6: cannot open shared object file: No such file or directory
Checking init scripts...
awk: error while loading shared libraries: libtinfo.so.6: cannot open shared object file: No such file or directory
dpkg: error processing package libpam0g:amd64 (--configure):
 subprocess installed post-installation script returned error exit status 127
Errors were encountered while processing:
 libpam0g:amd64
E: Sub-process /usr/bin/dpkg returned an error code (1)
root# 

Okay, hmmm, what about trying the same but with apt-get instead of apt? Let’s see:

root# apt-get --fix-broken install
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Correcting dependencies... Done
The following packages were automatically installed and are no longer required:
  guile-2.2-libs libncurses6 libpython3.7-minimal libsasl2-modules libzstd1
  mariadb-common python3.7-minimal
Use 'sudo apt autoremove' to remove them.
The following additional packages will be installed:
  libtinfo6
The following NEW packages will be installed:
  libtinfo6
0 upgraded, 1 newly installed, 0 to remove and 1326 not upgraded.
47 not fully installed or removed.
Need to get 0 B/325 kB of archives.
After this operation, 534 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
apt-listchanges: Can't set locale; make sure $LC_* and $LANG are correct!
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = (unset),
        LC_ALL = (unset),
        LANG = "en_US.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
locale: Cannot set LC_ALL to default locale: No such file or directory
Setting up libpam0g:amd64 (1.3.1-5) ...
locale: Cannot set LC_ALL to default locale: No such file or directory
Checking for services that may need to be restarted...awk: error while loading shared libraries: libtinfo.so.6: cannot open shared object file: No such file or directory
Checking init scripts...
awk: error while loading shared libraries: libtinfo.so.6: cannot open shared object file: No such file or directory
dpkg: error processing package libpam0g:amd64 (--configure):
 subprocess installed post-installation script returned error exit status 127
Errors were encountered while processing:
 libpam0g:amd64
E: Sub-process /usr/bin/dpkg returned an error code (1)

Nope.

All right, then. Let’s do it manually:

I’m not sure it was necessary, but at this point I ensured the locale by checking that the uncommented line “en_US.UTF-8 UTF-8” was present in /etc/locale.gen, running the command locale-gen as root, logging out and logging back in, and confirming the locale with locale -a.

Again, that locale dance may not have been necessary. What was necessary were the next steps:

Visit the Debian package pages for libtinfo6 and libpam0g, download the amd64 versions (using the sha256sum command to check the downloaded files against the SHA256 fingerprint listed at the bottoms of the Debian package pages), then install them manually:

root# dpkg -i libtinfo6_6.1+20181013-2_amd64.deb
root# dpkg -i libpam0g_1.3.1-5_amd64.deb

Those commands succeeded, and I confirmed that the packages were now installed:

root# apt-get install libtinfo6
Reading package lists... Done
Building dependency tree       
Reading state information... Done
libtinfo6 is already the newest version (6.1+20181013-2).
libtinfo6 set to manually installed.
0 upgraded, 0 newly installed, 0 to remove and 1325 not upgraded.
root# apt-get install libpam0g
Reading package lists... Done
Building dependency tree       
Reading state information... Done
libpam0g is already the newest version (1.3.1-5).
0 upgraded, 0 newly installed, 0 to remove and 1325 not upgraded.
root# 

Now the box was in working order again, and I could finish the dist-upgrade:

root# apt-get dist-upgrade
[...zillions of lines of package names omitted...]
1325 upgraded, 390 newly installed, 19 to remove and 0 not upgraded.
Need to get 41.7 MB/1,155 MB of archives.
After this operation, 1,054 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
[...zillions of lines of success omitted...]

Portrait of Elizabeth Warren.

I’ve been “All In For Warren” for a while now. I expect a lot more people to join us after tonight’s debate :-), but just in case you’re still on the fence, here are four brief arguments Why Warren:

  • She’s making the other Democratic candidates better. She’s offering so much vision that the others are picking it up. The longer she stays in the race, the better the eventual nominee will be. (I think it will be her anyway, so this item is more of an insurance-policy argument.)
  • She has the right enemies. Seriously, ask yourself: can you name one enemy of Joe Biden’s? No, you can’t. When Joe Biden walks into a room, his goal is for everyone in that room to like him. That is not what we need in our next President. Elizabeth Warren has the enemies you’d hope she would have.
  • She understands what is needed, and she’s proposing to actually do it. Most candidates understand what is needed, but they don’t dare propose to actually do it, because they can’t afford to scare off the big-dollar donors. Elizabeth Warren decided not to pursue big-dollar donors from the beginning. That’s freed her up to offer up a spot-on diagnosis of how scaled-up capitalism has captured the state and made its values the state’s values, and she’s saying what needs to be done about that. She doesn’t mind offending the people who pushed us into unsustainable inequalities of wealth, power, and dignity.
  • If she’s campaigning for President, then we’ll probably have a better Senate too. The best presidential campaigns have coattails. Elizabeth Warren’s will be particularly long, because she’s offering so much for other candidates to grab on to.

Want to help? Come on in, the water’s fine!

Visual demonstration of Simpson's Paradox (adapted from https://en.wikipedia.org/wiki/File:Simpson%27s_paradox_continuous.svg)

Do any news organizations have a Numeracy Editor?

For fifteen years, the New York Times had a Public Editor, whose job was to visibly uphold journalistic ethics. The Public Editor would publicly discuss errors, biases, or gaps in the paper’s coverage. (Some other news organizations continue to have a public editor position, though I think it’s not widespread.)

I’d like to propose something narrower: a Numeracy Editor. The Numeracy Editor’s job would be to help reporters and columnists use numerical and statistical reasoning well.

I’ve been pondering this idea for a while, and finally decided to write about it after reading Vatsal G. Thakkar’s excellent NYT Op-Ed Bring Back the Stick Shift a couple of weeks ago. It’s a good piece, but at one point it veers into unexpected non sequitur in an attempt to use statistics to support its argument:

Backup cameras, mandatory on all new cars as of last year, are intended to prevent accidents. Between 2008 and 2011, the percentage of new cars sold with backup cameras doubled, but the backup fatality rate declined by less than a third while backup injuries dropped only 8 percent.

The more you read that, the less it means. For a three-year period, the percentage of new cars sold with backup cameras doubled from whatever it was before — without knowing what it was before, this doesn’t tell us anything: the result of doubling a very miniscule percentage would still be a miniscule percentage, for example. Meanwhile, during that same three-year period, fatalities due to backups declined by some amount (less than a third) from whatever the rate was before — again, we don’t know. So does that decline represent a greater decline in backup fatalities than should be expected from whatever percentage of cars on the road newly have backup cameras? Or a smaller decline? There is no way to say. Also, we don’t know what percentage of cars driving on the road are new cars, which is highly relevant here.

If the author was trying say to that fatalities should have declined more, this paragraph does not support that case, but it doesn’t support any other case either. It throws some statistics into the air, as if to see how the wind catches them, but they don’t connect to each other and they have no bearing on the question at hand. As my friend Tom put it, it’s just a “number casserole”.

I certainly don’t mean to pick on on Thakkar — again, I liked the piece — or on the New York Times. This sort of thing happens in many publications; you can see it all the time, in the regular reporting just as much as in opinion editorials.

But given that this was the New York Times Op-Ed page — a forum that presumably takes quality control and editorial standards seriously — it’s worth asking: how did such a problematic paragraph make it through the filters? I think the answer is that there is no editor whose reputation are self-respect are on the line when numerical clunkers slip through. A few grammatical or spelling errors and someone’s job is in danger, but even glaring errors of statistical reasoning are currently costless.

I get that journalists and their editors tend to have backgrounds in language, political science, history, and other fields that don’t emphasize math. And that’s fine: this isn’t an “everyone should learn more math” argument. There are only a finite number of days in anyone’s life, there isn’t time to learn everything, and people make the choices they make for reasons. That’s exactly why a Numeracy Editor is needed: it would be her job to own this problem, and along the way help journalists learn the math they need. The writers would start to be more careful just knowing that someone is watching. A Numeracy Editor would have caught the problem in that Op-Ed right away, and once spotted, it’s easy to explain; the conversation with the author can take place before publication, as with any other kind of editing. Many errors of numerical or statistical reasoning are easy to understand once they’re pointed out (although there are also subtler cases, such as Simpson’s Paradox, that occur in real-life, policy-relevant situations and need to be watched for).

Unlike Public Editor, Numeracy Editor need not be a public-facing role. The main point is to help writers and other editors use math appropriately and to prevent mistakes. If the editor also wants to conduct a public discussion about using numbers and graphs in journalism, that would be a great public service too, but it’s a bonus. The role could do a lot of good purely behind the scenes.

Numeracy Editor should be an easier position to hire for than the broader role of Public Editor has been, because it doesn’t require nearly as much journalistic experience (the Numeracy Editor isn’t making hard judgement calls about how much anonymous sourcing is acceptable in a story, for example) and because the advice it provides would be less controversial.

Anyway, I don’t run a newspaper; all I have is this blog. I’d love to hear from anyone who works in or near journalism what they think of this idea.

(You can respond in a comment, or in this Twitter thread, or in this Identi.ca thread.)