OpenCores is a nice idea. Open sourced core designs that can be contributed to by anyone. Nice idea. Except that you MUST sign up before ever seeing a single scrap of source code, and that account creation is subject to human approval.

What the fuck?!

These people are the embodiment of failing to understand the point of open source. "Yeah, this is awesome, let's put stuff online for free, but only if people give us their name, company, affiliation, address, phone number and fill in 5 surveys!" Whose brilliant idea was that, for Pete's sake. I'm willing to bet their "community" is a tenth of what they were hoping for, and that it's almost entirely propped up by the idiotic company behind it.

Oh well, another neat idea down the drain. Nice try.

Ever get cold-called on your phone by a Ghanan 419 scammer? I just did.

What the fuck. Seriously.

I had a very brief but very interesting talk with Prof. Margaret Burnett.  She does research on gender and programming. at Oregon State University, but was in town for the International Conference on Software Engineering.  She said that many studies have shown that women are — in general — more risk averse than men are.  (I’ve also commented on this.)  She said that her research found that risk-averse people (most women and some men) are less likely to tinker, to explore, to try out novel features in both tools and languages when programming.

I extrapolate that this means that risk-seeking people (most men and some women) were more likely to have better command of tools, and this ties into something that I’ve been voicing frustration with for some time — there is no instruction on how to use tools in the CS curriculum — but I had never seen it as a gender-bias issue before.  I can see how a male universe would think there was no need to explain how to use tools because the figured that the guys would just figure it out on their own.  And the most guys might — but most of the women and some of the men might not figure out how to use tools on their own.

In particular, there is no instruction on how to use the debugger: not on what features are available, not on when you should use a debugger vs. not, and none on good debugging strategy.  (I’ve commented on that here.)  Some of using the debugger is art, true, but there are teachable strategies - practically algorithms — for how to use the debugger to achieve specific ends.   (For example, I wrote up how to use the debugger to localize the causes of hangs.)

Full of excitement from Prof. Burnett’s revelations, I went to dinner with a bunch of people connected to the research lab I did my MS research in.  All men, of course.  I related how Prof. Burnett said that women didn’t tinker, and how this obviously implied to me that CS departments should give some instruction on how to use tools.  The guys had a different response: “The departments should teach the women how to tinker.”

That was an unsatisfying response to me, but it took me a while to figure out why.  It suggests that the risk-averse pool doesn’t know how to tinker, while in my risk-averse model, it is not appropriate to tinker: one shouldn’t goof off fiddling with stuff that has a risk of not being useful when there is work to do!

(It has been emotionally very difficult for me to write this blog post today.  I think it is important and worthwhile, but I have a little risk-averse agent in my head screaming, screaming at me that I shouldn’t be wasting my time on this: I should be applying for jobs, looking for an immigration lawyer, doing laundry, or working on improving the performance of my maps code.  In other words, writing this post is risky behaviour: it takes time for no immediate payoff, and only a low chance of a future payoff.  It might also be controversial enough that it upsets people.  Doing laundry, however, is a low-risk behaviour: I am guaranteed that it will make my life fractionally better.)

To change the risk-averse population’s behaviour, you would have to change their entire model of risk-reward.  I’m not sure that’s possible, but I also think that you shouldn’t want to change the attitude.  You want some people to be risk-seeking, as they are the ones who will get you the big wins.  However, they will also get you the big losses.  The risk-averse people are the ones who provide stability.

Also note that because there is such asymmetry in task completion time between above-median and below-median, you might expect that a bunch of median programmers are, in the aggregate, more productive than a group at both extremes.  (There are limits to how much faster you can get at completing a task, but there are no limits to how much slower you can get.)  It might be that risk aversion is a good thing!

There was a study I heard of second-hand (I wish I had a citation — anybody know?) that found that startups with a lot of women (I’m remembering 40%) had much MUCH higher survival rates than ones with lower proportions of women.  This makes perfect sense to me; a risk-averse population would rein in the potentially destructive tendencies of a risk-seeking population.

Thus I think it does make sense to provide academic training in how to use tools.  This should perhaps be coupled with some propaganda about how it is important to set aside some time in the future to get comfortable with tools.  (Perhaps it should be presented as risky to not spend time tinkering with tools!)

UPDATE: There’s an interesting (though all-too-brief!) article that mentions differences in the biochemical responses to risk that men and women produce.  It says that men produce adrenaline, which fun.  Women produce acetylcholine, which the article says pretty much makes them want to vomit.  That could certainly change one’s reaction to risk..

It's Towel Day. Know where your towel is, and remember Douglas.

Heading out to California for a week or so. My towel is in my travel bag. Where is yours?

I won't pretend fun things aren't still fun out of fear of looking silly
-- xkcd 219

It's easy to regret your awkward conversations but hard to regret the ones you didn't have.
-- xkcd 235

We live trapped in loops, reliving a few days over and over, and we envision only a handful of paths laid out ahead of us. We see the same things each day, we respond the same way, we think the same thoughts, each day a slight variation on the last, every moment smoothly following the gentle curves of societal norms
Pound for pound, one of the best verticals
-- xkcd 137

Grad students aren't bad people, they just make terrible life choices
-- Bart Simpson

I think for the most part, if you're really honest with yourself about what you want out of life, life gives it to you.
-- Ted Mosby, How I Met Your Mother

If you're not prepared to be wrong, you'll never come up with anything original
-- Sir Ken Robinson

Simplicity is the ultimate sophistication
-- Leonardo da Vinci

When you're with someone you love, the most mundane things suddenly become wonderful
-- Hidden Leaves by V.C. Andrews

Though this be madness, yet there is method in't.
-- Polonius, Hamlet

The lady doth protest too much, methinks.
-- Hamlet

Any sufficiently advanced technology is indistinguishable from magic
-- Arthur C. Clarke

When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong.
-- Arthur C. Clarke

Penny: "I'm a sagitarious, which probably tells you way more than you need to know."
Sheldon: "Yes, it tells us that you participate in the mass culture delusion that the sun's apparent position relative to arbitrarily defined constellations at the time of your birth somehow affects your personality."
-- Big Bang Theory

Penny: "What time is it?"
Leonard: "Almost 6:30"
Penny: "I slept all day?"
Leonard: "Oh no, it's 6:30 in the morning"
Penny: "What the hell is your problem?"
-- Big Bang Theory

There's always the possibility that alcohol and poor judgment on her part might lead to a nice romantic evening.
-- Sheldon, Big Bang Theory

100% of the shots you don't take don't go in
-- Wayne Gretzky

Animals are such agreeable friends. They ask no questions and they pass no criticism.
--1857 British novelist George Eliot

I screwed up
-- Barack Obama

The truth is told by whoever is left standing.
-- Tom Zarek, Battlestar Galactica

I want to give my kids just enough so that they would feel that they could do anything, but not so much that they would feel like doing nothing.
-- Warren Buffett

Computer nerds are the sex symbols for the 21st century
-- CBC, Starcraft Documentary

Thinking that other people might be better than you is what makes you Canadian
-- Steven Colbert

Man and woman are born to love, marry and beget children. Woman is so physically constituted that she is the one who bears the child. Man, on the other hand, is given the duty of being the protector of his wife and, after they are born, of his children. Therefore, he must always be the one who takes the initiative. He must be strong, he must be willing, he must be physically able to take care of his charges, He must be the aggressor.

It is, therefore, necessary that the man be taller than the woman. The psychological reason for this is that he must always give the impression of being his woman's superior, both mentally and especially physically. The physical reason, with which we are more concerned, is that if he is taller that his woman, he is better able to kiss her. He must be able to sweep her into his strong arms, and tower over her, and look down into her eyes, and cup her chin in his fingers and then, bend over her face and plant his eager, virile lips on her moist, slightly parted, inviting ones. All of this he must do with the vigor of an assertive male. And, all of these are impossible where, the woman is the taller of the two. For when the situation is reversed, the kiss becomes only a ludicrous banality. The physical mastery is gone, the male prerogative is gone, everything is gone but the fact that two lips are touching two other lips. Nothing can be more disappointing.
-- Hugh Morris, The Art of Kissing (1936)

Ted: "What you and Tony have, what I thought for a second, you and I had, what I know that Marshall and Lily have, I want that. I do. I kept waiting for it to happen and waiting for it to happen and... I guess I'm just tired of waiting."

Stella: "I know that you're tired of waiting, and you may have to wait a little while more, but she's on the way Ted, and she's getting here as fast as she can."
-- How I Met Your Mother

I added a state legislatures partisanship layer to my election map, and also modified a metric which shows kind of how liberal an area is.  For every governor, US senator, or US congressman in a district that is a Democrat, I added one.  For every legislator who is a Republican, I subtracted one.  Now, with the new data, I also add one point for each state legislative chamber that is controlled by Democrats, and subtract one for each that is controlled by Republicans.

This gives me a range of -6 to plus 6 (governor, two US senators, one US congressman, one state senate, one state lower chamber), which I can show in shades of red to blue:

Some things are not surprising: the northeast is very blue; Idaho and Utah are very red.  However, I don’t get Arkansas.  I wouldn’t have thought that it would be culturally very different from its neighbours, yet most of the state has the maximum value of +6.

Is this all due to Clinton?  Did he build a really strong Democratic Party operation in Arkansas?  Or did he throw a bunch of money towards Arkansas, for which they are still grateful?

Can anyone familiar with Arkansas shed any light on this?

UPDATE:

A reader from Arkansas explained that the Arkansas Democratic party is very entrenched and strong, but that the populace is not particularly liberal.  Essentially, people who are Democrats in Arkansas would be Republicans just about anywhere else.  (This is similar to the Liberal Party in BC, which is the most conservative of the three viable parties in BC.  The Liberal Party in BC is much more conservative than the Canadian federal Liberal party.)

Jerome Dolittle recently reported via James Fallows that he had asked thirty Harvard first-year students to redesign universities from the ground up; 29 of 30 came up with, as the author put it, “something that looked very much like Harvard, except a little farther out of town”.  He seemed to take this as evidence that Harvard students were not very creative.

This was in response to an article by Randy Pollack talking about how uncreative Chinese MBA students were: when asked to come up with a the most original idea they could for a business, e.g. a restaurant chain, five of six groups came up with the idea of — a restaurant chain!  This was given as evidence of how the Chinese educational system did not foster creativity.

Dolittle seemed to think that the Harvard students were just as uncreative as the Chinese students, but I don’t think that comparison is fair.  I think that reforming education is a much more difficult problem for first-year students than thinking of a business would be for MBA students.

In addition to the age difference and the difference in academic focus, the MBAs had probably encountered hundreds if not thousands of businesses and products in their day-to-day life.  I would bet that the majority of the Harvard freshman had intimate knowledge of exactly one university.  It’s difficult to consider how you can change something if you don’t have an idea of the number of degrees of freedom you have.

Even faculty have difficulty being creative about reforming universities.  Recently there was a New York Times opinion piece End the University as we know it that got quite a bit of buzz in my circles, despite its recommendations being, in my opinion, not very creative:

1. Restructure the curriculum to be more interdisciplinary.
2. Abolish permanent departments, replacing them with problem-focused temporary departments (e.g. Water).
3. Collaborate more across institutions.
4. Allow dissertations in forms other than things that look like scholarly books.
5. Give graduate students real-world skills.
6. Abolish tenure.

Three recommendations (#1,# 3, and #5) are goals that institutions already aspire to.  They might not do it well, but they sure talk about it a lot.

I think people haven’t done #6 because it would lead to either a rise in cost or a decline in quality (or both), not because it wasn’t an obvious thing to try.

Aside from the increased administrative overhead and lack of departmental reputations that #2 would cause, project-based learning has been tried before to some extent.  (Hampshire College is highly individual-project based; Colorado College students take one compressed course at a time.)  One could also easily argue that each graduate student is supposed to create their “department of one”.  (When I got my first MS, I took classes in library science and intellectual property law because that’s what made sense for my area of interest.)

For #4, I believe that it already is possible to do a non-book thesis, particularly in the performing arts.   One friend’s “dissertation” was a symphony.  Another friend made a movie for her anthropology thesis.  While I have a copy of my thesis printed on dead trees, the important version of the thesis is the PDF available on-line.

Even though Taylor’s ideas are not that novel, they got buzz.  I think that means that it is really hard for most people to come up with ideas on how to reform university education.

In contrast to the inexperience of the Harvard freshmen, I have been downright promiscuous with universities.  (I have eight transcripts now, for examplet.) This familiarity helped me identify — fourteen years ago — some core functions of a university institution that I thought could be disaggregated: unidirectional information transfer, interactive learning experiences, caring/paying attention, and certification. I also showed how these pieces could be rearranged by private enterprise and social media. (In later blog posts, I showed how this disaggregation/disintermediation is already happening.)

Now, it is possible that it is a coincidence that I have both unusual intimacy with universities and was able to come up with creative insights about reform.  I suppose it’s also possible that I’m just amazingly brilliant. I don’t think so.  I think that if you see lots of X, then it becomes easier to think creatively about things you could do with X.

It might be that the Harvard students were in fact just as uncreative as the Chinese MBAs.   However, I don’t think Dolittle proved that.

Update: it turns out that lots of people have done exactly what I asked for: see Instruction-level Tracing:
Framework & Applications and the OCaml debugger.  Cooool! (Thanks DanE!)

In my user studies, programmers used the debugger far less than I had expected.  Part of that could perhaps be due to poor training in how to use a debugger — it is rare to get good training in how to use a debugger.

However, I think the answer is simpler than that: it is just plain boring and tedious to use a debugger.  One guy did solve a thorny problem by stepping through the debugger, but he had to press “step over” or “step into” ninety times.

And when you are stepping, you must pay attention.  You can’t let your mind wander, or you will miss the event you are watching for.  I can’t be the only person who has done step, step, step, step, step, step, step, boom, “oh crap, where was I in the previous step?”

Omniscient debuggers are one way to make it less tedious.  Run the code until it goes boom, then back up.  Unfortunately, omniscient debuggers capture so much information that it becomes technically difficult to store/manage it all.

I suggest a compromise: store the last N contexts — enough to examine the state of variables back N levels, and to replay if desired.

I can imagine two different ways of doing this.  In the first, the user still has to press step step step; the debugger saves only the state changes between the lines that the user lands on.  In other words, if you step over the foo() method, the debugger only notes any state differences between entering and exiting the foo() method, not any state that is local to foo().  If the user steps into foo(), then it logs state changes inside foo().

In the other method, the user executes the program, and the debugger logs ALL the state changes (including in foo(), including calls to HashTable.add(), etc.).  This is probably easier on the user, but probably slower to execute and requires more storage.

You could also do something where you checkpoint the state every M steps.  Thus, if you get to the boom-spot and want to know where variable someVariable was set, but it didn’t change in the past N steps, you can

• look at all your old checkpoints
• see which two checkpoints someVariable changed between
• rewind to the earlier of the two checkpoints
• set a watchpoint on someVariable
• run until the watchpoint.

CNN released a poll that showed that how often an American Christian goes to church corresponds strongly to how much he/she supports torture.

To a liberal, this makes no sense at all.  The Golden Rule says to treat others as we would like to be treated ourselves, right?  And most people would not want to be tortured, right?  What’s the deal?

It makes sense to me in light of Haight’s findings on morality.  The people who go to church the most often are those for whom belonging to a group is most important, which ties to in-group loyalty.  I can imagine that as in-group loyalty increases, desire to protect the in-group gets fiercer.  Meanwhile, I can imagine that as in-group loyalty increases, respect for/value of people in the out-group decreases.   To put it another way, the higher your loyalty to people who look like you, I bet the less interested you are in preserving the rights of people who don’t look like you.

This is not my value structure.

Update: A friend pointed me to a study that shows that participation in religious rituals (but not how often they prayed) predicts support for suicide bombing!  This was true across cultures — Palestinian Muslims, Israeli Jews, Mexican Catholic, Indian Hindus, Russian Orthodox Christians, UK Protestants, and Indonesian Muslims.  Furthermore, in some of the interviews, they “primed” the interviewee to think about religious affiliation before asking about support for suicide bombing; that turned out to significantly increase reported support for suicide bombing!

This paper thus seems to me to pretty strongly support the CNN poll, albeit indirectly. It seems to me that it not a great leap to replace

• a suicide bomber from the in-group who kills members of the out-group

with

• a torturer from the in-group who tortures members of the out-group.

I’ve been working on LOLcat subtitles for Sita Sings The Blues, and a friend asked me what the ISO 639 language code for LOLcat was.

It turns out that the ISO 639 language code LOL does exist, and is for the central African language Mongo.  Who knew that the kittehs were African!?!?

Update: My buddy Luther points out that the African Wildcat is the ancestor of the domesticated housecat.  As he pointed out, “Duh.”

Update2: Luther also asks, “I can haz bank scam?”

A few days ago, I read something somewhere about how disappointing it was that there wasn’t more hue and cry about the recent release of the torture memos.  (Sorry, my computer died and I was somewhat off the air for a few days, so don’t have a citation.)

Initially, I wasn’t going to blog about the release of the memos, but I feel I have to go on the record.  I feel I have to say something in order to be counted, to make the “torture bad” side of the argument one voice stronger.

Initially, I wasn’t going to blog because I didn’t think any more needed to be said.  I’ve already discussed why I think torture is a bad idea.  (Never mind discussions about it being morally wrong, it’s still just flat a bad idea.)  The things that I said — that I thought it was impossible to keep our torturing a secret, impossible to only torture guilty people, and that torture gave faulty information — proved true well before the memos were released.

I was angry and saddened at my country’s cavalier treatment of our military prisoners even before the memos were released.  I had already seen routine torture at Abu Ghraib, and it was already known that we had waterboarded some prisoners.  To hear that we waterboarded one guy 183 times was appalling, but I was already appalled that we did it at all.  The difference between someone who knocks over 90 year-olds for fun and one who doesn’t is far starker than one who knocks over three seniors over versus one who knocks over 183.

I was already angry and saddened at how many of my fellow citizens thought torture was a hunky-dory method; just one more tool in the toolchest for waging war.  To hear that Peggy Noonan thought we should not have released the memos — that we should just avert our eyes and walk on by was jaw-dropping, but my jaw has been on the floor for a long time.

The only thing that has changed is my attitude towards prosecution.  I had gone along with the argument that we shouldn’t spend political capital on dragging something along that would devolve into partisan bickering.  After seeing how stunningly in denial the Right-Wing is about torture being illegal, immoral, and ineffective, I now think we have to prosecute.  If we don’t — if we continue to let a large portion of the country think that torture is okay — then we will torture again.

This is not about retribution.  This is not about vengence.  This is not even about justice.  This is about prevention.  This is about deterrence.  This is about making sure that my country never goes down that slippery slope again.

1. "You have come to the Best Web Page on the Internet. With great background music and hundreds of links to the best web pages throughout the World Wide Web, how can you go wrong?"


2. MIDI background music! (You can even change the background music and there's a great selection of 90's music!)


3. JavaScript pop-up on the homepage


4. Frames!


5. Ghetto logo


6. Link and banner exchange galore


7. Most of the links don't work anymore because the websites have been closed down


8. The instructions for adding the site to your bookmarks and setting it to your homepage is for Netscape 3.0 and Windows 95


9. "Page me by ICQ"


10. Google is not listed under "Search Engines"


11. Beakman's World in the TV section


12. The website predates Starcraft as it's not listed under "Video and Computer Games"


13. The selection of "Free 'Must-Have' Programs" is rather amusing

I have been looking at unemployment figures today.  Here’s unemployment rate by county for 2008, from the Bureau of Labor Statistics.  Pure white corresponds to a rate of 2%; pure green corresponds to a rate of 10%.

Unemployment Rate 2008

Note that the unemployment rate is dimensionless, i.e. it’s the number of people looking for work divided by the number of people in the workforce.  The BLS makes its estimates by interviewing thousands of people each week and carefully asking them questions about their employment.  If they have worked at all in the past week (even part-time), that counts as employed.  If they haven’t looked for work in four weeks, they do not count as being in the workforce.  This means that retired people, stay-at-home moms, and people who have given up do not count.  (The BLS has a good explanation of their methods.)

Here is the unemployment rate by county for 2007:

Unemployment Rate 2007

Again, I think the more interesting picture is the difference between the two years; red where unemployment has gone up, blue where unemployment has gone down.  Full red means a change of +5 percentage points or more; full blue means a change of -5 percentage points or more.

2008 unemployment rate minus 2007 unemployment rate

There’s an awful lot of red there, alas.  The unemployment rate fell in 272 counties and rose in 2767 counties.

Things worth noting about the above maps:

• There are a fair number of state boundaries visible in the 2008 minus 2007 map.  For examples, Wisconsin is particularly visible, Wyoming is visible to a lesser extent, and there is also a line visible running along the north side of Oklahoma, Arkansas, Alabama, Mississippi, and South Carolina, and a line along the east side of Alabama.  I think this means that state policies actually do matter.
• West Virginia had lower unemployment in 2008 than 2007.  I presume that was demand for coal relating to the high price of oil in most of 2008.  The Oklahoma drops in unemployment also be due to the higher value of oil; I don’t know why North Dakota did better.
• Woods County, OK, which is the bright blue spot in near the center of the country in the 2008 minus 2007 map, has a relatively small population: 120 people were unemployed in 2008 versus 261 in 2007.  Oklahoma’s economy has a large fossil fuel component.

It’s also interesting to look at the difference between 2008 and 1998:

2008 unemployment rate minus 1998 unemployment rate

Thoughts:

• Poor Michigan.
• In addition to West Virginia, the rural West had higher unemployment in 1998 than in 2008.  I presume this has to do with the very strong market for resources (trees, minerals, coal, etc.) for most of 2008.

I have added the first three maps to my elections map page.   (Note that I have data such that I can put even more maps on the page, but I worry about the UI getting too cluttered.  Thus, if you really want to see some map, let me know.)

Update: A friend of a friend pointed me at Wisconsin Business Climate Statistics.  That page points out the Wisconsin has a very lean government, low taxes, low crime,and  tax exemption for energy used in manufacturing.  Frankly, it sounds like a Republican party platform — even though Wisconsin is a very Democratic-leaning state.

The UBC programming team took 34th place at the 2009 International Collegiate Programming Contest (ICPC) World Finals in Stockholm!  W00t!!

This marks the sixth year in a row that UBC has gone to the World Finals, despite being entirely undergraduates and entirely without World Finals experience.  (We have information on past teams, but don’t know the seniority of the 2004 team.)

Congratulations, team!

I recently got historical data on presidential election results by county from Robert Vanderbei, for presidential elections 1960-2004.  While it is interesting to look at the raw data, I find it even more interesting to look at the differences between years, like the 2008 vs. 2004 map I commented on already.  This helps separate how people felt about a particular pair of politicians from how liberal/conservative they are in general.  For example, here’s the 1960 (Nixon vs. Kennedy) map, with Democratic counties in blue and Republican counties in red:

1960 -- Kennedy vs. Nixon

And here’s the 1964 (Johnson vs. Goldwater) map:

1964 -- Johnson vs. Goldwater

1964 Difference

You can see even from the 1964 map that LBJ was not very popular in the South (presumably because of his civil rights work), but the difference map below really hammers it home.  In this map, it is blue if LBJ did better than Kennedy and red if the reverse.  You can see from the difference map that the South really hated LBJ:

1964 results minus 1960 results

Another interesting thing about the 1960/1964 maps is that there is no evidence at all of “the black belt”.  Here is a map of counties which were majority black in 2000, with darker green the stronger their majority:

Majority-black counties (2000)

I have to believe that blacks would have overwhelmingly voted for LBJ — if they were able.  I think this is a pretty vivid demonstration of how thoroughly their voting rights were repressed.

1968 vs. 1960

The 1968 (McGovern-Nixon) minus 1964 map is basically an inverse of the 1964 minus 1960 map, basically because the southern antipathy towards Johnson was so strong that it skews everything.  A more interesting map is to compare Humphrey vs. Nixon to Kennedy vs. Nixon:

1968 (Humphrey-Nixon) minus 1960 (Kennedy-Nixon)

Humphrey explicitly called for the Democrats to move away from states’ rights and towards civil rights, and that apparently played well in the upper Midwest and Northeast but not as well in the Southeast or West.  You can also see a faint outline of Minnesota (where Humphrey was from) and a strong outline of Maine (where Muskie, the Democratic VP, was from).  (Maryland, where Nixon’s VP Spiro Agnew was from, is too small to see in this picture.)  You can maaaybe start to see the majority-black counties in some states, but not in Georgia.

There are some blue areas in the above map, but those probably would be red if it weren’t for George Wallace.  Wallace ran as an independent, and did extremely well in southern states.  It is unlikely that he took any votes away from Humphrey, as he was an outspoken proponent of segregation.  While third-party candidates usually struggle to get over 10% of the vote, Wallace won a number of states outright.  Here is a map of counties that he won outright:

Counties won by Wallace in 1968

1972

Nixon was re-elected in a landslide.  Not only was McGovern staunchly anti-war during the Vietnam War, he was criticized for his first choice of running mate (who he fired).  The only obvious counties on this map that voted more for McGovern than for Humphrey were in McGovern’s home state of South Dakota:

1972 (McGovern vs. Nixon) minus 1968 (Humphrey vs. Nixon)

1976

The Carter/Ford minus McGovern/Nixon map looks almost exactly the opposite, as the Watergate scandal destroyed Nixon’s and Ford’s standing.  The South also rallied to Jimmy Carter, the first post-Civil War Southerner to be elected President.

1976 (Carter-Ford) minus 1972 (McGovern-Nixon)

1980

Jimmy Carter had his own troubles: the economy was in dire shape, in large part because of the rise in gas prices because of the second oil crisis.  Carter also was not a strong leader: my memory of the time is that he suffered from what I called “Democrat’s dilemma”: being able to see all sides to all issues and thus unable to take a strong stand.  Ronald Reagan, who exuded a forceful, “can-do” attitude, was more successful than the disgraced Ford almost everywhere:

1980 (Carter vs. Reagan) minus 1976 (Carter vs. Ford)

1984

Reagan got even more popular in large swaths of the country.  Mondale could only manage to erode some of Reagan’s support in spots.

1984 (Mondale vs. Reagan) minus 1980 (Carter vs. Reagan)

Note that many of the blue counties above are areas of high Native American population.  The map below shows counties where more than 30% of the people identify as Native Americans:

Counties with more than 30% Native American

I suspect that Reagan did something to upset Native Americans, but I don’t know what that was.

1988

George H.W. Bush was able to get elected in 1988, but he was pretty uniformly less successful than Reagan.

1988 (Dukakis vs. Bush41) minus 1984 (Mondale vs. Reagan)

1992

Bush continued to do worse in 1992, again pretty much across the whole country, losing to Clinton.  Note that you can see the outline of Arkansas (home of Bill Clinton) clearly and Tennessee (home of Clinton’s VP Al Gore) somewhat.

1992 (Clinton vs. Bush41) minus 1988 (Dukakis vs. Bush41)

Ross Perot made a strong third-party run in 1992.  I’m not sure who he took more votes from.

1992 third-party votes (mosty Perot)

1996

The Republicans made some inroads in 1996 in the West — especially in Bob Dole’s native Kansas (outline visible in the center of the country) — but it wasn’t enough.  Clinton gained support in the upper Midwest, Northeast, Florida, Louisiana, and Southern Texas (which is heavily Latino).

1996 (Clinton vs. Dole) minus 1992 (Clinton vs. Bush41)

2000

Bush43 and Gore had a famously close race, but Bush43 did better than Dole almost everywhere (or Gore did worse than Clinton, depending on how you look at it).

2000 (Gore vs. Bush43) minus 1996 (Clinton vs. Dole)

2004

Bush43 strengthened his lead in the middle and southeast of the country in 2004, but lost support in some Northern and Western places:

2004 (Kerry vs. Bush43) minus 2000 (Gore vs. Bush43)

I’ve written about the 2008 vs. 2004 map already, so I won’t talk about it here.   Instead, I think it is interesting to compare the 2008 election to the 1960 election, to see how the country’s party affiliations have changed:

2008 (Obama vs. McCain) minus 1960 (Kennedy vs. Nixon)

The biggest difference is that the Southeast is much, much more Republican now (except for minority-heavy areas: the Black belt and parts of Florida).

The New England states and the upper Midwest are much more Democratic.  Native Americans voted heavily for Obama.  Most importantly, perhaps, is that the Pacific coastal areas are much, much more Democratic than they were in 1960.  (Those areas have also experienced a great deal of population growth, so this change is bigger than it looks.)  The only area that seems like it stayed sort of the same is a belt running through Mossouri, Illinois, Indiana, and Kansas.

Note: The difference maps aren’t up on my maps page yet, but they hopefully will be soon.

I’ve had a constant question running in the back of my head for a long time, “Who would you have dinner with if you could?”  (I talked about this a little in the post about Steve Wozniak.)

I have expanded it a little to which three people I would like to be at the same table with, or even just to witness.  I’ve decided that my dream dinner team is Randall Munroe of xkcd, Nate Silver of fivethirtyeight.com, and Jon Stewart of The Daily Show.

How to go about pulling that off is a trickier matter.  Step one would probably be to do something really interesting, such that I would be a desirable dinner date…

Sam graciously presented me today with a new CSS stylesheet for my Zwe blog. It really was time for a bit of a change, as the previous stylesheet was starting to show its age after many upgrades and changes to the underlying engine. Hope you like it. I certainly do.

Thanks Sam!

Recently, the California Supreme Court heard arguments in a case designed to overturn California’s Proposition 8, which overturned the judicial decision that gay and lesbian people had the right to marry.  While I didn’t watch the hearings myself, I understand that Ken Starr (the defending attorney) basically put forth the belief that a majority vote could strip rights of  minority.

People who are better than I at guessing what the outcome will be by examining the questions, tone, and body language of the justices think that they will rule against overturning Proposition 8, in part because they think that the California Domestic Partnership gives all of the same rights as marriage.  Essentially, they are fighting over a word, with Starr’s side saying that a bare majority of the citizens can take away gay and lesbian people’s right to use the word “marriage”.

There was a Canadian political figure, Stockwell Day, who seemed to have similar beliefs in the rights of the majority over the rights of a minority.  He pushed for a law that would have required a referendum on any proposal supported by a petition signed by 3% of Canadian voters.  He stopped talking about this when Rick Mercer (sort of Canada’s Jon Stewart) called for a national petition forcing Stockwell Day to change his first name to “Doris”.

Perhaps the correct response to Proposition 8 is to put a constitutional amendment on the ballot requiring Ken Starr to change his first name to “Brenda”.

If you know us, you know we have raved about the view from our apartment.  It’s not the absolute best view in the world, but it is pretty stunning to a gal from the flat lands and buildings of Champaign, IL.

That picture (and an absolutely ginormous version) were taken and stitched together by Randy Stewart from Seattle.

Randy apparently walked into our apartment, said, “Oh my god!” or something such, and dashed into our bedroom to take pictures.   Our other dinner guest, from metropolitan Canada, was slightly perplexed/bemused by his reaction.  It is completely ordinary for people in cities in Canada to live in high-rise apartments.

Randy (and Jim and I) are from the US.  It is very uncommon for people to live in high-rises in the US.  If I think really hard, of all the thousands and thousands of people I have known, I can only think of six households who I know ever lived above the sixth floor, and one more who I think might have had a high-rise condo a few years ago.  (And three of those households were in the same building in Mountain View.)  That is it, period, total, everybody, and I had to think pretty hard to come up with that meagre ration.

Why are there so few high-rise residences in the US?  There are many factors.  I am by no means an expert, but these are a few:

• The American Dream of owning your own house is cliche for a reason.  It is assumed that if you don’t own your own house, at least you aspire to owning your own house.  To not aspire to have your own house is sort of like not wanting to own a TV.
• When renting, you generally get more square feet per dollar in a shared house than in an apartment, especially a shared apartment.  Even I, through all my moves, have only lived in four apartments in the US, and two of those were while I was a student.
• In some places, the only high-rises around were “the projects” — government-built and -run publicly-subsidized housing.  Thus high-rises were decidedly un-sexy.
• In California, where one-eighth of the US lives, cities can’t afford dense housing.
• California has earthquakes, which makes people nervous about high-rises, even though high-rises constructed to modern codes are much safer in earthquakes than older houses on landfill or alluvial floodplains.
• Traditionally, cheap gas has meant that it was feasible to live quite a ways from work.
• In the US, not having a car can have a significant negative impact on the quality of your life, parking is difficult in cities, and very few US cities have good public transportation systems.  The poor transportation is due partly to density, but also part to the easy availability of guns.  Many people in the US are afraid of taking public transportation.

You should also note that Vancouver has worked very hard to develop its downtown.  The fact that it has a vibrant and vertical downtown was the result of very deliberate and careful urban planning, not an accident of fate.

• gHacks


• Apple


• Chris Pirillo (YouTube video)


• Cameratown (YouTube video)


• Idenzine


• Softonic


• MacReviewCast (Podcast)


• AddictiveTips


• Techblissonline


• PhotographyBlog


• Rj Dollen (Cool art using Shape Collage)


• Digital Inspiration


• Lifehacker


• TWIP


• MacHeist (Where's Waldo)


• Macbreak Weekly (Podcast @ 1:33:45)

• chickageek


• GeeksRoom


• Geek.ms


• Softonic


• UpToDown


• Visual Beta


• TechnoBITA


• Entrebits


• Nivmar


• Sebalorenzo


• TecnoBITA


• Jorge Serrano


• arturogoga (YouTube)

• Baixaki (Screencast)


• Misto Quente


• bernabauer.com


• WinAjuda

• maestroalberto


• ManuIdea


• Pucciom


• Gekissimo


• Click|blog.it


• guidami


• ilarialab.com


• Tutto Volume


• polycoke


• Il Bloggatore

• Pobby-dog Blog


• Jinn's Blog


• Lawrence's Coffee Shop


• unkb.com


• UIUI


• Hong Kong Yahoo Blogs (wedding photos in the shape of a heart)
  
  
  
  • jw
  
  
  • p4pinky_yip
  
  
  • mobe_yuen
  
  
  • Katherine FanFan
  
  
  • fatpofatpo
  
  
  • mscaca1202
  
  
  • tracy_007hk

• Oshiete-Kun


• Gigafree


• yknakamimi


• hatena


• Nutspress


• Lifehacker.jp


• Moongift


• National Creators Conference

• SimFilePC


• Krang


• Drchoi Blog


• WalkPC

• fdoun


• Cyrille Borne


• Vincent Dutrait


• Glandouillage


• MaxiApple


• Guppytrucs Freeware


• ZDNet.fr

• Roman bloggt


• Gif-Bilder


• drweb.de


• Apfel


• Ehrensenf (Video)


• Blogwiese

• Lexi_Ru


• Russian Apple Community


• Deep Apple


• Fotokolo


• baboker (Video)


• catode

• PC blog


• Programosy


• Fotomaniak


• Fotopolis


• Skrzypol

• Catholica NETwork


• Topo's Page


• efoto

• beszeljukmac


• Techline


• kepszerkesztok


• szoftverbazis


• Sablonos Blog

• TurkMac


• pdfdergi

• PhotoPlace.nl


• Clickx


• fotoplezier


• Kindje op Komst

• DigiCamera

• Syamsu


• Agusbagus

• Serbain Macintosh User Group

• dahlstroms


• stationsvakt


• Veronicas Foto

• Rehla Online


• SUAE

• sfjalar

• Bedre


• LatFoto

This happened back in late 1994 or early 1995, but thinking about it still makes me laugh…

Ben Johnson, a graduate student at the University of Illinois developed a wonderful, wonderful webmail tool called @ATS to help NCSA provide email support for Mosaic, the first graphical Web browser.  I was good friends with Mitch, who was in charge of Macintosh tech support for Mosaic, so I heard about it.  I could see that it would be immensely helpful answering all the email questions I got in the course of my job as www.uiuc.edu webmaster, so talked NCSA/Ben into letting me use it also.  Ben warned me that it was still quite new and under development, but I didn’t care — it made my life much easier even in that early stage.

Back in those days, there wasn’t nearly as much spam.  The very first commercial Usenet spam had only come out in January 1994.  Thus I was a little surprised to get a message to the webmaster account with an extremely pornographic subject line, and a body that was a word salad of dirty words.

I forwarded the message to Mitch, and asked in the body of the message, “Do you get messages like this frequently?”  I got back a message from Mitch saying that yes, he did sometimes get messages like to the Mosaic support email address.  Fine.  I forgot about it.

The next day, I saw Ben.  He said that he had needed to do some troubleshooting of @ATS, and had to go into my account to check something out.  I had developed a webmail system myself, so empathized.  This seemed perfectly understandable and reasonable to me.  He continued by assuring me that he was only there for a moment and emphasized vigorously that he did not look in any of the messages, only at the subject lines.

I think he was puzzled but relieved when I burst out laughing.

I did explain why Mitch sent me a message with the subject

Subject: Re: verb your feminine-noun

and let Ben know that he didn’t need to inform the higher-ups about potential sexual harassment.

Wow, Nate Silver just nailed it today.  His posting on the two different meanings of the word “progressive” seemed very clear and resonated completely with me.  I am, based on his definitions, absolutely a rational progressive.

By popular demand, my custom LaTeX styles are now available for download. (All of them have been dedicated to the public domain, I disclaim all copyright.)

There are four for now, but the set will grow. The first lets you use OpenType fonts (which is pretty much any system font); the limitation is that this works only with a single distribution, XeTeX, available only for Mac OS X, and a new experimental build for Linux. A second style typesets all chapter titles, sections, subsections and subsections in a sans-serif font (instead of the default serif). Very effective when used in conjunction with the OpenType style.

The final two adjust margins to require less paper for draft prints and such. Be good, be green.

How often have you found yourself calling a friend to get the phone number of a mutual friend? And then having to hold the phone while your friend pulls up the contact list on their phone, then recites the number to you, and then you write it on paper because your phone won’t let you add contacts while you’re on a call, and then you misplace the number you wrote on paper, ad nauseum. Why isn’t there a single button that says “Send this Phone Number to the Current Caller”?

It’s a common problem. You’re out and about, and realize you need to call a specific person, but you don’t have their phone number (or more often, you have it on your desktop computer, or your laptop, but that doesn’t do you any good in the current situation.) So you decide that the best thing to do is to call a mutual friend and ask them.

When they receive a phone call from you, they’re fumbling to hold the call while they look in their address book. (That is, if they’re lucky, and if their phone actually lets them open the contact list while they’re on a call.) More often, what happens is that they tell you to hang up while they consult their address book. And then you have to hunt for a piece of scrap paper because your phone won’t let you add a number to the list like that.

What the world needs is a button next to each phone number in the contact list that only appears whenever you’re on a call. The button, when pressed, sends an SMS from you to the current caller, and contains within it the information from the contact record you just selected. It doesn’t have to be too fancy, a two-line VCF record should do nicely.

If the recipient’s phone understands this method of contact transfer, it can prompt the user and import it automatically. If not, the user can still read the SMS herself, and dial the number. No more paper, no more fumbling, no more “let me call you back”.

It’s so easy, a caveman could do it. If only phones implemented it!

I made a new map of where the stimulus spending was forecast to go. I pulled the data from the White House Web site a few days ago, so note that it is for the proposed stimulus bill, not the bill as it passed. Also, it shows per capita amounts, using 2006 population figures from Wikipedia, so there will be a minor error due to the years being different.

I didn’t make a lot of noise about this one because I hope to do another, better one when the numbers are more stable.

Jobs created or saved per capita

I wrote this recently, not as a blog post, but for another purpose. I figured I’d post it here like I do everything else.

Re•search: noun. Investigation or experimentation aimed at the discovery and interpretation of facts, revision of accepted theories or laws in the light of new facts, or practical application of such new or revised theories or laws.

—Merriam-Webster Dictionary.

The last part of that definition has always been the chief motivator for me in my research — practical application. While all research seeks to discover universal truths and deeper meaning, I strongly believe that researchers have a responsibility to contribute to society in other tangible ways as well.

Just as Gutenberg’s invention of the printing press in 1439 made literary works accessible to everyone, the Internet is likewise speeding up the propagation of knowledge now and will continue to do so in the decades to come. We are on the brink of a cultural revolution where ideas, prototypes, discussion and research know no boundaries of location or time. The Free Software Movement is promoting users’ freedom to understand and explore computer programs. The Creative Commons project encourages authors, scientists, artists and educators to distribute their creations under licenses that foster the sharing of ideas, encourage discussion, engender a culture of openness, and speed up innovation. This provides enormous opportunities for researchers to collaborate in real-time across institutions, countries and continents, and to serve the community by disseminating their research results via public blogs, videos, slides, prototypes and designs.

As a researcher in Human-Computer Interaction at the Dept. of Computer Science at Virginia Tech, I have developed several tools and prototypes that would be of benefit not just to researchers but also to computer users. It is by studying their habits that I designed these tools — to them, I owe these tools. I work in the area of Personal Information Management (PIM), and study how users access and manage information such as files, calendars, email messages, contacts and bookmarks on multiple devices. I release all such tools and software to the world at my web site under licenses that permit anyone to inspect the source code, build upon it, and benefit from it.

In the process of my research, I developed a program to access Google Calendar which now has over 25,000 users. A calendar converter program I wrote is used by an average of more than 200 users per day. During Sustainability Week 2008 at Virginia Tech, I released a Blacksburg Transit Schedule application for cell phones to encourage Blacksburg citizens to take the bus instead of driving. It is used by about 300 users every month and growing.

In the spirit of working on real products that are used by real people, I interned at Google three times during my Ph.D. (2005, 2006, 2007.) In 2007, my project enabled users of Google Book Search to clip personalized content from books and embed that into their own web site or blog. This enables teachers to excerpt from literary classics for their class home page, for literature scholars to debate the nuances of texts, and for commentators to dissect parts of books. My intern work was covered by several news outlets, chief among them, at Google’s Corporate Blog.

Academia encourages published work—publish or perish, they say—while original contributions such as new ideas and untested directions are undervalued in the traditional ways of evaluating research. The Internet changes that too. Several times when I have come up with ideas that may or may not be viable research projects, I have written about them on my blog. The public scrutiny and invaluable feedback I’ve received made it easy to separate the wheat from the chaff. My advisor has always been supportive of those ideas that were encouraging research directions: the latest among them resulted in a paper that has been nominated for the ACM SIGCHI Student Research Competition 2009.

An area that I have recently been concerned about is the open publication of raw data sets. I perform human experiments which are reviewed by the Institutional Review Board (IRB) for ethical compliance. There is inherent tension between the privacy implications of human experiments and the Open Science dream of being able to publish all experimental data publicly so that others may analyze it in novel ways. I plan to investigate the ethical, moral and legal responsibilities of such an endeavor, recognizing that we as researchers owe two allegiances: to our experiment participants and to the scientific community, in that order.

I am happy to be a researcher at a time in our history when competitive collaboration trumps closed confidentiality. Science and innovation can only progress faster when information is freely shared among researchers, scholars and citizens.

Dear diary,

Yesterday, I played Wii Sports for 5 hours. Today, I have aches in muscles I'd forgotten I had.

That was fun.

I made a demo of my new mapping framework, a choropleth map of the 2008 presidential election that you can combine with various demographic layers, zoom in, zoom out, change the colour mapping, and all kinds of good things.

The base layer is a map of the election results by county.  Here it is with all the controls clipped off:

Voting margin layer

Blue counties are ones where more people voted for Obama; red counties voted for McCain.  The intensity of the colour shows how big the spread between the two candidates was.  Counties are white if the race was close.

A liberal friend of mine said with some dismay, “That’s an awful lot of red!”  That’s true, but most of those red counties have very few people in them.   One way to get a sense of how few people live in the center of the country is to overlay ZIP code locations (presumably the centers of the ZIP codes):

Voting layer with ZIP codes overlay

(The ZIP layer is even more interesting in bigger images.  I’m showing cropped, tiny versions of the map; you can select medium and huge sizes too.)

Another way to see the population density is to overlay the population density layer on top of the voting layer.  I set the population density layer to go from full white at zero people per square mile to full green at 400 people per square mile:

Voting layer plus population density overlay

Note that 400 people per square mile is still very, very low compared to urban areas.  (New York City has around 50,000 people per square mile.  Some census tracts in San Francisco have around 200,000 people per square mile; San Quentin has a density of 250,000 people per square mile.)  I recommend playing with the colour mapping to get a sense of how dramatically unevenly the country is populated.

The next thing to look at is where the Obama voters were.  In addition to urban areas, racially diverse areas went for Obama pretty stunningly consistently.  If you lay the percent-White layer on top of the voting layer, it is really striking how often the blue of an Obama county coincides with the white of a low-percent-White county.  In the percent-White overlay in this map, full white corresponds to 70% of the population or less being White, while full green corresponds to 100% White:

Voting layer plus percent-White overlay

You can see that the West Coast and the South are more racially diverse, as well as some odd, isolated islands of blue in the middle of the country — mostly in the Mountain time zone.  What are those odd blocks?

Those isolated patches of high-Obama, low-White correlate with Native American population.  Here is a map of the percent-Native, with full white corresponding to 70% Native population and full green corresponding to 100% Native population.  (I turned off the voting layer in the next picture.)

Percent Native American Layer

Not every place with a high Native population went for Obama.  For example, Oklahoma has a high Native population, but Oklahoma went for McCain pretty reliably.

The sharp-eyed viewer will note a few blue counties in Idaho, Montana, and Wyoming that do not have a high percentage of Native Americans.  These are the urban areas of Butte and Missoula, plus the resort areas of Sun Valley and Jackson Hole.

A prominent feature on the electoral layer is the blue crescent in the South.  That corresponds neatly with high concentrations of African-Americans.  On the demographic layer on this map, full white corresponds to 0% African-American and full green corresponds to 50% African-American:

Voting layer plus percent Black layer

You can see correlation between Latino populations and Obama voters, although it isn’t as strong a correlation as elsewhere.  In the demographic layer of the map below, full white is 0% Latino and full green is 50% Latino:

Voting layer plus percent Latino layer

Finally, there are some pale counties in the Midwest that are surrounded by red counties.  I haven’t looked at each and every one, but I think they are college towns.  Here’s a zoomed-in overlay of median age over the voting layer, with full white corresponding to a median age of 25 and full green to a median age of 45:

Voting layer plus median age layer

The areas that voted for Obama that can’t be explained by population density, racial diversity, or college towns are the upper midwest (e.g. Minnesota, Wisconsin, and Iowa) and the upper Northeast (e.g. Maine and Vermont).   Obama spent a lot of time in Iowa, but I guess the others are just intrinsically liberal.

I also have point-based overlays for locations of Walmarts and health food stores.  During the campaign, Fivethirtyeight.com talked frequently talked about the Walmart-to-Starbucks ratio, so I wanted to show those.  Unfortunately, although I found a database of Starbucks locations, I wasn’t able to get permission to use it on my maps.  (It’s a pity, because Starbucks locations seemed to correlate very well with Obama votes.)  I bought data of Walmart locations and health food stores (figuring that health food stores would be a squishy liberal kind of thing) but eyeballing the data didn’t really convince me there was a correlation.  But you can go try it out for yourself.

You can also see the locations of all 756 Native / Alaskan / Hawai’ian reservations from the dots layer.

Finally, I have a layer that shows the total votes (Obama + McCain + everyone else) divided by the population.   I don’t know how much to trust this layer, as the demographic data I have is from the 2000 census.  Areas that gained or lost a lot of people in the past eight years will be skewed.  While the demographic data in all the demographic layers will be slightly off, it will be worse for this layer, especially in counties that don’t have a lot of people.  (Loving County, TX, had 79 presidential votes in 2008 but only 67 total population in 2000.)  Rural counties tend to have more land area than urban counties, so visually they will be overrepresented.

I encourage you to go play with the maps.  In particular, I had to use zoom level 3 in order to get maps that would fit on my blog, but you lose a lot of detail and information.  Also, when you aren’t worrying about running afoul of Google’s copyrights, you can let the map layer peek through to give you more context for what you are seeing.

Enjoy!

In an article for Slate, Daniel Gross points out that the Davos attendees, who are quick to lionize Great Men (yes, usually men) when there are big successes, are blaming the system now that things are going wrong.

I am not at all surprised.  As I have blogged before, Larissa Tieden’s research shows that people think that high-status people do good things and that low-status people do bad things.  In this context, low-status people are not powerful people who do bad things, but the invisible, the drones, the unseen — the system.

In this case, it is rather obvious that many, many people were complicit in the financial collapse: from those who made financial policy, to those who perpetuated the policy, to those who made bad deals, to those who took risky deals.  I don’t find fault with saying that the system caused the failure.

I do think that we under-recognize how many people are “complicit” in successes.  While it might be tempting to say that Google’s success, for example, was the success of the Great Men Larry Page and Sergey Brin, perhaps it really was the system: the fertile technical environment, the fertile financial environment, the fertile educational environment, the hard work of many early employees, and so on.

This doesn’t even factor in all the people who were positive influences on the people involved prior to Google’s birth.  For example, their early, key employees were alive, while in other times and places not all of them would have survived to adulthood.  The entire health care system — consisting of surgeons, doctors, epidemiologists, vaccine producers, hospitals, clinics, and insurance bureaucrats — were thus important.  It’s the system.

If the Davos attendees really recognizes the importance of all the other people in any success, and by extension in their success, it would probably be harder for them be comfortable about their wealth and power.

I have been struck and somewhat puzzled by how happy my friends’ dogs have been to see me. My old housemates’ dogs, for example, would get all excited to see me, even after a gap of several years.

I realized last weekend that the dog has no idea where I’ve gone, when I am coming back, or even if I am ever coming back. It must be like someone going off on a sailing ship 200 years ago: no idea where they are, when they will come back, or even if they will come back.

A monkey riding a hippo!

Bears boxing!

 

The Asian squat!

As seen in a restaurant (Hint: this is not a fish)

Some other restaurant

Yet another restaurant

LEE KUM KEE!!!

Grandma's childhood home with almost all of her grandchildren

Disneyland!

From time to time since the election, I have burst into tears.  Oftentimes it hits me without warning.

Today, I burst into tears twice.  The first was at seeing a picture of Tommie Smith, John Carlos, and their wives in a group hug with Obama’s inauguration on the TV in the background.  Tommie Smith and John Carlos were the two who gave the “black power” salute on the podium at the 1968 Olympic Games.  That picture really gave me a sense that we have closed a chapter in our national history.  The story of racism in the United States has not ended, but a chapter has closed.  Slammed shut.

The second time, I could only eke out a “Honey!”  Jim asked what, but I couldn’t answer because I was convulsed with sobs.  Jim rushed to me, concerned.  All I could manage was, “He signed it!” and point at the article saying that Obama had signed an anti-torture executive order.    Thank you, Mr. President.  I have been waiting for this for a long time.

Watching the inauguration today, I couldn’t help but be reminded of gay pride parades.

There are very few mass events where everybody is really happy.  Most large gatherings are sporting events, and there is almost always an undercurrent of hostility somewhere.  If nothing else, the losing team’s fans are unhappy — usually.  (I went to the 1994 Men’s World Cup match between the US and Brazil.  Everybody knew that the Brazilians were way better than the USA, so in this case, the losers didn’t care that they were losing.  Furthermore, it was the Brazilians!  Their spirit of joy and fun was infectious.)

Maybe rock concerts are also places of fun sometimes, but I never had much fun at rock concerts.   The band was always too far away to see and yet too loud.

The C-SPAN feed of the inauguration didn’t have any commentary, but instead just broadcast the ambient noise of the crowd.  For two miles, almost all you could hear was people screaming their heads off as the motorcade (and microphone) moved down the route.

Ten years ago, when I marched in the SF Pride Parade with PFLAG, I experienced two miles of people cheering their hearts out.  The straight public was not as accepting of gay and lesbian people back then, and thus (I guess) it meant a lot to see straight people standing up for gay rights.  So they cheered.

In recent years, the cheers for PFLAG have been polite, but not overwhelming like before.  That’s a good thing — that means that public acceptance is greater, so PFLAG isn’t as needed.  The cheers for PFLAG were a reflection of how bad things were elsewhere, and how PFLAG represented a beacon of hope.  It is a very good sign that the cheers for PFLAG are tamer now.

Similarly, the cheers for Obama reflected how bad things were.  It would not have been so moving for African-Americans if African-Americans had not faced such brutal ill-treatment in the US.  It would not have been so moving for me, a white person, if I had thought that G. W. Bush had done a competent job.

I hope that at the inauguration of the next non-white president, the crowds are much smaller and tamer.

Postscript: Apparently there were exactly zero arrests at the inauguration.  I once asked a cop at the SF Pride Parade what it was like to work the parade.  His answer: “Four hundred thousand people, no arrests, no injuries, what’s not to like?”  (I had just gotten off of my shift as a Safety Monitor, and had first-hand knowledge that “no injuries” was a sligh exaggeration, but the injuries were all very mild — e.g. people skinning their knees, not e.g. getting beaten, hit by cars, falling 30 feet, etc.)

Randall Munroe’s XKCD has inspired interesting product features in the past. A recent one has sent a lot of Mac users scurrying to set up an audio doorbell on their Mac Minis.

Here’s how you can do it.

The Source Code

<?php
if (isset($_GET['say'])) {
  $cmd = sprintf('say "%s"', preg_replace('/[^\w\d ]/', '', $_GET['say']));
  `$cmd`;

} else if (isset($_GET['up'])) {
  $cmd = 'set output_vol to output volume of (get volume settings)
  set volume output volume (output_vol + 10)';
  `osascript -e "$cmd"`;

} else  if (isset($_GET['down'])) {
  $cmd = 'set output_vol to output volume of (get volume settings)
  set volume output volume (output_vol - 10)';
  `osascript -e "$cmd"`;
}
?>
<form action="<?= $_SERVER['PHP_SELF'] ?>" method="get">
  <p><input type="text" name="say">
  <input type="submit" value="Say"></p>
  <p>Volume:
  <input type="submit" name="up" value="Up">
  <input type="submit" name="down" value="Down"></p>
</form>
<p>Hey Randall, here's how you
  <a href="http://xkcd.com/530/">change the volume</a>.<br/>
  &ndash;<a href="http://manas.tungare.name/">Manas</a>.
</p>

Where to Put It

Copy the code to a new file, name it “say.php” (or whatever else you want to call it) and put it in Macintosh HD /Library/WebServer/Documents/. Remember, this is the top-level /Library directory, not the one under your own user account. You also need to make sure that Web Sharing has been turned on under System Preferences > Sharing.

How to Use It

Open a browser, and type in:

http://localhost/say.php

from your own machine.

To do this from another machine connected to the same router, use the Bonjour name of your Mac (this can be found under System Preferences > Sharing.) E.g.

http://Manas-Desktop.local/say.php

To do this from a machine outside your router, you need to have configured your router correctly. And if you can do that, you don’t need me to tell how to do the rest of it.

Bonus feature: this script also lets you increase and decrease the volume. Perhaps then, Randall’s roommate might have heard him.

Warning: Security Issues

To achieve what this script does, it needs to take input from the Web and use it in a command that executes in a shell. The input is sanitized and I believe that such a risk is minimal. However, if you’re concerned about this possibility, do not install this script. You are responsible for what you do with this script and your machine. If you’re really concerned, put it inside an authenticated session.

Update: Added a command injection filter suggested by Mac OS X Hints user skicker.

Do not take my silence on Caroline Kennedy as an endorsement.  I’ve just been busy.

I am almost as offended that Caroline Kennedy is being considered for New York senator as I was when McCain picked Palin.  I say “almost” because Senators do not have authority to fire nuclear missles.  Selecting Caroline Kennedy is merely a bad idea, not scary.

I am dismayed with my own party that they are not raising a bigger stink about her.  Somehow the magic of the Kennedy name seems to be enough to quiet lots of people.  Huh?

Now, I am willing to believe that being part of a political family can be useful.  You do learn things at the dinner table.  (My father was a professor of Physics;  I was in college before I realized that not everybody drew free-body diagrams on napkins over dinner.)  So Caroline Kennedy might have learned something from her father.

Except that she stopped having dinner with her father when she was six.

Maybe her mother helped instill political savvy into Caroline.  I wasn’t there, I don’t know, but I never heard much about Jackie Kennedy Onassis being much into politics.

I think Sarah Palin would be much more credible as a senator.    Palin’s limitations — her lack of sophistication in both foreign and national-level — made her IMHO a poor choice to represent the U.S. to the world.   However, she’s held office, she understand politics, is a proven campaigner, and is strongly Alaskan.  She is certainly a legitimate choice to represent Alaska to the nation.  (I wouldn’t vote for her, but that’s because I don’t like her politics, not because I think she’s unqualified.)

Kennedy, sorry.  I don’t see how she qualifies.

10 years ago, on this day, I received the first payment for a tiny shareware program I wrote. It was a one-trick-pony, a birthday reminder program.

Folks around me will tell you I’m bad at remembering things, and always choose to write everything down. So I wrote a program to keep track of friends and birthdays, and popup a reminder on the right day, perhaps a few days earlier if I needed to buy a gift first. As a teen writing and distributing my first-ever commercial piece of software, I had named it eponymously as ManasTech Birthdays. I wrote it, released it, and sold a few copies (nothing astronomical, but yeah, it was fun to see the checks coming in.)

As I look back nostalgically, several thoughts cross my mind today. The more things have changed, the more they have remained the same. I loved writing software then, I love writing software now. I used to sneak out of classes and stay up late nights to code up the latest idea that crossed my mind. Today, I procrastinate on my research and take time off to write little scripts and tiny widgets, that — before you know it — have hundreds of users and need regular maintenance.

Then, the norm was to release products as shareware, without the hassles of finding a publisher or having to stock shelves with packaging. All you needed was an Internet connection and a program that solved a need. Today, the norm is to free your software, and release it into the wild. The Free Software Movement has been gaining ground ever since.

The most interesting aspect is perhaps the domain in which I’ve been working. The Birthday Manager helped store your personal information — contacts and calendar events. Today, I’m involved in Personal Information Management research, which is pretty much the same topic. I even wrote a paper detailing how people manage their calendars. Little had I known 10 years ago that I would be fascinated enough with personal information to pursue a career in it.

Sometimes I wonder what the next 10 years will be like.

Update: P.S. For those curious about how I got hold of the check image for this blog post: yes, I’m a digital packrat and have a copy of all my files dating back to 1997 on my current laptop.

I'm trying to get my laptop to display on dual external monitors. I'd appreciate some guidance in getting this configuration set up

What I have

• A Dell Latitude D620 with nVidia Corporation G72M [Quadro NVS 110M/GeForce Go 7300]
• A Dell docking station with dual-link DVI output
• 2 Dell 2007FP Monitors
• Ubuntu 8.04

What I want

• When I dock my laptop, the two external monitors are used (I don't really need to use the internal screen as a third monitor)
• When my laptop is undocked, my internal screen activates.

Make a New Year’s Resolution to start backing up your data regularly. Not just local files, but even data from the cloud. Here’s how to backup your data from a few of the most common online services. More importantly, I’ve also included instructions on how to restore from that backup.

Twitter

How to backup?

Copy this URL to a new browser window.

http://twitter.com/statuses/user_timeline/USERNAME.xml?count=10000

Then replace the string USERNAME with your actual Twitter username. Press enter to start downloading. If your browser does not prompt you with a file download box, but instead opens the file showing a bunch of text, choose File > Save As to save your backup to a secure location.

How to restore?

You cannot restore this data into Twitter (neither to your own account, nor to a different account.) But you will have access to your witticisms and interesting web links that you posted to amuse your friends. Do with it as you please.

Gmail Messages

How to backup?

Use an IMAP client such as Mail.app on the Mac, or Thunderbird on any platform. Make sure it’s configured to download and cache every email and every attachment.

How to restore?

You can access your messages from these programs even if Gmail is down. If you need to transfer messages to another account, add that new account in the same program as a new IMAP account, then drag-and-drop messages from your old account to your new account to transfer them there.

Gmail Contacts

How to backup?

Login to your Gmail / Google Apps email account, then open the Contact Manager. Click on the Export button in the top-right corner. For maximum compatibility with other applications, choose the third option for data format, vCard format. (It’s a standard format for contact information exchange.)

How to restore?

The vCard format is fairly standard. Gmail itself can read back the same file without trouble. To import into Mac OS X Address Book, simple double-click the .vcf file and let the import proceed. Microsoft Outlook also supports importing addresses from vCard files.

Google Calendar

How to backup?

Google Calendar publishes feeds of your calendar in the iCal format. If you save this feed to a file, you can use it as a backup. On the left side of your main calendar, there is a list labeled “My Calendars”. For each calendar that you want to backup, click on the little downward-pointing arrow next to the calendar name, and select “Calendar Settings”.

Calendar Settings" /> Calendar Settings" title="My Calendars > Calendar Settings" width="327" height="265" class="alignnone size-full wp-image-248" />

On the Settings page, under Calendar Details, locate the section labeled “Private Address”. Click on the button labeled ICAL and copy the URL there. Open a new browser window and paste the URL there. This will start downloading a file; save it to a safe location — this is your calendar backup. Lather, rinse, repeat for each calendar you want to backup.

Private URL > ICAL" /> Private URL > ICAL" title="Calendar Details > Private URL > ICAL" width="510" height="106" class="alignnone size-full wp-image-249" />

How to restore?

The iCalendar format (also abbreviated as iCal or .ics) is a standard calendar format. You can import the backed up calendar file into Google Calendar, Apple iCal or Microsoft Outlook simply by opening it.

Google Reader

How to backup?

Login to Google Reader, then come back here and click on this link: Export Google Reader subscriptions as OPML. Save the file that your browser will prompt you to download. This is your backup.

How to restore?

Google Reader and lots of other feed readers know how to import OPML files. In case of Google Reader, go to Settings > Import/Export to import it back. For desktop software, try looking for an “Import from OPML” menu item somewhere.

Need instructions for more services? Write a comment and I’ll try to provide them.

OmniFocus and WebDAV

OmniFocus is a GTD app for Mac OS X and iPhone. Obviously, the iPhone version would be pretty much useless if the two did not sync. The good folks at Omni have made this possible, but it requires either a little money or a little time. I have more of the latter than the former, so I chose to use my own WebDAV server (which I can run for free on my desktop Mac) instead of a paid subscription to Apple’s .Mac or Mobile Me service.

Setting up WebDAV on Leopard

The good news is that all the bits and pieces of software that you need to run a WebDAV server on Mac OS X 10.5 Leopard are already installed. You only need to configure them correctly and turn them on. Some experience with Terminal is preferred, and you should be familiar with executing UNIX commands. Let’s start!

1. Start Apache. (If you haven’t already) You will need to enable Web Sharing, since the WebDAV service will be provided by Apache, the web server on Mac OS X. You do not necessarily need to have a web site running, but you will need to activate and run Apache. Go to System Preferences > Sharing, and turn on the box labeled Web Sharing.
   
2. Enable WebDAV support in Apache. Edit the file /etc/apache2/httpd.conf, (remember to use sudo to edit it) and locate this line:

   LoadModule dav_module libexec/apache2/mod_dav.so

   Make sure it is not commented (there should be no "#" at the beginning of the line.) Then locate this line (towards the bottom of the file):

   Include /private/etc/apache2/extra/httpd-dav.conf

   Again, make sure it is not commented out. It is disabled by default, so you need to remove the "#" from this line.

3. Configure WebDAV. Next, edit the file /etc/apache2/extra/httpd-dav.conf. Add a section in it to create our new WebDAV share. Here’s what the new section should look like. As a security precaution, you should also go ahead and delete the /usr/uploads share that is set by default.

   Alias /webdav "/Library/WebServer/WebDAV"
   
   <Directory "/Library/WebServer/WebDAV">
     Dav On
   
     Order Allow,Deny
     Allow from all
   
     AuthType Basic
     AuthName WebDAV-Realm
     AuthUserFile "/usr/webdav.passwd"
   
     <LimitExcept GET OPTIONS>
       require user YourUserName
     </LimitExcept>
   </Directory>
   

   On line 1, the name following the Alias keyword is the URL you’d like for your new WebDAV share. If you want the share to be located at http://your-server-name/your-fancy-webdav-share, then line 1 should read:

   Alias /your-fancy-webdav-share "/Library/WebServer/WebDAV"

   On line 9, we specify the authentication scheme as Basic, not Digest. The security conscious will note that this sends unencrypted passwords over plain text. In my tests, OmniFocus was not able to communicate with the server with the Digest authentication scheme. Remember not to use a particularly important password for this account.

   On line 14, substitute the username you would like to use for your WebDAV account. Note this down, because you will need this again in the next step.

4. Create user accounts and passwords. Use the htpasswd tool to create your password file.

   sudo htpasswd -c /usr/webdav.passwd "YourUserName"
   New password:
   Re-type new password:
   Adding password for user YourUserName

5. Create the necessary directories.

   sudo mkdir -p /Library/WebServer/WebDAV
   sudo mkdir -p /usr/var

6. Setup permissions correctly.

   sudo chown -R www:www /Library/WebServer/WebDAV
   sudo chown -R www:www /usr/var
   sudo chgrp www /usr/webdav.passwd

7. Restart Apache gracefully.

   sudo apachectl graceful

8. Test your server. Optionally, you can test your WebDAV configuration using litmus, a WebDAV server test tool. It is distributed as source code with no binaries, so you will need to compile it first, for which you will need Apple’s Developer Tools. You can test your server manually by using a graphical client such as Goliath. Try uploading a file and see if you can access it again.

That’s it, you can now point OmniFocus to http://your-server-name/webdav and provide the credentials you created earlier. With this setup, you will immediately be able to access your WebDAV server over your local network. If your machine has a static public IP address, you will also be able to sync from outside your local network.

If, on the other hand, your machine is behind a router, you will need to configure port forwarding on your router. If you do not have a static IP, you will need to set a dynamic hostname via services like DynDNS.

Possible Error Messages

This is by no means a zero error configuration, and sometimes things might go wrong. Here are some of the common error messages and how to fix the relevant errors:

• The locks could not be queried for verification against a possible "If:" header.
No such file or directory: Could not open property database
  The web server is not able to access the password file. In our example, you need to ensure that /usr/webdav.passwd can be read by the Apache user, www. To do that, run the following chmod command.

sudo chgrp www /usr/webdav.passwd

A hat tip to Vivek for helping test these instructions on a clean Leopard installation.

When most professors have closed-door policies and need weeks of lead time before being able to schedule a meeting, here’s why I love working here!

Nothing hammers home the sense that your life has settled down for a while like signing a 1-year mobile phone contract that says "I'll be staying here at least a year".

I promise, I can explain. It started out rather simply, then got a little out of hand. A week or so later, I'm still having an immense amount of fun.

It all started when Google gave me an awesome Christmas present: An HTC Dream. It's a very shiny mobile phone, and what's more, it's an unlocked developer edition. It's hacking time!

This is where things get a bit complicated. Lemme take you through the reasoning.

My first idea when I saw this beast was to try to get emulators running on it. A phone is nice, but a phone that can play vintage games is even better. I decided on playing with the Sega Genesis first, as I have rather fond memories of Sonic the Hedgehog.

First obstacle: Android (the open source OS that Google develops) currently can run only Java code. There is currently no open source Genesis emulator written in Java. Most of them are written in C, or in extreme cases, even in x86 assembler. There is currently no official way to execute native code on the android phone. I'd like to make this software available to everyone.

I therefore need to write a Genesis emulator in Java.

Okay, that should be simple. The Genesis is a relatively old console, so it can't be too elaborate. I mean, it's no PS3. All I need is an emulator for the CPU, a decoder for the ROM format, and some audio and graphics hookups within Android, and I should be good to go.

Well, first, the Genesis has three processors. A Motorola 68000 CPU, a Z80 sound processor, and a custom made graphics processor. Let's start with the 68k CPU. Apparently, there is no well known open source Java emulator for the 68k.

I therefore need to write a Motorola 68k emulator in Java.

But Java sucks. I mean, it's obviously a successful language, but I find no pleasure at all programming in Java. It is pure pain without an IDE on the level of Eclipse or Netbeans, and those IDEs aggravate me in various ways. The the the language language language is is is way way way too too too verbose verbose verbose (verbose verbose).

Plus, after consulting the 68k specs and sampling a few C implementations, it looks like an extremely repetitive task: most opcodes have around 20 variants, depending on addressing modes and various flag bits. It would be very tedious to implement this by hand, not only because it'd be in Java, but because it'd be even more mind numbing and uninteresting. However, the kinds of variants that are needed are quite amenable to be described at a high level, leaving the repetitive task of actual implementation to a program. And I can use a language that I enjoy for that, say, Python.

I therefore need to write a Motorola 68k emulator generator in Python.

After a bit of prototyping, I came to the conclusion that implementing this in Python would also be rather tedious, for a variety of reasons. First, I started off badly by writing a generator that goes straight from high level description language to a Java source code string, mushing several levels of abstraction together. Second, the description needed to generate the variants quickly lead me to combinatorial explosions, or to independent components that began interacting with each other in hilarious ways. Not good.

Plus, one day, when Android does have a supported way of running native code, I'd probably want an emulator in C or C++, running on the CPU directly, instead of under the Dalvik virtual machine. At which point all my work will have been for nothing.

I therefore need something of an emulator compiler, that parses the high level description into an execution tree for the opcode implementations, which a code generator then translates into a variety of output languages, such as Java, C++ or Brainfuck.

Python is nice, but I don't think that writing compilers is one of its fortes, despite what the PyPy folks appear to think. Manipulating the code representations is cumbersome at best, and the divide between the living Python code and the dead data it manipulates is rather wide. Manipulating code as data and vice versa is one of the often described merits of the Lisp family of programming languages. I've been wanting to get back to Common Lisp as a language and poke around with it more, and it feels like the ideal language in which to build a compiler. What could be more code-as-data-as-code than a program that takes apart a description of a program and puts it back together again in another form?

I therefore need to write an m68k emulator compiler in Common Lisp, at first targeting the Java programming language, and later possibly other languages.

And that is how, a week after Google gives me a mobile phone, I find myself writing Common Lisp code, for a compiler that compiles a lisp-like language into Java code, that will be compiled into Dalvik VM bytecode, running on an ARM-based embedded system, which when executed will emulate a Motorola 68000 CPU.

I feel like I've just had a Wikipedia attack. You know, that thing where you go to Wikipedia to look up something very specific, say how Tesla coils can be used to play music, and end up three hours later reading through an analysis of 14th century persian battle tactics, with no idea how you got there. That's kinda how I felt when I came up for air yesterday and looked back. "So, I got a phone... And now I'm writing a compiler... I'm pretty sure I have a good reason..."

Oh, and the emulator compiler is starting to work. I was very rusty in Common Lisp, but in a couple of days of hacking and prototyping, I'm starting to get somewhere. I can already generate the implementation of the simplest variant of the 68k ADD opcode. The "source" looks like this, with comments added

(instruction
   ;; Instruction name, with variant information.
   "add_dreg_to_dreg"

   ;; The description of the meaning of the 16 bits of the opcode.
   ((:literal 4 #b1101)      ; 4 constant bits, with the given binary value
    (output-register 3 dest) ; The output register, whose number is coded
                             ; over 3 bits. Its value is available in the
                             ; 'dest' variable.
    (:literal 6 #010000)     ; More constant bits, describing the addressing mode.
    (input-register 3 src))  ; Input register, similar declaration to dest.

   ;; How to perform this operation, in a subset of Common Lisp.
   (setf dest (+ src dest)))

The above instruction definition produces an opcode object that contains two things: information for the instruction decoder, so that it can identify this instruction, and the intermediate representation of the implementation of that instruction:

;; The constant bit values in the opcode, and the mask to test them,
;; for the instruction decoder.
(debug-print-opcode-mask the-above-opcode-object)

--> Output: 1101---010000---

;; The intermediate representation of the implementation.
(opcode-ast the-above-opcode-object)

--> (LET ((SRC (REGISTER-VALUE :DATA
                               (VM-OPCODE-BITS 3 0)))
          (DEST (WRITABLE-REGISTER-VALUE :DATA
                                         (VM-OPCODE-BITS 3 9))))
      (SETF DEST (+ DEST SRC)))

This opcode can then be fed to the Java code generator, to produce the output implementation:

(java-gen-opcode the-above-opcode-object)

--> public static void op_add_dreg_to_dreg(unsigned short opcode) {
      unsigned long src = mDataRegisters[(opcode >> 0) & 0x7];
      unsigned long dest = mDataRegisters[(opcode >> 9) & 0x7];
      dest = dest + src;
      mDataRegisters[(opcode >> 9) & 0x7] = dest;
    }

There is still a lot to be done. For one, the ADD opcode is supposed to update the CPU's state flags with information about the result of the addition. After that, the addressing modes other than to/from a numbered register must be supported. Implementing more opcodes will surely bring more things that need to be implemented.

Once a solid base is laid, a higher-still level of description must be layered on, so that all the variants of an instruction are produced from a single implementation definition. Once all that is done, a C++ backend would be nice. And why not attempt to generalize the compiler infrastructure, so as to support the compilation of emulators other than m68k CPUs?

By the time I get anywhere near that, I suspect that it will have become possible to easily write and release native code for Android, making all of my efforts unnecessary. But I don't care, this is damn fun!

Some people are of the opinion that I should get my head examined.

• There are *way* more asians in Winnipeg then there used to be. It used to be that you'd go to a bar and you could literally count the number of non-white people on one hand. I don't know where all these people came from. It seems that the generation just a few years younger than mine is a lot more multicultural, even in Winnipeg.


• Girls definitely dress a lot skankier here than in Toronto. Even with a windchill of -40, girls wear super short skirts and dresses. Girls in Toronto seem to dress rather conservative in comparison.


• Tonight it'll warm up to -18 with a windchill of -28. I think we've finally seen the end of this cold snap! (This is *not* a sarcastic statement, this is the actual view of people over here)

I realized I haven’t picked up a book in weeks, (non-academic book, that is), but I’ve read more than my fair share of blogs in that same time. I wonder if part of the reason is the longer time commitment required by a book. This prevents it from being read quickly and keeps it forever on my wish list. If so, then how about a service that breaks down books into blog-post-sized chunks and publishes them every few days?

The idea is inspired by, — nay, stolen from — Kevin Kelly, who is reissuing his 10-yr old book as a blog (hat-tip to Seth Godin’s post on the topic). His reasons are different, though. The book is out-of-print, and is already available as a downloadable PDF from his web site. Making it available as a blog is just another way of spreading his ideas wider, which is a great idea.

But apart from that, I like the idea of chopping up a book into chapter-sized chunks and making them available to readers one at a time. Not for any economic reasons, but because attentional resources are so scarce these days. A few times during the day, I have some free time which I use to read a few blog posts. If I ever thought about picking up a book during these breaks, I wouldn’t do it, simply because of the (arguably artificial) time commitment issues it raises in my mind. But talk about a chapter-sized, or even smaller blog post, and I’d read it.

Of course, not all book content has an affordance for this kind of splicing and dicing. If it takes several minutes for a reader to re-establish context from the last blog post, the purpose is lost. Some authors would consider their books a work of art too precious(ssss) to split it up into anything smaller. That’s also the reason why bands are often reluctant to sell singles instead of entire albums (apart from the record labels preferring to sell you 9 lame tracks bundled with 1 great track for $10 instead of $1, thank you very much.) But several non-fiction books could verily adapt to such a format.

The book-as-blog need not be free (as in no charge.) Sure, charge me for it. Implementation would be easy, charge me a micropayment and give me a secret watermarked feed URL. With so much new content licensed under a Creative Commons attribution license, it’s also possible to develop a web service that does this for liberally-licensed and public domain works. This is compatible with Creative Commons Attribution (BY), Attribution-ShareAlike (BY-SA), Attribution-Noncommercial (BY-NC), and Attribution Non-commercial Share-Alike (BY-NC-SA) licenses (but I’m not a lawyer, this is not legal advice, blah blah.)

Maybe something like this will finally get me back to the several-books-a-month club I used to be a member of, until I discovered this newfangled shiny thing called the Internet.

Am I the only person who is concerned that justice is moving too hastily on Blagojevich?  He stands accused of some pretty appalling stuff, but the key word is accused.  I realize that many civil liberties have been badly compromised in the past eight years, but I thought that the US still (mostly) believed in “innocent until proven guilty” for its citizens.  To throw him out of office before a trial would be unfair.

There is also some speculation that he didn’t do anything illegal.  He sure looks like a stupid, arrogant slimebucket, but that isn’t illegal.  He wanted to use the appointment to his advantage, sure, but there is lots of influence-trading that doesn’t get prosecuted, e.g. people donating to a candidate being rewarded with ambassadorships.  (It is always less shocking to discover how much illegal activity goes on, than how much is perfectly legal.)

It is important for civil liberties to ensure that the government not be allowed to deny anyone — even people we don’t like — fair, equitable process under the law, including the presumption of legal innocence.

Actual flight path and door-to-door transportation as recorded by my GPS device
(This is just a screenshot b/c Google Maps couldn't handle that big of a file)

• The taxi driver was giving me career advice. He thought that I should learn Mandarin.


• Cabbie was talking to me in Cantonese. I hadn't heard that much Chinese for a long time.


• It's still harrowing walking through the airport metal detector carrying a cat without a leash or anything.


• Tiki didn't move or make a sound the entire flight. I don't think the woman on the aisle seat knew she was even there.


• During the initial announcement, the flight attendant tried to make a reference to the commercial where the guy thinks he's on a flight to Hawaii, but is actually on a flight to Winnipeg and goes "...going to Winnipeg..." No one laughed.


• When asked who was from Winnipeg, the entire flight went in an uproar. No one goes to Winnipeg at this time of the year unless they are from there.


• The games on the Zune are actually pretty fun.


• Upon arrival, the flight attendant neglected to notify us of the local time (one hour behind)


• When it was announced that the current temperature in Winnipeg was -27, it wasn't a big deal or Earth shattering news.


• While waiting for my luggage, I saw a woman drinking a slurpee. This is why Winnipeg is the slurpee capital of the world.


• I miss the feeling of the cold fresh air in your lungs.


• When driving here, if it's not a big road, you actually drive on top of snow since the roads aren't cleared right to the asphalt and salt doesn't work b/c it's too cold.


• There are 3 cats within arm's reach of me right now as I am sitting on my bed.


• Mi-Mi has reclaimed her spot on my bed and refuses to budge. Tiki is confused. Tiggy is indifferent.


• The fish is big now.


• The dog is big now.

I am baffled by a concern that seems endemic in Canada: that the US is going to steal Canadian water.  The way they talk about it, it’s almost like they think there are already secret contingency plans drawn up that one more dry season in California will trigger.

This seems totally preposterous to me:

• I have never heard anyone in the US talk about routing Canadian water to the US.  I remember about ten or twenty years ago, hearing people talk about a canal to Oregon, to the Columbia river, but it wasn’t something that people were taking seriously.  It was sort of like how in the late seventies there were people talking about building space colonies.  There were a few people thinking about the theoretical possibility, but there wasn’t any real thought that they were practical.
• Which states might run out of water?  Let’s suggest California, Arizona, New Mexico, and Texas.  States in the South.   Where is Canada?  Way way north.  Where is the closest water to California?  Oregon.  Don’t need to go any farther.  Where is the closest water to Texas?  The Mississippi.  Don’t need to go any farther.  What is the easiest water to get to from Arizona and New Mexico?  Probably the Mississippi again.  Maybe you’d object that the Mississippi water isn’t very nice by the time it gets to Louisiana.  Maybe it is, but if they are out of water, they can’t be that choosy.  The next place they could look would be Lake Michigan; you don’t have to go over any mountains and you don’t have to cross any international borders.
• Why should the Canadians worry about water when they could worry about oil instead?  The US has a history of belligerency related to oil; I don’t know of any US belligerency related to water.
  • One Canadian, in response to that question, said “Yeah, but we already sell the oil.” Yeah, but Canada could sell the water, too. And Canada has a lot more water than it has oil.

So I find it a very odd concern. I am not suggesting that Canadians should think of the US as an entirely and always consistently benevolent country. I’m sure there are things Canadians should be nervous about. But water? That is so far down on the list of things that I would worry about that I find it very odd.

To be fair, I also have heard a man from Michigan be concerned about California stealing Michigan’s water.  (He went on an extended rant about how people shouldn’t live in places that required importing large quantities of natural resources.  I wonder how he would enjoy winter in Michigan without large imports of fossil fuels.)  Maybe they got this idea from the Canadians.

A buggy is like a car, but drawn by a horse.

Radio is like TV, but with no picture, only sound.

A letter is like email, but written on paper and delivered in days or weeks.

The Encyclopedia Britannica is like Wikipedia, but printed in 26 volumes, and occupies half a room.

A telegram is like SMS, but you need to send it from the telegraph office.

A newspaper is like Google News, but is only updated once a day.

TV is like YouTube, but you can only watch what’s playing right now.

A Victrola is like an iPod, but with fewer songs and lower fidelity.

Some collages of Tiki made using Shape Collage

Arranging photos into a collage

Snowboarding snowman collage

Volleyball collage

Wow.  Obama announced his six national security advisers today, and white men were in the minority.  Eric Holder and Susan Rice are not white; Hillary Clinton, Janet Napolitano, and Susan Rice are not men.

A few days ago, I read that Condoleeza Rice phoned Barak Obama twice during the Mumbai terrorist attacks and was profoundly moved by the mental image.  It’s not that there haven’t been black people in positions of power before.  I’m sure that Rice phoned General Powell more than once.  (While I dislike almost everything about G. W. Bush, I do have to give him props for not being afraid to appoint people of colour to high positions.)

What struck me was that it was a very powerful black person in one administration phoning a very important black person in the next administration.  This demonstrates that it is not tokenism, nor a fluke of one administration.  It says that having people of colour in positions of high responsibility is not odd or unusual.  And that’s the way it should be.

Warning: this is a long and geeky post.

From time to time in the past few years, I have mentioned that I was a little puzzled as to why more people didn’t render tiles on-the-fly for Google Maps, as I do in my U.S. Census Bureau/Google Maps mashup.

I have reappraised my attitude.  I have been redoing my mapping framework to make it easier to use.  I have reminded myself of all the hurdles I had to overcome, and discovered a number of annoying new ones.

First pass

I originally wrote my mapping framework in an extreme hurry.  It was a term project, and a month before the end of the term, I realized that it would be good for personal reasons to hand it in a week early.  The code functioned well enough to get me an A+, but I cut a huge number of corners.

Language/libraries/database choice

It was very important to minimize risk, so I wrote the framework in C++.  I would have liked to use a scripting language, but I knew that I would need to use a graphics library and a library to interpret shapefiles.  The only ones I found that looked reasonable were C-based libraries (Frank Warmerdam’s Shapelib library andThomas Boutell’s gd library).   I knew it was possible using a tool called SWIG, but I hadn’t ever used SWIG and had heard that it was touchy.  Doing it in C++ was guaranteed to be painful, but I knew what the limits of that pain were.  I didn’t know what the limits of pain of using SWIG would be.

Projection

I also had problems figuring out how to convert from latitude/longitude to pixel coordinates in the Google tile space.  At the time (December 2005), I had a hard time simply finding out what the mathematics of the Mercator transformation were.  (It is easier to find Mercator projection information now.)  I was able to figure out something that worked most of the time, but if you zoomed out past a certain level, there would be a consistent error in the y-coordinates.  The more you zoomed out, the bigger the error.  I’m pretty sure it’s some sort of rounding error.  I looked at it several times, trying to figure out where I could possibly have a roundoff error, but never did figure it out.  I just restricted how far people could zoom out.  (It also took a very long time to render tiles if you were way zoomed out, so it seemed reasonable to restrict it.)

Polygon intersection

I remember that I spent quite a lot of time on my polygon intersection code. I believe that I looked around the Web and didn’t find any helpful code, so developed it from scratch on little sleep. (Remember, I was doing this in a frantic, frantic hurry.) I ended up with eight comparisons that needed to be done for each polygon in the database for every tile. More on this later.

Rendering bug

The version I handed in had a bug where horizontal lines would show up at the bottom of tiles frequently, as you can see in the bottom left tile:

It was pretty obvious that the bug was my fault, as gd is a very mature and well-used graphics library.  My old office partner Carlos Pero had used it way back in 1994 to develop Carlos’ Coloring Book, so it was clear to me that the problem was my fault.

After I handed in my project, I spent quite a lot of time going through my code trying to figure out where the problem was with no luck.  Frustrated, I downloaded and built gd so that I could put breakpoints into the gd code.  Much to my surprise, I discovered that the bug was in the gd library!  I thus had to study and understand the gd code, fix it, report the bug (and patch), and of course blog about it so that other people wouldn’t have the same problem.

Pointing my code to the fixed gd

Then, in order to actually get the fix, I had to figure out how to statically link gd into my binaries. I like my ISP (Dreamhost) and wasn’t particularly interested in changing, but that meant I couldn’t use the system-installed gd libraries.  Statically linking wasn’t a huge deal, but it took me at least several hours to figure out which flag to insert where in my makefile to get it to build statically.  It was just one more thing.

Second pass

I have graduated, but haven’t found a job yet, so I decided to revamp my mapping framework. In addition to the aesthetic joy of making nice clean code:

• It would be an opportunity to learn and demonstrate competence in another technology.
• I had ideas for how I could improve the performance by pre-computing some things.
• With a more flexible framework, I would be able to do some cool new mashups that I figured would get me more exposure, and hence lead to some consulting jobs.

Language/libraries/database choice

Vancouver is a PHP town, so I thought I’d give PHP a shot. I expected that I might have to rewrite my code in C++ eventually, but that I could get the basics of my improved algorithms shaken out first.  (I’m not done yet, but so far, I have been very very pleased with that strategy.)

I also decided to use MySQL.  While the feeling in the GIS community is that the Postgres‘ GIS extensions (PostGIS) are better than the GIS extensions to MySQL, I can’t run Postgres on my ISP, and MySQL is used more than Postgres.

I had installed PHP4 and MySQL 4 on my home computer some time ago, when I was working on Mapeteria.  However, I recently upgraded my home Ubuntu installation to Hardy Heron, and PHP4 was no longer supported.  That meant I need to install a variety of packages, and I went through a process of downloading, trying, discovering I was missing a package, downloading/installing, discovering I was missing a package, lather, rinse, repeat.  I needed to install  mysql-server-5.0,  mysql-client-5.0, php5, php5-mcrypt, php5-cli, php5-gd, libgd2-xpm-dev, php5-mysql, and php5-curl.  I also spent some time trying to figure out why php5 wouldn’t run scripts that were in my cgi-bin directory before realizing/discovering that with mod_php, it was supposed to run from everywhere but the cgi-bin directory.

Note that I could have done all my development on my ISP’s machines, but that seemed clunky.  I knew I’d want to be able to develop offline at some point, so wanted to get it done sooner rather than later.  It’s also a little faster to develop on my local system.

I did a little bit of looking around for a graphics library, but stopped when I found that PHP had hooks to the gd library.  I knew that if gd had not yet incorporated my horizontal lines bug fix, then I might have to drop back to C++ in order to link in “my” gd, but I figured I could worry about that later.

Projection

I made a conscious decision to write my Mercator conversion code from scratch, without looking at my C++ code.  I did this because I didn’t want to be influenced in a way that might lead me to get the same error at way-zoomed-out that I did before.  I was able to equations on the Wikipedia Mercator page for transforming Mercator coordinates to X-Y coordinates, but those equations didn’t give a scale for the X-Y coordinates!  It took some trial and error to that out.

Data

For the initial development, I decided to use country boundaries instead of census tract boundaries. The code wouldn’t care which data it was using, and it would be nice to have tiles that would render faster when way-zoomed-out. I whipped up a script read a KML file with country boundaries (that I got from Valery Hronusov and used in my Mapeteria project) and loaded it into MySQL.  Unfortunately, I had real problems with precision.  I don’t remember whether it was PHP or MySQL, but I kept losing some precision in the latitude and longitude when I read and uploaded it.  I eventually converted to uploading integers that were 1,000,000 times the latitude and longitude, and so had no rounding difficulties.

One thing that helped me enormously when working on the projection algorithm was to gather actual data from Google.  I found a number of places on the Google maps where three territories (e.g. British Columbia, Alberta, and Montana) came together.  I would determine the latitude/longitude of those points, then figure out what the tile coordinates, pixel X, and pixel Y of that point were for various zoom levels.  That let me assemble high-quality test cases, which were absolutely essential in figuring out what the transformation algorithm should be, but it was very slow, boring, and tedious to collect that data.

Polygon intersection

When it came time to implement my polygon bounding box intersection code again, I looked at my old polygon intersection code again, saw that it took eight comparisons, and thought to myself, “That can’t be right!”  Indeed, it took me very little time to come up with a version with only four comparisons, (and was now able to find sources on the Web that describe that algorithm).

Stored procedures

One thing that I saw regularly in employment ads was a request for use of stored procedures, which became available with MySQL 5.  It seemed reasonable that using a stored procedure to calculate the bounding box intersection would be even faster, so I ran some timing tests.  In one, I used PHP to generate a complex WHERE clause string from eight values; in the other, I passed eight values to a stored procedure and used that in the WHERE clause.  Much to my suprise, it took almost 20 times more time to use the stored procedure!  I think I understand why, but it was interesting to discover that it was not always faster.

GIS extensions

My beloved husband had been harping on me to use the built-in GIS extensions.  I had been ignoring him because a large part of the point of this exercise was to learn more about MySQL, including stored procedures, but now that I found that the stored procedure was slow, it was time to time the built-in bounding box intersection routine.  If I stored the bounding box as a POLYGON type instead of as two coordinate pairs, then it took half the time.  Woot!

Rendering

I discovered that despite my having reported the horizontal lines bug fifteen months ago, the gd team hasn’t done anything with it yet.  Needless to say, this means that the version of libgd.a on Dreamhost has the bug in it. I thought about porting back to C++. I figured that porting back would probably take at minimum a week, and would raise the possibility of nasty pointer bugs, so it was worth spending a few days trying to get PHP to use my version of gd.

It is possible to compile your own version of PHP and use it, though it means using the CGI version of PHP instead of mod_php. I looked around for information on how to do that, and found a Dreamhost page on how to do so.. but failed utterly when I followed the directions. I almost gave up at that point, but sent a detailed message to Dreamhost customer support explaining what I was trying to do, why, and what was blocking me. On US Thanksgiving Day, I got a very thoughtful response back from Robert at Dreamhost customer support which pointed me at a different how-to-compile-PHP-on-Dreamhost page that ultimately proved successful.  (This is part of why I like Dreamhost and don’t really want to change ISPs.)

Compiling unfamiliar packages can be a real pain, and this was no different.  The Dreamhost page (on their user-generated wiki) had a few scripts that would do the install/build for me, but they weren’t the whole story.  Each of the scripts downloaded a number of projects (like openSSL, IMAP, CURL, etc) in compressed form, extracted the files, and built them.  The scripts were somewhat fragile — they would just break if something didn’t work right.  They were sometimes opaque — they didn’t always print an error message if something broke.  If there was a problem, they started over from the beginning, removing everything that had been downloaded and extracted.  Something small — like if the mirror site for mcrypt was so busy that the download timed out — would mean starting from scratch.  (I ended up iteratively commenting out large swaths of the scripts so that I wouldn’t have to redo work.)

There was some problem with the IMAP build having to do with SSL.  I finally changed one of the flags so that IMAP built without SSL — figuring that I was unlikely to be using this instance of PHP to do IMAP, let alone IMAP with SSL — but it took several false starts, each taking quite a long time to go through.

Finally, once I got it to build without my custom gd, I tried folding in my gd.  I uploaded my gd/.libs directory, but that wasn’t enough — it wanted the gd.h file.  I suppose I could have tried to figure out what it wanted, where it wanted it, but I figured it would be faster to just re-build gd on my Dreamhost account, then do a make install to some local directory. Uploading my source was fast and the build was slow but straightforward. However, I couldn’t figure out how to specify where the install should go. The makefiles were all autogenerated and very difficult to follow. I tried to figure out where in configure the install directory got set, but that too was hard to decipher. Finally, I just hand-edited the default installation directory. So there. That worked. PHP built!

Unfortunately, it wouldn’t run. It turned out that the installation script had a bug in it:

cp ${INSTALLDIR}/bin/php ${HOME}/${DOMAIN}/cgi-bin/php.cgi

instead of

cp ${INSTALLDIR}/bin/php.cgi ${HOME}/${DOMAIN}/cgi-bin/php.cgi

But finally, after all that, success!

Bottom line

So let me review what it took to get to tile rendering:

1. Choose a database and figure out how to extract data from it, requiring reading and learning.
2. Find and load boundary information into the database, requiring trial and error.
3. Choose a graphics library and figure out how to draw coloured polygons with it, requiring reading and learning.
4. Gather test cases for converting from latitude/longitude into Google coordinate system, requiring patience.
5. Figure out how to translate from latitude/longitude pairs into the Google coordinate system, requiring algorithmic skills.
6. Diagnose and fix a bug in a large-ish C graphics library, requiring skills debugging in C.
7. Download and install PHP and MySQL, requiring system administration skills.
8. Figure out how to build a custom PHP, requiring understanding of bash scripts and makefiles.

So now, I guess it isn’t that easy to generate tiles!

Note: there is an entirely different ecosystem for generating tiles, one that comes from the mainline GIS world, one that descends from the ESRI ecosystem. I expect that I could have used PostGIS and GeoTools with uDig look like fine tools, but they are complex tools with many many features.  Had I gone that route, I would have had to wade through a lot of documentation of features I didn’t care about.  (I also would have had to figure out which ISP to move to in order to get Postgres.)  I think that it would have taken me long enough to learn / install that ecosystem’s tools that it wouldn’t have been worth it for the relatively simple things that I needed to do.  Your milage may vary.