Urban/Rural Populations

More evidence that modern American politics are driven more by the urban/rural divide than any other single factor. https://t.co/ohPeY3Onw5

— Dr. John Barentine (@JohnBarentine) October 24, 2016

I saw this tweet this morning as part of a thread, and it didn't quite seem right to me.  The urban/rural factor is certainly one component, but it seems like race and income should also be major factors.  Unfortunately I didn't fully dig through the census site to find a data set that contains all those variables, so this is just an analysis of the census county based urbanization statistics and the 2012 presidential election outcomes on a county basis.  To be completely fair, this is only an analysis of the subset of those two data sets that have identical spelling/spacing for the county names, so 3069 counties from the 3222 available in the census sample.  A quick check seems to indicate that Alaska and Puerto Rice aren't matched between the two.

In any case, plotting the vote fraction for Barack Obama as a function of urban population fraction doesn't show a overly strong trend:

There is a lot of scatter in this relation, but there is an uptick for the most urban counties.

Smoothing the data by taking medians/robust sigma values in 0.02 wide bins shows the uptick clearer.

The purple "fit" is simply a constant with a break to linear placed to match the binned data.

This suggests that for the vast majority of counties, there's little change, but once a certain threshold of urbanization is reached, the median county becomes more likely to vote Democratic as it urbanizes.  A fuller description would try to disentangle this effect from racial, economic, educational, and simple population based components.

One interesting thing to note is that only about a third of the country lives in a county below that break point.  This plot is just the CDF of the population data plotted as a function of the urbanization metric.

How does bedtime change with age?

I saw something earlier today that made me wonder how people's bedtime changes as they get older.  A quick google search pointed me to this study, which is probably the best/easiest to find data set that I'm likely to find just sitting online waiting.  There are a few obvious flaws:

1. It relies on users of the Jawbone UP for the data sample.  I have never heard of this device, and I'm guessing a lot of people are in the same boat.
2. Of those that do know of the Jawbone UP, they're probably younger than the average person, just because young people tend to use new tech at a higher rate.
3. There's likely class/income/etc. biases, as not everyone has the money to spend on a $50 fitness doodle.
4. There isn't actually any age data in the data set.

That last one isn't really that big of a show stopper.  

First step, scan through the source code to find the file that actually contains the data being used for the interactive map.  It's called counties.prod_.js, and nicely lists the county and state, bedtime in 12-hour format (minus 7, likely to ensure that there isn't a problem at the midnight boundary), as well as some other data I parsed out and saved in case I want to revisit something later.

Second step, trawl through the Census data for a county-by-county population breakdown, with age information.  That's here (although the full country data is actually 112MB, not the 11MB the page claims).  Then it's just a matter of pulling out the population data for 2015 (the closest match to the sleep data), setting the age for entire age groups at the midpoint, and calculating the weighted average age for each county (using the population in the age group as the weight).

So what's the result?

Other than me not truncating the best fit line.

The answer seems to be "Yes.  Bedtime is slightly earlier for older counties."  There's a bit of a plateau in the 16-20 range, but there's a reasonable decrease, even with the scatter.

Of course, using the full population isn't really the best, since the users of the Jawbone UP are likely adults, and not kids.  Redoing this analysis with just people older than 20 (due to the way the census data is binned into 5-year groups):

Basically the same, just shifted to an older age.

Working to get a consistent set of census data.

It's annoying, because the census doesn't have a single format that they use for all historical data.

Particularly in the effectively arbitrary old age cuts.  Why were they fine during the 80s, then slightly worse in the 90s, and then really bad in the 2000s?  No clue, but that's the data I have from the census, so that's what I'm using.  

The gap is from the lack of data between 2010 when the previous decade estimates stop and 2014 when the future projections begin.

This is kind of interesting too.  I initially started just plotting the population with age=0, with the intent to visualize generations.  The baby boom is really obvious in the purple curve.  I then added samples at different ages, lagged to use a consistent time base.  This thought this would give a probe of immigration, but that doesn't seem to be the case, as there aren't any major gaps in the first three samples in the 1850-1900 range.  I think the sag in the age=60 is just life expectancy issues.  That's also clearly apparent in the beyond 2000 area as well.  The projection yielding the 2014-2060 isn't predicting that to improve too much it seems.

This is kind of like reruns. Maybe it's a "remastered" post?

First up, killing the Supreme Court.  Again.  But still with numbers and statistics, because that's the best way to do things.  Assume the Senate decides to stop being dumb.  Then, Merrick Garland gets a hearing and since he's basically fine, he gets a seat on the Supreme Court.  Since my least favorite justice is dead.

So here's the cumulative "how many justices are alive" plot.  Honestly, according to this, if the Senate doesn't stop acting like children, Obama might have two more people to appoint before the end of his term.  It's good that the Republicans aren't running serious options this year, since that sets up a good shift when they don't become president.

And the by name individual plot.  I've seen a lot of stuff talking about how Garland is "already old" so it "doesn't matter" if he gets confirmed or not.  This is stupid.  The cumulative plot clearly shows that the next president is very important for determining the Supreme Court's future.

In any case, he's younger than the median justice, and is likely to be on the court for another ~15 years.  Or, you know, the next four presidential terms.  Also, it's interesting to note the benefit of appointing women to the court.  Roberts was born in 1955, and Sotomayor in 1954.  That's the unit the script uses for sorting the key.  But, looking at the graph, Sotomayor is likely to be on the court ~3 years longer.  I should also enable the grid display next time I do this.

The Simpsons cast vs aging.

I came home, watched a Simpson's episode that was sufficiently bland that I sat down and reimplemented the thing I did for the Supreme Court for a generic data file.  Feeding the main voice actor cast for the Simpson's into it (and quantizing on years because working out partial years would have taken more time than I wanted to spend.

Individually.

Summed.

Based on this, The Simpson's is unlikely to lose any of the main cast until 2020 at the earliest.  Assuming they keep going after that, they can continue for about another twenty years before they lose half of the cast.  However, Harry Shearer is that first line, and he voices about a third of all characters on the show.  That would probably be hard to recover from.

I accuse Mr. Green of committing the crime in the Ballroom with the Lead Pipe!

Or, more to the actual topic of this post, I accuse old people of opposing progress with their not dying.

Confused yet?

So I came across something today that pointed me to this survey of gay marriage polling tracks over time.  I then noticed that this page also includes historical trends for interracial marriage as well.

My obvious question to myself was, "how much of that trend can you explain simply by assuming things are improving because old people die?"

That led to a search for detailed cohort data, because I need to know when people are dying.  Google google google, and I find this page, which claims the image is from the Census people, although they've since rearranged their webpage so the link doesn't work anymore.  I'm accepting that, because who lies about cohort data?  Crazy statisticians, I guess.

Here's the cohort image, copied in case that link disappears too.

I digitized that image, and then write a perl script to calculate the fraction of the population on a given date that were born before a certain year.  This number subtracted from 100 gives the fraction born after that date.  What does that look like?

Due to the five-year blocking of the cohorts, there's a stair-step effect.  Interpolate a best fit line through that data, basically through the midpoints of each step.

Huh.  That worked far better than I expected.  Basically, ignoring all other factors, you can pretty much explain the entire trend if you just assume that everyone born after 1960 (who would have been kids during the Civil Rights Movement) is cool with interracial marriage, and that everyone born after 1980 (I'm going to use the Simpsons as my cultural shift here, because I don't want to go with "people my age") is cool with gay marriage.

The Supreme Court

Specifically, killing them.

Hang on there, Secret Service.  We're doing it statistically, so it's just an math problem.  No one is actually getting hurt, and I mostly did it to see how soon it's likely that there will be a vacancy due to death.  Someone could resign at any time, and that would make a vacancy, but that's not something that can really be predicted.  Death is a bit tricky, too, but due to the fact that despite the best attempts of some people, we still have a government that collects piles of data, so we can use the cohort life expectancy data from the Social Security Administration.

From that data, we're going to use the "High-cost" columns, and use the "at age 65" data.  The logic behind this is based on two facts.  First, the majority of the difference between the "at birth" and "at age 65" expectancies is due to infant mortality and things like that.  If you survive to 65, you didn't die as a baby.  The supreme court is universally comprised of people who did not die as a baby.  The choice of the "High-cost" is that being a supreme court justice is a pretty good job, so they're likely to have good health care.  The final assumption I have to make is that people born before 1940 have life expectancies similar to those born in 1940.  The table only goes back so far, and I don't really want to extrapolate it.

The next assumption is that the probability of surviving a year is based on the logistic function, largely due to a lack of better options.  The form I chose was P(age) = 1 / (1 + exp(-(expectancy - age))).  This sets the 50% probability at the expectancy age.  I could scale it, but I don't have any good idea of how to do that, so I'm just letting that be 1.0.

Using this table of birth dates and ages, we can make the following plot:

This shows the probability that that justice is alive.  I'm sorry that this claims Ruth Bader Ginsburg is 52% dead.

Ok, using these probabilities, we can plot up the number of justices statistically alive by simply summing these curves:

This suggests that (barring the Senate being jerks), Obama is likely to appoint two justices.  His successor falls into a generational gap between the Breyer and Thomas, and isn't likely to appoint anyone, even in an eight year term.  It's fairly linear over the next five terms, with about one each term.  This assumes that all newly appointed justices are younger than the current youngest, and that we're only concerned with the current set.

So again, barring the Senate terrible, we should have a significantly more liberal court within the next four years.

Batman and Robin are happy about that fact, too.

I seem to always have trouble on my return flights from Austin.

I seem to always have trouble on my return flights from Austin.  I've been stuck in Denver and Phoenix, and I was reasonably sure that I was going to end up stuck in San Francisco as well.

I arrived at AUS with plenty of time to spare, and went through the "Premiere Line" for security, which means I get to skip over all the waiting and go directly to the driver's license check.  Yay, frequent flier benefits!

I then got pizza:

which put me behind the worst fast food orderer I've seen in a long time.  First he didn't know what pizza he wanted. Then, after choosing and moving on to pay, he decided he wanted a second slice, and asked if it could just be put into the same box.  Since a box holds one slice, that couldn't happen, so he accepted the second box.

Anyway, by the time I got to the gate, I noticed an important fact: there was no plane connected to the jetbridge.  This continued for almost an hour, until the flight came in late, leading to the second problem: there was no crew to fly the now-available plane to SFO.  I've never been able to understand how airlines can manage their logistics so poorly.  You know you have a flight at airport X at time T.  That flight requires a set of N crew members, who need to be replaced after some number of hours.  Therefore, you need some number of crew sets k = ceil(dT / H).  Not having a crew I think has been the problem in both my Denver and Phoenix flight issues.

Upon arriving at SFO, I discovered the actual reason for the delay for the flight: the airport only had one open runway, and all traffic had to use that.  I didn't know anything about this, suggesting that United didn't do a very good job of notifying travelers.  The one benefit of this was that all flights were delayed, so leaving Austin an hour late didn't decrease my layover very much.  I went to the gate, and saw what I thought was my name noting that I'd been given an upgrade to first class, as seat 3-A was listed for someone with my same first initial and first three characters of my last name.

So, using this set of data for last name frequencies, and this data for first names (assuming first initial fractions are reasonably constant over time, so I don't need to convolve an age cohort function against time variable first names), suggests that my first name is more likely than random (0.068126 compared to 1/26 = 0.038462), but that my truncated last name isn't that common (0.0024997 compared to SMI = 0.010087).  Therefore, randomly selecting from the US population gives me a upgrade name that has a probability of 1.7030e-04.  Now, confining this to a plane makes it a birthday problem, and this seat map suggests a United 757-200 has 188 seats.  This leads to a probability that someone else on the plane has the same upgrade name as me of 0.031512 (Edit: For comparison, Roger Smith has a probability of 0.10684, significantly worse than me.) This is higher than I would have thought, but since it's a birthday problem, that's pretty much always the case.  Still, this would give me a ~97% confidence that I had the upgrade to first class, meaning I would be able to count on a meal on the flight.

I had some emergency chocolate that I'd packed before leaving home, and I discovered it while sitting there.  It had a raspberry filling.

Tl;dr: I didn't get the upgrade, this flight fell into the 3.15%.

In any case, I made it home without much trouble (despite all that), and was able to get to see a really nice sunset over the Pacific Ocean:

I'm sure the clouds have a specific name too. Altocumulus?  Is that right?