Graphing the Glissando in "Rhapsody in Blue"
George Gershwin's "Rhapsody in Blue" famously opens with a trilling clarinet that rockets up more than two octaves in what's known as a "glissando," in which an instrument's pitch bends across a span of notes in a continuous climb or descent rather than stopping at discrete frequencies. Here's an example from the Columbia Symphony Orchestra, which probably sounds familiar:
Simulating the 2017 Solar Eclipse From Anywhere in the U.S.
This is one of my all-time favorite projects at TIME: A precisely accurate simulation of the total solar eclipse that traversed the entire continental United States on August 21, 2017, tracing a path in the umbra zone that totaled 174,250 square miles by our calculation. It was the event of the summer, and we wanted to reproduce what it would look like from anywhere.
How to fix Dropbox conflicts automatically
If you use Dropbox regularly, you have probably managed to screw it up at some point by modifying a file on two machines without syncing it between edits. When this happens, Dropbox will save both versions of the file and rename one of them like so:
Visualizing Algae Growth
The subreddit /r/dataisbeautiful/ runs a monthly visualization contest based on a different dataset each time. The Jan. 2018 contest challenges designers to visualize the growth rates of 19 species of algae under 8 different combinations of light and temperature, for a total of 152 data points.
Pascal's Triangle as a Galton Box
There's a wonderful math toy known as a "bean machine" or "Galton board" that demonstrates normal distributions though a simple mechanism: A large number of small metal balls—or beans, if you prefer—are fed through an staggered lattice of metal pegs, which, when the ball collides with the peg, causes it to drop either to the left or the right. Here's a simple demonstration from Wikipedia.
Google Ngram Data for Every Word in the CMU Pronunciation Dictionary
Google provides fantastic data on the frequency of words and ngrams in all the books it has scanned, in many cases going back centuries. The only drawback is that these files are enormous. The unigrams alone are 4GB when compressed, not including numbers and punctuation.
Visualizing Random Walks with Canvas
I don't use Canvas all that often on the client since SVG-based tools—mainly D3—are better suited for editorial data visualization. But I was thinking about random walks recently, which involve drawing thousands of dots and refreshing them several times a second, and figured it would be an interesting use case for Canvas's capabilities. Here's a little line that start's out with 10,000 points and updates 10 times a second, drawing 100 new random points and shedding the oldest 100. (Green is the newest, red is the oldest.)
A metalsmith plugin to prerender mathematical equations
I recently used MathJax for the first time to dynamically render equations in a post using the LaTeX format. The MathJax documentation recommends loading the script that converts the LaTeX format to beautifully formated math equations asynchronously. It works great, but there's typically a lag where a user sees the raw, ugly LaTeX for about a second before it gets rendered.
A curious property of simple functions
The other day, I was thinking about how, when you first learn calculus, the way you find the derivative of a function at a certain point is to find the slope of a line that passes through two very close points.
Getting states borders from counties using TopoJSON's merge function
I use the immortal Mike Bostock's TopoJSON for every map I make at TIME. Whenever we're looking at geographies more granular than states—usually counties, congressional districts or PUMAs—I like to have state boundaries drawn over the map to help orient the viewer.
Bargaining with Stack Overflow and other stages of developer grief
A few weeks ago, we published a little app on Time.com that uses unit-level Census data to show you how many single people in your city meet your dating standards. Using about 15 million records from the IPUMS project, we asked the user for his or her preference in education, income and other demographic questions and estimated what percent of the population matched those criteria.
Rendering exported musical scores in the browser
The transcriptions I've posted on Mechanical Scribe were generated by a Windows-only program called Mozart. I can't stand the way the embedded PDFs look on the page, and would much prefer to render the scores directly. Fortunately, there is a fantastic, actively developing project called VexFlow that is capable of doing just this.
The Joy of Screen Scraping
I was tickled pink the other day to see that over 1,000 people have downloaded the downcache module I maintain on NPM. The way it works is simple: You request a web page, just like you would with any other HTTP module in Node, and in addition to returning the page it saves a copy of the response to your machine. If you request it again, it loads the cached version instead of making another trip to the server.
A plugin for versioned posts in Metalsmith
Recreating the Atlantic's Netflix database with Node
The Atlantic published a delightful exploration last week of Netflix's surgically precise category descriptions, which include gems like "Critically-acclaimed Irreverent Crime Movies" or "Dramas starring Charlotte Rampling" (whoever she is). Because Netflix identifies the categories with integers in the URLs, Alexis Madrigal was able to scrape each one of the 76,897 unique categories names. It's data mining at its best.
Use your LESS files in Browserify with this Node module
Update: Since May 2017, node-lessify is now being actively maintained by GitHub user @dxcanas, who generously took over the project after I moved from browserify to webpack. I'm maintaining this post for archival reasons but encourage you to start with the current documentation on NPM.
We need new sentences that use every letter of the alphabet
I've been spending a lot of time with Google Webfonts recently, looking for a semi-coherent font ensemble for this site. By default, every font is displayed in the sentence "Grumpy wizards make toxic brew for the evil Queen and Jack."
Shostakovich Piano Concerto No. 2, Andante
The second movement of Dmitri Shostakovich's second piano concerto is one of the most hauntingly beautiful pieces ever written. I arranged the minimal score for solo piano my senior year of college for a piano recital and have tinkered with it a bit here and there since.
Synesthete.org's incredible grapheme-color synesthesia test
For as long as I can remember, I have associated specific colors with every number and letter. I first encountered a description of this condition in Richard Feynman's "What Do You Care What Other People Think?":
The most popular two-letter combos in English
I was pondering how one would make a premium gibberish generator the other day, which led to a short investigation of which pairings of letters are legal in English and how common each combination is. This diagram shows a basic heat map of letter combinations, using '^' to mark the beginning of a word and '$' to mark the end. The data here is extracted from every item in the "2 of 12" list of dictionary words available on Kevin's Word List page.
Using D3 with native HTML: DIVs as datavis
Last week, the White House released 100 pages of printed emails documenting the intelligence community's public response to the Sept. 11, 2012 attacks on the American diplomatic compound in Libya. There are 91 unique messages in the documents and a high level of redundancy due to long reply-chains being printed multiple times.
What to do with 100 pages of emails? Make an inbox
Facing pressure from Republicans in Congress, the Obama administration released 100 pages of printed out emails in May relating to its crafting of talking points about the Sept. 11, 2012 attacks in Benghazi. The emails were literally printed and given to select reporters in white binders.
The dataviz-themed birthday cake
I was put in charge of decorating my birthday cake this year, so I attempted to combine my love for data visualization with my love for the Phillies. Here's the result:
Guessing the beat from user-entered musical intervals
As I mentioned a few days ago, I'm a big fan of Soundslice, a project by Adrian Holovaty and PJ Macklin for creating interactive guitar tabs. One of the coolest features is the ability to tap out measures on the computer keyboard as a recording of the sound you're transcribing plays, thus marking out measures to then fill in with chords.
Using a Raspberry Pi for time-lapse photography
Putting aside for a moment why a 30-year-old has an ant farm, I thought I would post the code I use to take minute-by-minute photographs of the colony as it grows:
Binify + D3 = Gorgeous honeycomb maps
Most Americans prefer to huddle together around urban areas, which raises all sorts of problems for map-based visualizations. Coloring regions according to a data value, known as a choropleth map, leaves the map maker beholden to arbitrary political boundaries and, at the county level, pixel-wide polygons in parts of the Northeast. Many publications prefer to place dots proportional in area to the data values over the center of each county, which inevitably produces overlapping circles in these same congested regions. Here's a particularly atrocious example of that strategy I once made at Slate:
Modern Jazz Quartet's Air on the G String
Any of us who made $50 on weekends in high school playing weddings dread Bach's "Air for the G String" (Orchestral Suite No. 3 in D Major, BWV 1068) nearly as much as 'Pachelbel's Canon.' But Milt Jackson's dazzling solo over this first half of this old standby, followed by John Louis's devotional variation on the second half, make this recording among my all-time favorites.
We need a Wikipedia for music transcriptions
Transcribing the improvised solos of the greats is a core part of jazz education, but it's rare that people share their (often hand-written) notes on, say, exactly what Clifford was doing in that second chorus of "Step Lightly." (Let me know if you need that one. I might still have it.)
Most people refer to the sort of work you see on Mechanical Scribe as "interactive multimedia" or "interactive graphics." I've always liked the term "hypermedia" for these projects. The term was coined alongside "hypertext" in 1963 by a man named Ted Nelson, but only one would stick. "Hypertext"—literally "beyond text"—is etched in the 'H' and 'T' of HTML, but you don't often hear the second term. Nelson commented on this in his book Literary Machines: