Dev Diary 2020-07-07

Let’s talk about the Twitter API for a second. A tweet is not a “tweet,” it’s a “status”. If someone is following you, they are called a “follower”. If you follow them, in the API they’re called a “friend”. I can’t tell if it’s a cheerful sentiment, a historical anomaly, or developers writing independent parts of the API with no guiding naming spec. If you’ve worked on the Twitter API and have answers, there’s a free beer waiting for you.

My project today is scraping all the latest tweets that I want and dumping them into a nicely formatted PDF. Nothing fancy, although it would be great to somehow get this printed into a newspaper form-factor. Given the rate of newspapers folding (har), maybe I could buy a linotype press for real cheap.

One of the first snags I managed to hit was the rate-limiting. The Twitter API has a rolling 15 minute window limits, and my Twitter API usage up to this point has been primarily POSTs, which have never been more than a few a minute. What I hadn’t realized is that where they come down hard on API use hard is with the GET calls, which makes sense - they want folks to pony up for the premium APIs (I haven’t checked what their limits are, but the starting costs appeared to be several hundred a month - well out of my cost range for a hobby project). Fortunately, I don’t follow a ton of people, so for now I can get away with it (outside of the development process, anyways).

I managed to get the tweets exported to a Markdown file (around 10MB) and then sent it along to pandoc for PDFlatex export. And… well, my laptop fans went full tilt, and the kernel started killing processes left and right as it soaked up several gigs of RAM before dying. I guess there’s still a bit of work left to do on this one :-P