Mining twitter for useful information

The tale of a weekend project

I love the occasional weekend project. Getting something launched quickly can be rejuvenating, especially if you’re in the middle of a long term project like a startup.

In this blog post I’m going to walk through the code that is at the core of a small site I built in a weekend called twtspire.com. The point of the site is to mine twitter for app and website ideas that would make for fun and/or useful projects to work on. I figured people would be venting on twitter about pain points that could be solved with an app or just tossing out app ideas, and I wanted to surface those ideas so that I could build stuff people actually wanted.

Read More


Linux command line tricks you need to know: IO redirection and piping

The linux command line has a rich set of utilities and functionality. One of its beauties is that there are many utilities with a simple purpose that are in themselves very useful, but can also be combined to do an amazing array of jobs. This is a fundamental part of linux with tricks built into the bash command line shell to make that very easy.

Read More


Python HTML Parsing benchmarks (using Arc90’s Readability)

If you’re not familiar with the Readability project by Arc90, it is a bookmarklet that removes the clutter from web sites while you’re surfing the web. When I say clutter I mean the ads, widgets, navigation elements etc. that often surround the main content of the website. Readability gives you a pristine view of the actual content.

Read More


Scripting GNU Screen with Python

GNU Screen is one sweet linux program that has boosted my productivity a lot. You can read about some of its features here. As a developer who spends most of his time in the terminal, GNU Screen is a godsend. Notably, GNU Screen sessions persist even after you’ve lost a connection, so losing a work in progress before a save is one less thing to worry about, and you can also split the terminal screen into numerous windows using GNU Screen.

Read More


Page 1 of 1