toofishes.net

Welcome to toofishes.net

Welcome to my website, which is about 95% composed of my blog with a variety of other pages strewn about.

If you want to know more about who I am or what I do, check out the About Me page. Hint: I’m an open source and freelance software developer who updates his soapbox blog every once in a while with some helpful tips or hopefully relevant ramblings.

Last 10 Blog Posts

My New Company - Simple Bit Labs

In the past few months, I’ve started my own company to give freelance software development a try again. Simple Bit Labs is a registered Illinois LLC, and I’m hoping to do things a little differently than I first did 13 years ago when freelancing.

My goal is to not just engage in consulting-style work (although I’m open to it!), but instead, develop some software products that people might find useful. While the marketing hype would have you believe every software thing needs to turn into a unicorn startup, I don’t believe this is the case. There are plenty of neglected problems out there that could use a little help.

OpenVPN and ATA over Ethernet (AoE) interaction

Starting about 3 months ago, I started seeing some odd looking messages in my logs basically once a minute from OpenVPN.

Django syndication feed views and New Relic

Using New Relic together with Django? It is a fairly cool and noninvasive web performance monitoring situation that I’ve been testing out over the past month or so. One thing I figured out made it a lot easier to get unique web transaction name per syndication Feed subclass, which wasn’t happening out of the box. You’ll want to add a __name__ attribute to the class, giving you something like the following:

from django.contrib.syndication.views import Feed

class NewsFeed(Feed):
    __name__ = 'news_feed'

You can see an example of this in action for multiple feeds in the Archweb NewsFeed implementation. This made feeds show up as “/feeds:news_feed” and “/feeds:package_feed” rather than all showing up under one web transaction name.

IPv6 follow-up: OpenVPN transport over IPv6

Over 18 months ago I dual-stacked my Linode and made almost all services accessible via both IPv4 and IPv6. Although no where near as popular as IPv4, I do see hits on HTTP, HTTPS, NTP, and git ports on my machine over IPv6.

The one service I couldn’t set up over IPv6 was OpenVPN. This has finally changed with the release of 2.3.0. OpenVPN now supports IPv6 as both a transport and payload when using a tun style device. The only thing I was really looking for was the transport side, so I’ll detail the configuration of that here.

Concise summary query for pg_stat_statements

PostgreSQL 9.2 made the pg_stat_statements module a heck of a lot more useful now that it does query normalization. See the documentation for how to set the module up, as it is totally worth it if you are trying to track down the slow database queries in your application.

This is my goto query for finding statements worth optimizing. In the months I’ve been giving this a go, I’ve been able to quickly pinpoint the queries causing the most delay in returning data without being overwhelmed by one-off reporting queries.

MySQL and /tmp on tmpfs

Short but sweet post here. The short version is this: if you are currently running MySQL and don’t have /tmp on a tmpfs, you’re getting burned and performance will be extremely poor. The graphs below pretty much speak for themselves. We went from far too many slow queries and very high I/O on the only two disks this machine has (the box hosting the Arch Linux AUR) to almost no slow queries at all, normal I/O, and reduced CPU and IO wait.

Ten Years of Arch Linux

I’ll join the party, although I missed the actual date everyone keyed onto by a day. Thanks, Allan McRae, Andrea Scarpino, Daniel Isenmann, Dave Reisner, Ionuț Mircea Bîru, and Pierre Schmitz for already doing one of these. Also thanks to every current or former developer, TU, or active community member for all your work over the past ten years.

So here are the dates I can pull together:

  • 2006-05-24: First entry in pacman.log on the first computer I installed it on.
  • 2006-06-06: Registered on the Arch Linux forums.
  • 2006-09-14: First package (if I remember correctly), weka, submitted to the AUR.
  • 2006-11-15: First message to the pacman-dev mailing list.
  • 2006-12-12: Aaron Griffin asks me if I would be interested in being a pacman developer.
  • 2007-01-29: I officially become an Arch developer, focusing on pacman-related stuff.
  • 2007-03-25: pacman 3.0.0 is released, the first version to have libalpm included, and the first version I both contributed to and helped drive the final release.
  • 2008-06-04: Brought Allan McRae on as a developer for his work on pacman/makepkg as well as his attitude in the forums helping many a misguided user.
  • 2008-11-23: My first patch to archweb.
  • 2009 and 2010: “Stuff”, because I’m too lazy to look up any more events that happened.
  • 2011-03-24: Wrote my most commented-on blog post to date, The real story behind Arch Linux package signing.
  • 2011-06-13: Sponsored Dave Reisner (aka falconindy) to be a developer as he had contributed significantly to pacman and thought it would be great to get recognition for his work. In hindsight, perhaps this was a mistake. :)
  • 2011-10-13: Pacman 4.0.0 with package signing “finally” released.

python-pgpdump, a PGP packet parser library

PGP file formats and data are not the simplest thing to wrap your head around, so pgpdump is a very handy tool to have available. Although similar to gpg --list-packets, pgpdump output is a bit more verbose and descriptive.

pgpdump is a PGP packet visualizer which displays the packet format of OpenPGP (RFC 4880) and PGP version 2 (RFC 1991).

There is an online version available as well if you have never used it before and want to see what it is. Here is the output when parsing a detached signature file.

MySQL deadlocking on simple inserts

Poor MySQL, always getting picked on. However, it is pretty bad that a relational database in the year 2012 cannot handle simultaneous non-conflicting inserts on a table when transactions are involved.

First, some background. The InnoDB documentation does have a section dealing with deadlocks and possible ways to avoid them. There, it offers this wonderful fact:

You can get deadlocks even in the case of transactions that just insert or delete a single row.

i18n console output in C

Example console output

Most console applications start with the basics- no usage of gettext, English-only output, no need for aligned table columns. When this is a personal pet project, it isn’t a big deal, but once your application starts getting used on a widespread basis, i18n becomes an almost necessary requirement.

In pacman, Arch Linux’s package manager, we are all in with gettext and the entire application (outside of debug-only messages) can be translated. Our Transifex project page shows we currently can show output in 26 different languages, which is quite impressive.

Not seeing the post you are looking for? Head to the archives.

All Tags