toofishes.net

Archweb gets a refresher

I finally got around to giving the site that is the face of Arch Linux its long-deserved refresher today. Of course, you probably just clicked that link and went “I don’t see anything new!”. Most of what I coded, merged, and deployed today was the behind the scenes stuff. However, it should help set up any future usability changes we want to make and make development in general a heck of a lot easier.

The biggest drawback up to this point with the site was its two-faced nature. It had been split a while back into two seperate archweb_pub and archweb_dev Django codebases. This was a bit before my time, but I believe it was for caching concerns. We want the main site to be heavily cached as it does get hit relatively heavily (side note: I forgot to enable caching right away today after deploying and it was quite noticeable). However, the developer site was meant to be uncached so when doing things like editing news items, you could immediately see the results of your work.

The fact that these two sites served slightly different audiences wasn’t a problem. The problem came with keeping them in sync. They shared the same database, and thus the same model classes and even several helper functions and views. Thus making a change to the models was a huge chore as you would have to make it in both places (copy and paste, oh no!), then fix up the code, and then deploy them both…at the same time. Top that all off with the fact that no migration framework was in place, and you have a project that doesn’t exactly scream “fun”.

Well this weekend I finally decided to get this fixed on the live site. I should say at this point that Ismael Carnales, an Arch community member, started the hardest part of this work back in November by undertaking the task of merging the two applications back together. He put nearly 30 git patches together to accomplish this task. I talked with him about a lot of this stuff back then, and pulled in his work after we worked though a few of the trickier issues. However, his work sat for a long time without actually being deployed (pretty much all my fault).

Last week, Evangelos Foutras, an Arch Linux Trusted User, sent a few patches to the ML for archweb. This was a bit of a kick in the ass for me to get moving on this again. I looked at the patches, they looked good, and I pulled them into this new merged code base. I then added a few small patches and fixes of my own, and got the new, merged codebase deployed to our developer site yesterday.

Today I woke up to another email with a few more proposed patches, including the integration of south, a Django schema migrations tool, into the project. I grabbed these and ran with them as it would help fix a long-outstanding bug that bothered me- the inability to use foreign keys correctly with package maintainers. I then made more progress in a day than I had in six months on this web app, fixing quite a few of the easier bugs in Flyspray (FS#12752, FS#13166, FS#16752, FS#17141, FS#17144) as well as scratched a few other itches.

I’d like to thank Ismael and Evangelos for contributing a ton of work to this release that should get us pointed in a good direction. I think the project will be easier to pick up and run with now that it isn’t two separate codebases, and the addition of south should make it less scary to change the models and have to worry about changing the data.

Next things that are on the radar are (1) figure out a better way to do package maintainers, and (2) try to get file lists back into the database and thus back on the website.

Tags

See Also