OpenSourcery

A while ago, I talked about my plans for Catalyst-Action-REST, and since then I've made some progress on refactoring to use roles instead of classes. Unfortunately, I've been stalled for the last few weeks, and I could use a nudge to get back on track.

My brain is stuck, basically, on naming issues. This feels a little silly and trivial, but it's something I've run aground on twice now, so I need to fix it somehow. The biggest problem is "serialize" -- Catalyst-Action-REST uses it both as a generic name for "things that either serialize or deserialize" and specifically for "things that serialize" (that is, "convert data to formatted text"). This means I end up wanting to use it to describe a bunch of things:

• Serialization formats, like YAML or JSON
• Common code shared by all serialization formats
• Actions implementing specific serialization formats
• The role for actions that do either serialization or deserialization
• The specific action role that does serialization

You can see the results of this sort of naming confusion in the current code on github: Catalyst::ActionRole::SerializeFormat is a pretty goofy namespace.

In my previous post I threatened to split serialization out into its own distribution. I'm not sure how this would overlap with something like Data::Serializer, which is currently used by some of C-A-REST's serialization formats. Maybe it'd be simpler to just punt to Data::Serializer for all the serialization and deserialization, but even then I'd have to keep some class names around for backwards compatibility.

Thoughts?

One of the talks I submitted for this year's YAPC::NA is on converting legacy CGI and mod_perl applications to run on Catalyst instead, focusing on ways to get Catalyst in between existing code and the webserver so that incremental refactoring is a possibility, rather than "maintain as-is" or "rewrite entirely". I've done this successfully for elementalClinic, and I wanted to find a few more applications, maybe with a little more complexity (the nice way of saying "insanity", in this context), to try my hand at.

Unfortunately, I failed. Instead of a crazy application that would test the limits of my ability to get Catalyst into those hard-to-reach places, I found Bugzilla. (Yes, Bugzilla has some very strange internals, as someone who has hacked on it has told me, but it's not a mixture of three separate templating engines/application systems spread across your webserver's document root.) The end result was anticlimactic -- I got Bugzilla running on Catalyst's standalone http server after only a few hours and some patches to Catalyst::Controller::CGIBin and HTTP::Request::AsCGI. I ended up spending as much time on making it easy for other people to duplicate my results as I did on the actual Bugzilla-specific code, which is, in its entirety:

around wrap_perl_cgi => sub {
  my $next = shift;
  my $code = shift->$next(@_);
  sub { $Bugzilla::_request_cache = {}; $code->(); Bugzilla::_cleanup() };
};

This is just a quick hack, and for normal users of Bugzilla it's probably not a big deal. If the Bugzilla maintainers wanted to start converting to Catalyst, though, it would be a great place to start; for example, it'd be trivial now to set up some Catalyst actions to clean up Bugzilla's urls from e.g. /show_bug.cgi?id=17 to /show_bug/17. That could start out as simple as this:

sub show_bug :Local :Args(1) {
  my ($self, $c, $id) = @_;
  $c->request->params->{id} = $id;
  $c->go('/CGI_show_bug_cgi');
}

Later, more logic could be brought into Catalyst controllers, and eventually the old show_bug.cgi path could exist only for backwards compatibility. (I don't know that the Bugzilla team cares about this at all, but it is the kind of thing that people often would like to do with their own applications.)

Not all of the Catalyst code is perfect. I did have to spend a few hours
fixing bugs in the CGI-wrapping controller code, and there's at least one more
I know of related to handling file uploads. Still, it's a great place to
start, and each bug fixed in the library is time you don't have to spend
working on when you want to modernize your applications.

Part of my job involves development on elementalClinic (emC). emC is a few years old, and has undergone a lot of architectural change; it started as a collection of CGI scripts, grew into a collection of scripts, templates, and modules, and made the transition to a mod_perl application in late 2006.

Now, mod_perl is fine for writing Apache modules; but I have a web application, and I don't need most of the power that mod_perl exposes. Being tied so tightly to Apache has caused some problems and inconveniences along the way. For example, any tool to deal with Perl module dependencies becomes much more complicated when one of them is mod_perl, and running tests through Apache has made certain kinds of test failures much more difficult to debug.

We'd talked internally about moving emC to Catalyst at some unspecified point in the future. Catalyst is mature, well-tested, and very powerful, and while I don't love everything about it, there are a lot of people working on it (besides, I have a commit bit, so if I don't like how things work it's partly my own fault anyway). I've been using Catalyst for another project recently after being away from it for a while, and decided that maybe it wouldn't take as much effort to start moving emC over as I thought.

So, on Thursday, I told Randall that I was going to take a few hours from the next couple of weeks, and see how far I got in just replacing the dispatch mechanism from emC -- converting the controllers and the session and so on is something we can do incrementally, but getting Catalyst in "underneath" everything is a big first step.

It turns out that the answer to "how far can I get" is "all tests passing"; with just over 5 hours' work, Catalyst is now sitting between the outside world and emC's controllers. The immediate benefit is that emC can run on any of Catalyst's engines: mod_perl, standalone, prefork, FastCGI, etc. The long-term benefits are no more maintenance of hand-rolled dispatching code and the ability to incrementally replace even more hand-rolled code with modules from CPAN.

A huge factor in this is that Matt Trout had already written Catalyst::Controller::WrapCGI, a controller for running existing CGI scripts seamlessly inside a Catalyst application. A shout out to Rafael Kitover, too, for helping me find and fix a problem with it under mod_perl, and then releasing a new version with my patch included.

My first target for Catalyzing emC's code is probably going to be the functional tests, which use WWW::Mechanize and currently each have to spin up their own Apache process -- Catalyst's test module can fake up HTTP requests in-process, which is really nice for speed during the test-edit-test cycle, and it already has a WWW::Mechanize-based wrapper, so I shouldn't need to change too much code to make use of it.

This morning I merged the Catalyst branch into emC trunk. I'm excited about how easy it was to slip Catalyst in underneath the existing emC code, and I'm looking forward to the changes it'll make possible in the future.

A while ago, I told Jay Shirley (in a moment of weakness) that I'd co-maintain the Catalyst-Action-REST distribution. Over the past week, I've closed several bugs, and I feel comfortable enough with the codebase to start making more sweeping changes.

C-A-REST is certainly useful in its current state, but it has its warts. At the same time, I've been thinking about the fact that Catalyst 5.80 (svn) is built on Moose, and about how Moose could possibly help with some of C-A-REST's rough spots.

Here's a list of things I'd like to improve in the near future, in rough order of how much I've thought them through:

• Using roles instead of classes for every type of object (Action, Request, and Controller) will make it much easier for C-A-REST's modules to play nicely with other Catalyst extensions. The old classes will stay around for backcompat, but new code will be able to use with 'Catalyst::Controller::Role::REST' instead. (See also Catalyst::Controller::ActionRole, thanks to Florian Ragwitz.)
• Serialize/Deserialize plugins are currently only loaded from under Catalyst::Action:: -- there's no way to look under MyApp::Action:: or any other namespace. There's also no good way to pass configuration to plugins (a common request). Finally, they all share a lot of code, like looking up data in the stash or reading the body, that needs to be refactored.
• Serialize/Deserialize are symmetrical -- that is, they're forced to use the same Content-Type. This is OK for basic use, but it puts a pretty hard barrier on extending the basic de/serialization mechanism for application-specific content types, and I'd like to make it easier to specify that certain content types are only used in one direction or another.
• REST status helpers have clunky syntax and don't cover enough HTTP status codes. Calling $self->method($c, ...) makes me sad; we've talked about using a REST view and/or a REST response role instead. Whatever the solution, the REST component of Catalyst::Controller::REST needs to feel less bolted-on than it currently does.
• REST and Serialize/Deserialize are unnecessarily conflated. I'm not entirely sure they should even be in the same distribution, but they definitely need to be easier to use separately. Switching to roles may be enough to make me happy, here.

Check out Catalyst-Action-REST on github, and let me know what you think.