Tuesday, July 31, 2007

restdoc

I've been working on a new tool for Zero, and I'm going to post some background information here (rather than the forum) because it will be easier to dig up later on.

Designing and implementing Zero applications usually starts with the creation of a REST API, which can then be documented using a REST table (or Gregorio table, as we sometimes call them). Comparing a REST table to a WSDL document is an excellent way to demonstrate ease-of-use differences between REST and WS-*. Upon reading one of these tables, one already has the mental model needed to write code that uses the documented service; REST tables manage to fit the important details (URI structure, method names, status codes, etc.) into a fairly spartan layout, while WSDL documents require pages of XML that is not really meant to be parsed by humans.

Of course, there is no official mapping of these REST APIs to the code that fulfills them, and so there is no RESTful code generation or documentation extraction like there is in WS-Land. I've had to create a lot of REST tables while working on Zero, and while it wasn't hard, it was kind of tedious because I had to duplicate the same information in my code comments[1]. Having documented my REST APIs manually and not wanting to do it again, I decided to create a tool that would read through my code, pull out JavaDoc-style comments, and turn them into a pretty HTML page full of REST tables.

RESTdoc works just like JavaDoc, except that it reads Groovy and PHP scripts that follow Zero's REST conventions[2]. RESTdoc's own JavaDoc provides the following overview:
To use RESTdoc, you must first include RESTdoc-style comments in your Zero scripts. RESTdoc comments are just like JavaDoc comments, with the following new tags: success, error, and format. The new tags are used to list the HTTP status codes for success and errors, as well as the expected data format in the request or response body for each method. RESTdoc will ignore any method that is not a Zero REST method (onList(), etc.) or that does not have RESTdoc-style documentation preceding its definition. Like Zero itself, RESTdoc only supports Groovy and PHP scripts, but it is possible to add other languages in the future.

To run RESTdoc from your Zero application's directory, just type:

restdoc

By default, RESTdoc will look in ./app/resources for Zero REST scripts and it will save the generated HTML page in ./docs/rest. If you want to run RESTdoc from a script that is not in your application's directory, you can use the input and output flags to override the defaults:

restdoc -input /my-app/app/resources -output /my-docs

Finally, RESTdoc does not produce any output if all goes well. You can turn on console logging using the verbose flag if you're having trouble debugging a problem:

restdoc -verbose

I've converted some of our sample applications to use RESTdoc-style comments, and so far, the tool works well. The HTML isn't beautiful, but neither is the stuff generated by JavaDoc. The important thing is that programmers can write comments using the familiar JavaDoc style and automatically get a nice document describing their application's RESTful public interface.

When I first got the idea to make this tool, I was pretty excited because I saw it as an opportunity to apply my elite compiler skillz. It's pretty hard to find a job where you get the chance to work on a new programming language, but creating developer tools often puts you in a situation where some level of formal AST-based parsing and analysis is needed to implement a feature correctly, and I enjoy those situations immensely. Of course, once I got started, I realized that parsing Groovy, PHP, and possibly Java source files with a complete compiler front end would require me to drag in a ton of dependencies, dependencies that zero.core already has but zero.tools does not. I was pretty sure the Zero tooling team would not be happy about my request to add N MB of Groovy and PHP-related JARs to their distribution, not to mention the possibility of adding Java-related JARs of questionable origin[3]. After looking at all my options, I realized that I would need to trade in my elite skillz for some dubious regex-based hacks if I was going to make a tool that was consumable by the rest of the team. Oh well.

The current code uses all sorts of string searching and pattern matching that would best be handled by a parser generator, but is instead handled by me. The net of this is a 32 KB binary, which includes the Ant task that enables RESTdoc to be called using Zero's command line interface. I'm not sure if RESTdoc will end up being included in Zero, but if it does, I'll be sure to update this post.

[1] Also, I hate making HTML tables. Hate.

[2] I've designed the code so that it can easily handle the addition of new languages to Zero, but I don't see that happening anytime soon.

[3] IBM Legal questions the origin of everything, no matter how obvious it may seem.

Labels: , ,

2 Comments:

Patrick Mueller said...

No examples?

I've parsed XML before with regex's; a shower is required after performing such acts.

July 31, 2007 2:28 PM

 
Dan said...

I just posted about this on the Zero forum, and there's a small screenshot and instructions on how to try the code (for those of us with access to SVN).

I didn't have to parse XML by hand, just Groovy and PHP scripts. I'm sure there are problems with it, but most of the edge cases are covered and I was able to get the tool working in a little over a day. Can't beat that.

July 31, 2007 2:55 PM

 

Post a Comment

<< Home