Literate programming as source code comments
For a long time, I've been interested in literate programming, an idea originaly developed by Donald. E Knuth in its CWEB system when programming the TeX system. His idea is quite simple: write a program not as a piece of code but as a book, where actual code is interleaved with pieces of documentation about that code. For example, for more than two years now I've been using Noweb to write the code of demexp. Noweb is very nice. Most notably, it is language agnostic so it can be used on all the different pieces of source you have in a project.
The main issue with literate programming is a technical one: the
mixed source code and documentation is written into a file,
e.g. file.ml.nw, from which are derived compilable code
(file.ml) and documentation
(file.ml.tex). This approach is not compatible with
proprietary IDE environments that your are forced to use for very
specific programming (e.g. Intel's IXP SDK for IXP2x00 Network
Processors or Xilinx's ISE for FPGA). Moreover, the more I think about
it, the more I believe that compilable source code should be the
reference point:
- it can thus be shared easily with other programmers that don't want to use literate programming tools;
- program order is important in certain languages (e.g. OCaml) and should be kept;
- most substitution mechanisms in literate programming tools I know about (Noweb, original Knuth's CWEB) is a very fragile macro systems that can lead to bugs.
I have thus started to think about embedding literate programming chunks as language comments. A special processing on source code extracts documentation from literate markup.
My original idea was quite simple: use a Wiki-like syntax to write the documentation, with a few set of tags:
- references and links within one source file or between source file;
- usual emphasized, bold, italics, code, etc. markups;
- inclusion of images;
- (optionally) syntax coloring and index creation.
One could even consider transforming an existing Wiki system storing its data in text file, like DokuWiki, to write the literate parts of the code.
More recently, Felix BREUER has proposed to use TeXmacs as a literate programming tool. I have proposed him my idea of documentation as source comments and he seems to like it, so we will try to experiment that idea.
To conclude, while I'm pretty sure that literate programming as source code comments is the way to go, I'm not sure of the right path between a wiki-like syntax and a TeXmacs approach. While TeXmacs approach is very seducing because TeXmacs is a full fledge scientific editor, with image creation and inclusion, formulas, references and links, its file format is not very readable and could clutter a lot the source code, making it unreadable. Felix has proposed an original solution to this issue: put the TeXmacs code at the end of the source file (albeit still embedded in it) and use a system of references for edition within TeXmacs. On the other hand, a wiki-like system is always readable, even for people not using the literate programming approach. Time and experiment will tell what is the right approach.
2006-02-11T14:14:26Z [] permanent link
