"Linux Gazette...making Linux just a little more fun!"

Using m4 to write HTML.

By Bob Hepple bhepple@pacific.net.sg

Contents:

• 1. Some limitations of HTML
• 2. Using m4
• 3. Examples of m4 macros
• 3.1 Sharing HTML elements across several page
• 3.2 Managing HTML elements that often change
• 3.3 Creating new text styles
• 3.4 Typing and mnemonic aids
• 3.5 Automatic numbering
• 3.6 Automatic date stamping
• 3.7 Generating Tables of Contents
• 3.7.1 Simple to understand TOC
• 3.7.2 Simple to use TOC
• 3.8 Simple tables
• 4. m4 gotchas
• 4.1 Gotcha 1 - quotes
• 4.2 Gotcha 2 - Word swallowing
• 4.3 Gotcha 3 - Comments
• 4.4 Gotcha 4 - Debugging
• 5. Conclusion
• 6. Files to download

1. Some limitations of HTML

I immediately turned to the WWW and looked up all sorts of tools - but quite honestly I was rather disappointed. Mostly, they were what I would call Typing Aids - instead of having to remember arcane incantations like <a href="link">text</a>, you are given a button or a magic keychord like ALT-CTRL-j which remembers the syntax and does all that nasty typing for you.

Linux to the rescue! HTML is built as ordinary text files and therefore the normal Linux text management tools can be used. This includes the revision control tools such as RCS and the text manipulation tools like awk, perl, etc. These offer significant help in version control and managing development by multiple users as well as in automating the process of extracting from a database and displaying the results (the classic "grep |sort |awk" pipeline).

The use of these tools with HTML is documented elsewhere, e.g. see Jim Weinrich's article in Linux Journal Issue 36, April 1997, "Using Perl to Check Web Links" which I'd highly recommend as yet another way to really flex those Linux muscles when writing HTML.

What I will cover here is a little work I've done recently with using m4 in maintaining HTML. The ideas can probably be extended to the more general SGML case very easily.

Contents

2. Using m4

Instead of editing *.html files, I create *.m4 files with my favourite text editor. These look something like this:

m4_include(stdlib.m4)
_HEADER(`This is my header')
<P>This is some plain text<P>
_HEAD1(`This is a main heading')
<P>This is some more plain text<P>
_TRAILER

The format is simple - just HTML code but you can now include files and add macros rather like in C. I use a convention that my new macros are in capitals and start with "_" to make them stand out from HTML language and to avoid name-space collisions.

The m4 file is then processed as follows to create an .html file e.g.

m4 -P <file.m4 >file.html

This is especially easy if you create a "makefile" to automate this in the usual way. Something like:

.SUFFIXES: .m4 .html
.m4.html:
	m4 -P $*.m4 >$*.html
default: index.html
*.html: stdlib.m4
all: default PROJECT1 PROJECT2
PROJECT1:
	(cd project2; make all)
PROJECT2:
	(cd project2; make all)

The most useful commands in m4 include the following which are very similar to the cpp equivalents (shown in brackets):

m4_include:

includes a common file into your HTML (#include)

m4_define:

defines an m4 variable (#define)

m4_ifdef:

a conditional (#ifdef)

Some other commands which are useful are:

m4_changecom:

change the m4 comment character (normally #)

m4_debugmode:

control error disgnostics

m4_traceon/off:

turn tracing on and off

m4_dnl:

comment

m4_incr, m4_decr:

simple arithmetic

m4_eval:

more general arithmetic

m4_esyscmd:

execute a Linux command and use the output

m4_divert(i):

This is a little complicated, so skip on first reading. It is a way of storing text for output at the end of normal processing - it will come in useful later, when we get to automatic numbering of headings. It sends output from m4 to a temporary file number i. At the end of processing, any text which was diverted is then output, in the order of the file number i. File number -1 is the bit bucket and can be used to comment out chunks of comments. File number 0 is the normal output stream. Thus, for example, you can `m4_divert' text to file 1 and it will only be output at the end.

Contents

3. Examples of m4 macros

In many "nests" of HTML pages, each page shares elements such as a button bar like this:

[Home] [Next] [Prev] [Index]

This is fairly easy to create in each page - the trouble is that if you make a change in the "standard" button-bar then you then have the tedious job of finding each occurance of it in every file and then manually make the changes.

With m4 we can more easily do this by putting the shared elements into an m4_include statement, just like C.

While I'm at it, I might as well also automate the naming of pages, perhaps by putting the following into an include file, say "button_bar.m4":

m4_define(`_BUTTON_BAR', 
	<a href="homepage.html">[Home]</a>
	<a href="$1">[Next]</a>
	<a href="$2">[Prev]</a>
	<a href="indexpage.html">[Index]</a>)

and then in the document itself:

m4_include button_bar.m4
_BUTTON_BAR(`page_after_this.html', 
	`page_before_this.html')

The $1 and $2 parameters in the macro definition are replaced by the strings in the macro call.

Contents

3.2 Managing HTML elements that often change

It is very troublesome to have items change in multiple HTML pages. For example, if your email address changes then you will need to change all references to the new address. Instead, with m4 you can do something like this in your stdlib.m4 file:

m4_define(`_EMAIL_ADDRESS', `MyName@foo.bar.com')

and then just put _EMAIL_ADDRESS in your m4 files.

A more substantial example comes from building strings up with multiple components, any of which may change as the page is developed. If, like me, you develop on one machine, test out the page and then upload to another machine with a totally different address then you could use the m4_ifdef command in your stdlib.m4 file (just like the #ifdef command in cpp):

m4_define(`_LOCAL')
.
.
m4_define(`_HOMEPAGE', 
	m4_ifdef(`_LOCAL', `//127.0.0.1/~YourAccount', 
		`http://ISP.com/~YourAccount'))

m4_define(`_PLUG', `<A REF="http://www.ssc.com/linux/">
	<IMG SRC="_HOMEPAGE/gif/powered.gif" 
	ALT="[Linux Information]"> </A>')

Note the careful use of quotes to prevent the variable _LOCAL from being expanded. _HOMEPAGE takes on different values according to whether the variable _LOCAL is defined or not. This can then ripple through the entire project as you make the pages.

In this example, _PLUG is a macro to advertise Linux. When you are testing your pages, you use the local version of _HOMEPAGE. When you are ready to upload, you can remove or comment out the _LOCAL definition like this:

m4_dnl m4_define(`_LOCAL')

... and then re-make.

Contents

3.3 Creating new text styles

m4_define(`_MYQUOTE',
	<BLOCKQUOTE><EM>$1</EM></BLOCKQUOTE>)

If, later, you decide you prefer <STRONG> instead of <EM> it is a simple matter to change the definition and then every _MYQUOTE paragraph falls into line with a quick make.

The classic guides to good HTML writing say things like "It is strongly recommended that you employ the logical styles such as <EM>...</EM> rather than the physical styles such as <I>...</I> in your documents." Curiously, the WYSIWYG editors for HTML generate purely physical styles. Using these m4 styles may be a good way to keep on using logical styles.

Contents

3.4 Typing and mnemonic aids

I don't depend on WYSIWYG editing (having been brought up on troff) but all the same I'm not averse to using help where it's available. There is a choice (and maybe it's a fine line) to be made between:

<BLOCKQUOTE><PRE><CODE>Some code you want to display.
</CODE></PRE></BLOCKQUOTE>

and:

_CODE(Some code you want to display.)

In this case, you would define _CODE like this:

m4_define(`_CODE',
	 <BLOCKQUOTE><PRE><CODE>$1</CODE></PRE></BLOCKQUOTE>)

Which version you prefer is a matter of taste and convenience although the m4 macro certainly saves some typing and ensures that HTML codes are not interleaved. Another example I like to use (I can never remember the syntax for links) is:

m4_define(`_LINK', <a href="$1">$2</a>)

Then,

<a href="URL_TO_SOMEWHERE">Click here to get to SOMEWHERE </a>

becomes:

_LINK(`URL_TO_SOMEWHERE', `Click here to get to SOMEWHERE')

Contents

3.5 Automatic numbering

m4 has a simple arithmetic facility with two operators m4_incr and m4_decr which act as you might expect - this can be used to create automatic numbering, perhaps for headings, e.g.:

m4_define(_CARDINAL,0)

m4_define(_H, `m4_define(`_CARDINAL',
	m4_incr(_CARDINAL))<H2>_CARDINAL.0 $1</H2>')

_H(First Heading)
_H(Second Heading)

This produces:

<H2>1.0 First Heading</H2>
<H2>2.0 Second Heading</H2>

Contents

3.6 Automatic date stamping

This page was updated on m4_esyscmd(date)

which produces:

This page was last updated on Fri May 9 10:35:03 HKT 1997

Of course, you could also use the date, revision and other facilities of revision control systems like RCS or SCCS, e.g. $Date: 2002/10/09 22:24:22 $.

Contents

3.7 Generating Tables of Contents

Using m4 allows you to define commonly repeated phrases and use them consistently - I hate repeating myself because I am lazy and because I make mistakes, so I find this feature absolutely key.

A good example of the power of m4 is in building a table of contents in a big page (like this one). This involves repeating the heading title in the table of contents and then in the text itself. This is tedious and error-prone especially when you change the titles. There are specialised tools for generating tables of contents from HTML pages but the simple facility provided by m4 is irresistable to me.

3.7.1 Simple to understand TOC

The following example is a fairly simple-minded Table of Contents generator. First, create some useful macros in stdlib.m4:

m4_define(`_LINK_TO_LABEL', <A HREF="#$1">$1</A>)
m4_define(`_SECTION_HEADER', <A NAME="$1"><H2>$1</H2></A>)

Then define all the section headings in a table at the start of the page body:

m4_define(`_DIFFICULTIES', `The difficulties of HTML')
m4_define(`_USING_M4', `Using <EM>m4</EM>')
m4_define(`_SHARING', `Sharing HTML Elements Across Several Pages')

Then build the table:

<UL><P>
	<LI> _LINK_TO_LABEL(_DIFFICULTIES)
	<LI> _LINK_TO_LABEL(_USING_M4)
	<LI> _LINK_TO_LABEL(_SHARING)
<UL>

Finally, write the text:

.
.
_SECTION_HEADER(_DIFFICULTIES)
.
.

The advantages of this approach are that if you change your headings you only need to change them in one place and the table of contents is automatically regenerated; also the links are guaranteed to work.

Hopefully, that simple version was fairly easy to understand.

Contents

3.7.2 Simple to use TOC

The Table of Contents generator that I normally use is a bit more complex and will require a little more study, but is much easier to use. It not only builds the Table, but it also automatically numbers the headings on the fly - up to 4 levels of numbering (e.g. section 3.2.1.3 - although this can be easily extended). It is very simple to use as follows:

1. Where you want the table to appear, call Start_TOC
2. at every heading use _H1(`Heading for level 1') or _H2(`Heading for level 2') as appropriate.
3. After the very last HTML code (probably after </HTML>), call End_TOC
4. and that's all!

The code for these macros is a little complex, so hold your breath:

m4_define(_Start_TOC,`<UL><P>m4_divert(-1)
  m4_define(`_H1_num',0)
  m4_define(`_H2_num',0)
  m4_define(`_H3_num',0)
  m4_define(`_H4_num',0)
  m4_divert(1)')

m4_define(_H1, `m4_divert(-1)
  m4_define(`_H1_num',m4_incr(_H1_num))
  m4_define(`_H2_num',0)
  m4_define(`_H3_num',0)
  m4_define(`_H4_num',0)
  m4_define(`_TOC_label',`_H1_num. $1')
  m4_divert(0)<LI><A HREF="#_TOC_label">_TOC_label</A>
  m4_divert(1)<A NAME="_TOC_label">
	<H2>_TOC_label</H2></A>')
.
.
[definitions for _H2, _H3 and _H4 are similar and are 
in the downloadable version of stdlib.m4]
.
.

m4_define(_End_TOC,`m4_divert(0)</UL><P>')

One restriction is that you should not use diversions within your text, unless you preserve the diversion to file 1 used by this TOC generator.

Contents

3.8 Simple tables

Other than Tables of Contents, many browsers support tabular information. Here are some funky macros as a short cut to producing these tables. First, an example of their use:

<CENTER>
_Start_Table(BORDER=5)
_Table_Hdr(,Apples, Oranges, Lemons)
_Table_Row(England,100,250,300)
_Table_Row(France,200,500,100)
_Table_Row(Germany,500,50,90)
_Table_Row(Spain,,23,2444)
_Table_Row(Denmark,,,20)
_End_Table
</CENTER>

...and now the code. Note that this example utilises m4's ability to recurse:

m4_dnl _Start_Table(Columns,TABLE parameters)
m4_dnl defaults are BORDER=1 CELLPADDING="1" CELLSPACING="1"
m4_dnl WIDTH="n" pixels or "n%" of screen width
m4_define(_Start_Table,`<TABLE $1>')

m4_define(`_Table_Hdr_Item', `<th>$1</th>
  m4_ifelse($#,1,,`_Table_Hdr_Item(m4_shift($@))')')

m4_define(`_Table_Row_Item', `<td>$1</td>
  m4_ifelse($#,1,,`_Table_Row_Item(m4_shift($@))')')

m4_define(`_Table_Hdr',`<tr>_Table_Hdr_Item($@)</tr>')
m4_define(`_Table_Row',`<tr>_Table_Row_Item($@)</tr>')

m4_define(`_End_Table',</TABLE>)

Contents

4. m4 gotchas

Unfortunately, m4 is not unremitting sweetness and light - it needs some taming and a little time spent on familiarisation will pay dividends. Definitive documentation is available (for example in emacs' info documentation system) but, without being a complete tutorial, here are a few tips based on my fiddling about with the thing.

4.1 Gotcha 1 - quotes

m4's quotation characters are the grave accent ` which starts the quote, and the acute accent ' which ends it. It may help to put all arguments to macros in quotes, e.g.

_HEAD1(`This is a heading')

The main reason for this is in case there are commas in an argument to a macro - m4 uses commas to separate macro parameters, e.g. _CODE(foo, bar) would print the foo but not the bar. _CODE(`foo, bar') works properly.

This becomes a little complicated when you nest macro calls as in the m4 source code for the examples in this paper - but that is rather an extreme case and normally you would not have to stoop to that level.

Contents

4.2 Gotcha 2 - Word swallowing

The worst problem with m4 is that some versions of it "swallow" key words that it recognises, such as "include", "format", "divert", "file", "gnu", "line", "regexp", "shift", "unix", "builtin" and "define". You can protect these words by putting them in m4 quotes, for example:

Smart people `include' Linux in their list
of computer essentials.

The trouble is, this is a royal pain to do - and you're likely to forget which words need protecting.

Another, safer way to protect keywords (my preference) is to invoke m4 with the -P or --prefix-builtins option. Then, all builtin macro names are modified so they all start with the prefix m4_ and ordinary words are left alone. For example, using this option, one should write m4_define instead of define (as shown in the examples in this article).

The only trouble is that not all versions of m4 support this option - notably some PC versions under M$-DOS. Maybe that's just another reason to steer clear of hack code on M$-DOS and stay with Linux!

Contents

4.3 Gotcha 3 - Comments

Comments in m4 are introduced with the # character - everything from the # to the end of the line is ignored by m4 and simply passed unchanged to the output. If you want to use # in the HTML page then you would need to quote it like this - `#'. Another option (my preference) is to change the m4 comment character to something exotic like this: m4_changecom(`[[[[') and not have to worry about `#' symbols in your text.

If you want to use comments in the m4 file which do not appear in the final HTML file, then the macro m4_dnl (dnl = Delete to New Line) is for you. This suppresses everything until the next newline.

m4_define(_NEWMACRO, `foo bar') m4_dnl This is a comment

Yet another way to have source code ignored is the m4_divert command. The main purpose of m4_divert is to save text in a temporary buffer for inclusion in the file later on - for example, in building a table of contents or index. However, if you divert to "-1" it just goes to limbo-land. This is useful for getting rid of the whitespace generated by the m4_define command, e.g.:

m4_divert(-1) diversion on
m4_define(this ...)
m4_define(that ...)
m4_divert	diversion turned off

Contents

4.4 Gotcha 4 - Debugging

Another tip for when things go wrong is to increase the amount of error diagnostics that m4 emits. The easiest way to do this is to add the following to your m4 file as debugging commands:

m4_debugmode(e)
m4_traceon
.
.
buggy lines
.
.
m4_traceoff

Contents

5. Conclusion

<!--#include file="junk.html" -->

The problem is that:

• The work of including and interpreting the include is done on the server-side before downloading and adds a big overhead as the server has to scan files for `include' statements.
• Consequently most servers (especially public ISP's) deactivate this feature.
• `include' is all you get - no macro substitution, no parameters to macros, no ifdef, etc, etc.

There are several other features of m4 that I have not yet exploited in my HTML ramblings so far, such as regular expressions and doubtless many others. It might be interesting to create a "standard" stdlib.m4 for general use with nice macros for general text processing and HTML functions. By all means download my version of stdlib.m4 as a base for your own hacking. I would be interested in hearing of useful macros and if there is enough interest, maybe a Mini-HOWTO could evolve from this paper.

There are many additional advantages in using Linux to develop HTML pages, far beyond the simple assistance given by the typical Typing Aids and WYSIWYG tools.

Certainly, this little hacker will go on using m4 until HTML catches up - I will then do my last make and drop back to using pure HTML.

I hope you enjoy these little tricks and encourage you to contribute your own. Happy hacking!

6. Files to download

using_m4.html	:this file
using_m4.m4	:m4 source
stdlib.m4	:Include file
makefile

Contents