18 Jul 2023

expandfile Features for Multicians.org

It's my program, so I added various features to help maintain my web pages for multicians.org, the first application for expandfile. This note describes the features of expandfile that support the multicians website.

Enabling these features

These features of expandfile are disabled by default. To enable them, set the variable %[_xf_expand_multics]% to 'all' or 'nosql'.

Either setting enables the formatting macros {: :} {= =} {+ +} {- -}.
They wrap their argument with a SPAN tag specifying a CSS style specified in the configuration.

The all setting also enables the link generation macros {{ }} {[ ]} {@ @} {! !} {* *}.
These macros access a MySQL database at expansion time and generate <A HREF=...> links to targets specified in Multics-specific database tables. Using these macros requires that a database server is running and that MySQL server access parameters are set. See the description of the sqlloop builtin. Parameter values described below are used in macro expansions.

You can set _xf_expand_multics for an invocation of expandfile by setting it in the command line:

Command
expandfile _xf_expand_multics=all foo.htmx > foo.html

Alternatively, you can set this value for invocations of expandfile by setting it in a configuration file (useful in Makefile):

Contents of config.htmi
... %[*set,&_xf_expand_multics,=all]% ...
Command
expandfile config.htmi foo.htmx > foo.html

Formatting Macros

Macro constructs are included in expandfile to simplify some repetitive formatting.

To prevent these macros from being expanded and interpret them as literal text, precede them by a backslash (\). For example, \{\: ... :}, \{\= ... =}, \{\+ ... +}, \{\- ... -}, \{\* ... *}, \{\[ ... ]}, \{\{ ... }}, \{\! ... !}, \{\@ ... @} .

Text Formatting Macros

Four of these macros format text strings: they wrap a string of characters with a SPAN tag that references a configuration-specified CSS style.

You Write Translation Default
{:string:} <span class="%[_xf_bracketcolonclass]%">string</span> bracketcolonclass (cmd)
{=string=} <span class="%[_xf_bracketequalclass]%">string</span> bracketequalclass (pathname)
{+string+} <span class="%[_xf_bracketplusclass]%">string</span> bracketplusclass (code)
{-string-} <span class="%[_xf_bracketminusclass]%">string</span> bracketminusclass (special)

Multics Link Generation Macros

Four more macros generate HREF tags. The argument string contains a keyword with no spaces, a space, and a string. The keyword is looked up in a Multics specific MySQL table for the macro, and a formatted hyperlink is generated, with the string as anchor text. All of these sequences require that SQL database parameters be defined in the expandfile configuration in order to access MySQL. In addition, each sequence uses other configuration values to specify table and column names used in the query that looks up the values.

You Write Link type Translation Table referenced Parameters Notes
{[id anchor_string]} Multicians list <a href="peoplefile.html#id" title="(did value)">anchor_string</a> m multicianstable (m), peoplefile (multicians.html), multicianskeycol (nametag), multiciansvalcol (did)  
{{word anchor_string}} Multics glossary <a href="mgX.html#id" title="(summary)">anchor_string</a> glossary glosstbl (glossary), glossbase (mg), glossvalcol (def), glosskeycol (tag) X is first letter of word. "summary" is first 9 words of definition.
{!id anchor_string!} External reference <a href="(exturl value)" title="...">anchor_string</a> extref extreftable (extref), extrefkeycol (extname), extrefvalcol (exturl), newwindow (target="_blank"), tinyglob (globe image)  
{@htmlfile anchor_string@} Internal reference <a href="htmlfile" title="(description)">anchor_string</a> pages loclinktable (pages), loclinkkeycol (filename) Uses columns nextstory, kind, author, title, description of pages table. If the htmlfile begins with minus, look up a link to the "nextstory".

These macros can be nested: for example, one can write {{execcom {:exec_com:}}} to reference the Glossary entry for the exec_com command, formatting the command name as a command.

One additional macro is not yet implemented, but is reserved for future development when we figure out how to make it work.

You Write Translation Parameter
{*program_name*} (reference to Multics source code at web.mit.edu) multicssourceroot (http://web.mit.edu/multics-history/source/Multics/ldd/)

*callv Macros

An HTMX source file can invoke the *callv builtin function to expand a block as a macro with arguments. This feature can be used to simplify page source files and avoid formatting errors. The macro library file htmxlib.htmi provides general macros for referencing graphics and formatting numbers. Macros like firstnonempty, wrap, and fileinfo found in htmxlib.htmi are used to simplify the source of various pages.

Some pages' source files define *callv macros used only on that page, to simplify repetitive formatting.

Multics *callv Macros for Picture Insertion

Some multicians.org web pages that insert a graphic use macros like getimgdiv, getimgpopdiv, getimgdiv75, and getimgpopdiv75 from htmxlib.htmi to insert graphics into the page. These macros handle High DPI screens correctly when -2x versions of photos are available. (For example, look at Phase One.)

Some multicians.org web pages use the jQuery script "FancyBox" to display picture galleries, using a thumbnail image that shows a larger image when clicked. The pages include the library file usefancybox.htmi in the HEAD section of the page, and then each picture is specified with the *callv macro fancybox_li found in htmxlib.htmi. (For example, look at Project MAC 25th Anniversary.)

Some multicians.org web pages use the JavaScript file lightbox.js to display a small picture and pop up a bigger one on click. The pages include the library file use-dhtml-lightbox.htmi in the HEAD section of the page, and dhtml-lightbox.htmi in the BODY, and then each picture is specified with the *callv macro lightbox_li found in htmxlib.htmi. (For example, look at It Can Be Done.)

Multics *callv Macros for Links to External Pages

The multicians.org web site uses library macros for generating HREF links to the external URL of documents listed in the Bibliography. For example, there are various formats for links to bibliography items. Here are some examples of macro calls:

writeformattedtable
%[*callv,bibliolink,="Mac100",="Project MAC Progress Report I"]%Project MAC Progress Report Ibiblio
%[*callv,bibliolink2,="Mac100"]%Project MAC Progress Report I, July 1963 - July 1964, Massachusetts Institute of Technology, Cambridge MA, July 1964, DTIC AD-465088biblio
%[*callv,mtblink1,="MTB-017"]%MTB-017 The Storage Problem (1973-11-21)biblio
%[*callv,mtblink2,="MTB-017"]%MTB-017biblio

The macro bitsaversmultics in mxlib.htmi generates a link to Multics manuals stored at bitsavers.org. The configuration value conf_bitsavers_multics points to the root of the Multics document tree at that site.

The macros mitsourcearc, mitsourcefile, and mitinfoseg in mxlib.htmi generate links to Multics source files stored at http://web.mit.edu/multics-history/source/Multics. The configuration value conf_mhs_inf points to the URL of the Multics info segment directory. The configuration value conf_mhs_src points to the URL of the Multics source directory. The configuration value conf_mhs_ldd points to the URL of the Multics LDD directory.

Configuration Values

The Multics text formatting macros use some values that I set in config.htmi: 'bracketcolonclass' 'bracketequalclass' 'bracketplusclass' and 'bracketminusclass' were mentioned above. The macros that do MySQL lookup need the database parameters from config.htmi '_xf_hostname, '_xf_database', _xf_username', and '_xf_password' in order to connect to MySQL. In addition, the other tables and columns named needed to construct references are listed above for each link macro.

Expandfile variableMeaningDefault value
%[_xf_expand_multics]%Enable Multics expansions. Can be blank, 'nosql' or 'all.empty
%[_xf_bracketcolonclass]%class used with {: :}cmd
%[_xf_bracketequalclass]%class used with {= =}pathname
%[_xf_bracketplusclass]%class used with {+ +}code
%[_xf_bracketminusclass]%class used with {- -}special
%[_xf_multicssourceroot]%value to be used with {* *}TBD
%[_xf_hostname]%MySQL server hostname 
%[_xf_database]%MySQL database name 
%[_xf_username]%MySQL userid 
%[_xf_password]%MySQL password 
%[_xf_extreftable]%MySQL table queried with {! !}extref
%[_xf_extrefkeycol]%MySQL column queried with {! !}extname
%[_xf_extrefvalcol]%MySQL column returned with {! !}exturl
%[_xf_innewwindow]%text decoration used with {! !}new window:
%[_xf_newwindow]%class used with {! !} target="_blank" rel="noopener"
%[_xf_tinyglob]%globe icon used with {! !}
%[_xf_peoplefile]%filename base used with {[ ]}multicians
%[_xf_multicianstable]%MySQL table queried with {[ ]}m
%[_xf_multicianskeycol]%MySQL column queried with {[ ]}nametag
%[_xf_multiciansvalcol]%MySQL column returned with {[ ]}did
%[_xf_loclinktable]%MySQL table queried with {@ @}pages
%[_xf_loclinkkeycol]%table column queried with {@ @}filename
%[_xf_fixlocallink]%flag used with {@ @}empty
%[_xf_mxrelative]%prefix used with {@ @}empty
%[_xf_glosstable]%MySQL table queried by printgloss3.plglossary
%[_xf_glosskeycol]%MySQL column queried by printgloss3.pltag
%[_xf_glossvalcol]%MySQL column returned by printgloss3.pldef
%[_xf_glossbase]%filename base in printgloss3.plmg

Multics website Makefile

The Multics website's Makefile has recipes that expand any .htmx file that is older than the corresponding .html. In addition, each MySQL table is loded from a .sql text file, and the Makefile ensures that if the .sql file changes, it is loaded into MySQL and then relevant .htmx files are generated. make takes care of ordering the generation steps.

When .html files are generated, the Makefile executes HTML Tidy to check if the files have HTML syntax errors.

There is also an "install" recipe in the Makefile that executes rsync to copy modified .html files and graphics to the web server. It causes a "make all" if necessary before executing, so to make a one-line change I just need to edit the file and type "make install".

More information on the Multics website build process is in Multics Web Site Build Process.

Graphical Features for Multics Website

expandfile generates bar charts from SQL data for https://multicians.org/. For example, MSPM Authors and Site Timeline . expandfile also generates data rows from SQL to supply parameters to JavaScript display applets for timelines and pie charts: for example, Multics History and Site Timeline.