A Org-based templating mechanism

In a previous post I mentioned the way I use Emacs Org code blocks and the so-called Noweb syntax as a templating mechanism. The idea is to have a template version of a document containing a certain number of placeholders and a separate file to store the placeholders' values. The template and the data file are then fed into a build process that recombines things and produces the final document.

I regularly use this pattern for automation, i.e. to generate documents that tend to repeat the same general structure, like quotes, invoices, some contracts, etc. In order to produce a new version of a document I (manually or programmatically) tweak the data file and rerun the build process, which makes for a pretty pleasant experience. In this case the output of the build process is usually a text or PDF file.

I also use this pattern for confidentiality reasons, i.e. in those cases where I want to be able to share a certain project without revealing some of its specific details. An example of this is my system configurations project where I keep the configuration files for some of my machines. The files are publicly available, except for some details that are irrelevant or security sensitive and that are versioned separately. In this case, the objective of the build script is not producing a document but rather declaratively updating or reconfiguring my machines.

The way I've implemented this templating mechanism has slightly evolved over time. Here's a quick recap of how I currently do this.

The templating mechanism is based on:

• an Emacs Org template file that includes various placeholders,
• a JSON file for data where the placeholders' values are defined,
• a build script that recombines the template and the data to produce a final document.

After a few iterations I finally settled on using the JSON format for the data file. JSON is well supported and can be easily parsed from most languages. To illustrate how this works, let's look my system configuration project. Here's an excerpt from my (sample) data file:

{
  "msmtp-accounts": [
    {
      "name": "jane@example.com",
      "user": "jane@example.com",
      "host": "mail.example.com",
      "port": 587,
      "from": "jane@example.com",
    },
    {
      "name": "jdoe@example.net",
      "user": "jdoe@example.net",
      "host": "mail.example.net",
      "port": 587,
      "from": "jdoe@example.net",
    },
  ],
  ...
}

Here's an excerpt from the template file:

* Guix configuration

#+begin_src scheme :noweb yes
...
(define my/msmtp-configuration
  (home-msmtp-configuration
   (default-account "<<get-scheme("msmtp-default-account")>>")
   (accounts
    (map (lambda (a)
           (msmtp-account
            (name (assoc-ref a "name"))
            (configuration
             (msmtp-configuration
              (user (assoc-ref a "user"))
              (host (assoc-ref a "host"))
              (port (assoc-ref a "port"))
              (from (assoc-ref a "from"))))))
         (vector->list <<get-scheme("msmtp-accounts")>>)))))
...
#+end_src

* Helper functions

#+name: get-emacs-lisp
#+begin_src emacs-lisp :var key="" data-file="data.json"
(setq json-array-type 'list)
(alist-get key (json-read-file data-file))

#+name: get-scheme
#+begin_src scheme :session :var key="" data-file="data.json"
(use-modules (json))
(call-with-input-file data-file
  (lambda (input)
    (assoc-ref (json->scm input) key)))
#+end_src

Note the use of :noweb yes for the code block. This tells the build script to use the Noweb syntax and to interpret any <<get-scheme("key")>> (or similar) expression as a reference to a value in the data file.

Note that we have different helper functions, one for each relevant language, in this case Guile Scheme and Emacs Lisp. This makes it possible to interpret JSON correctly depending on the context, e.g. to have JSON's false translated to #f in Guile Scheme and to nil in Emacs Lisp.

In this particular case the build process can be triggered via the org-babel-tangle function from within Emacs. It can also be automated as a Makefile with the help of a couple of scripts, see the project repository for details.