litmark

unworriedsafari

2022-06-04

litmark is a flexible tangler that enables literate programming using Pandoc Markdown documents. litmark is not a weaver. At the moment that job is left to Pandoc.

This document borrows some terminology from Noweb. To quote the Wikipedia article:

A Noweb input text contains program source code interleaved with documentation. It consists of so-called chunks that are either documentation chunks or code chunks.

It’s also possible to include code chunks in other code chunks by using a code chunk reference.

In litmark, code chunks are Markdown code blocks with a heading. The precise syntax of these code blocks is described in the section Code Chunks. Anything outside such code blocks is considered to be a documentation chunk.

The syntax for code chunks is compatible with Pandoc Markdown, while the syntax for code chunk references is programming-language dependent and customizable. This way the developer can avoid having to litter the code with escape sequences due to chunk-reference syntax.

The goal is to enable literate programming using plain Markdown documents as input with a syntax that eliminates the need for escape sequences for as much as possible.

Table of Contents

Install

Requirements

Setup

If you are reading this on a webpage, first obtain the source archive and extract it somewhere.

Once you have obtained the source, do

> cd path/to/litmark
> make
> make install PREFIX=<install-directory>

The default PREFIX is ~/.local/bin. You should keep the litmark directory around after running make install because make install only creates symlinks to results in this directory.

Uninstall

> cd path/to/litmark
> make uninstall PREFIX=<install-directory>

Command-line Invocation

litmark is invoked from the command-line as follows:

> cat path/to/file/file.md | litmark <reference>

The first argument is the reference that is to be expanded and printed. Another way of saying that is that the target chunk of the reference is being tangled. When a reference is put on the command-line, it is interpreted in the fallback language. More on that in the section References. The result is printed to standard output.

So, if this is the content of hello.md

## Hello, world

``` {.lua .chunk}
print("Hello, world")
```

Then

> cat hello.md | litmark "<Hello, world>"

Would result in

print("Hello, world")

Tutorial

Overview

Tangling is actually nothing more than macro expansion. Code chunks are macro definitions and referencing a code chunk amounts to expanding the macro. Code chunks can also be parameterized and indentation is preserved.

Code Chunks

Code chunks are written using fenced code blocks that

For example:

## Example: Code Chunks

``` {.lua .chunk}
print("Hello, world")
```

Result of tangling Example: Code Chunks:

print("Hello, world")

The heading of the code chunk becomes the name of the chunk. There must be exactly one blank line between the heading and the opening fence.

References

Other chunks can be referenced by placing their names between between two characters: the start and end delimiters. These delimiters can be specified per programming language. If a specification for a particular language is absent, the fallback specification is used.

Delimiter Specification
Language Start Delimiter End Delimiter
Fallback < >
C++ @ >

Fennel does not have its own delimiter specification. So, an example with a code chunk reference for the Fennel language looks like this:

## Example: References 1

``` {.fennel .chunk}
"Hello, world!"
```

## Example: References 2

``` {.fennel .chunk}
(print <Example: References 1>)
```

Result of tangling Example: References 2:

(print "Hello, world!")

The algorithm for recognizing code chunk references only sees balanced pairs of start and end delimiters. This means a lot of escaping is avoided: it’s not necessary to escape individual start and end delimiters unless they could form a balanced pair.

Nevertheless, sometimes escaping is desired. The start and end delimiters can be escaped by prepending them with a backslash.

Escaping a delimiter prevents it from being recognized as part of a balanced pair. This also means it’s not always necessary to escape the other delimiter.

## Example: Escaping Delimiter 1

``` {.fennel .chunk}
"Hello, world!"
```

## Example: Escaping Delimiter 2

``` {.fennel .chunk}
(print \<Example: Escaping Delimiter 1>)
```

Result of tangling Example: Escaping Delimiter 2:

(print <Example: Escaping Delimiter 1>)

The section Syntax Rules for Parameters and References contains details on how to customize the delimiters for each programming language. Note that you shouldn’t use [ and ] as start and end delimiters. These are reserved for parameters.

Parameters

Code chunks can be parameterized by including parameters in the name of the code chunk surrounded by square brackets. A parameter is a macro in the scope of the chunk’s body. To use it, use the same syntax as for chunk references.

## Hi, [x]!

``` {.fennel .chunk}
"Hello, <x>!"
```

To pass a parameter, use the same syntax as for introducing it: surround the value with square brackets.

## Example: Parameters 1

``` {.fennel .chunk}
(print <Hi, [beautiful world]!>)
```

Result of tangling Example: Parameters 1:

(print "Hello, beautiful world!")

Here, the same syntax rules apply as for start and end delimiters of chunk references. The algorithm for recognizing parameters is the same as for recognizing chunk references, except that the start and end delimiters are fixed ([ and ]). Note that it’s not necessary to escape square brackets outside of chunk headers or references.

Note that you can use references inside parameters as well. So, in order to pass a parameter that was received, you could do something like this

## Hi, beautiful [x]

``` {.fennel .chunk}
(print <Hi, [beautiful <x>]!>)
```

## Example: Parameters 2

``` {.fennel .chunk}
<Hi, beautiful [world]>
```

Result of tanling Example: Parameters 2:

(print "Hello, beautiful world!")

However, keep in mind that parameter expansion is also just a simple text substitution operation. The result of substitution is the same regardless of what the language of the code block is.

Duplicate Chunks

When multiple chunks with the same name occur, you can choose what the behavior should be. For each code block, you can specify a mode attribute that controls what litmark does with the code block in relation to an existing chunk of the same name. The following table shows the possible behaviors.

Code Block Modes
Mode Behavior
a Append to existing chunk (default behavior)
w Overwrite existing chunk

Processing code blocks proceeds in a top-to-bottom fashion. So, an example of mode=w would be:

## Example: Duplicate Chunk mode=w

``` {.fennel .chunk}
"Hello, world!"
```

## Example: Duplicate Chunk mode=w

``` {.fennel .chunk mode=a}
"Hello, universe!"
```

## Example: Duplicate Chunk mode=w

``` {.fennel .chunk mode=w}
(print "Hello, universe!")
```

Result of tangling Example: Duplicate Chunk mode=w:

(print "Hello, universe!")

And an example of mode=a:

## Example: Duplicate Chunk mode=a

``` {.fennel .chunk}
"Hello, world!"
```

## Example: Duplicate Chunk mode=a

``` {.fennel .chunk}
"Hello, universe!"
```

## Example: Duplicate Chunk mode=a

``` {.fennel .chunk mode=a}
(print "Hello, universe!")
```

Result of tangling Example: Duplicate Chunk mode=a:

"Hello, world!"
"Hello, universe!"
(print "Hello, universe!")

Note that it is possible to use different languages for duplicate chunks.

Note also that chunks which differ only in their parameter names cannot be distinguished, so it is considered an error to create such chunks.

Indentation

Indentation is preserved for multi-line chunks. That is, the chunk will be aligned with the position of the reference. This also works recursively. If two references appear on the same line, the first one is fully expanded before the second one, so the expansion of the second reference is appended to the last line of the expansion of the first reference. The indentation of the second chunk is computed after the first reference is expanded. For example:

## Indentation Example: Hello

``` {.c .chunk}
"Hello, <Indentation Example: World><Indentation Example: Exclaim>"
```

## Indentation Example: World

``` {.fennel .chunk}
literate
world
```

## Indentation Example: Exclaim

``` {.fennel .chunk}
!
```

## Indentation Example: Hello, world

``` {.fennel .chunk}
(print <Indentation Example: Hello>)
```

Result of tangling Indentation Example: Hello, world:

(print "Hello, literate
               world!")

Implementation

This section contains the implementation of litmark.

litmark is written in Fennel. Fennel lets you program in Lua with a Lisp syntax. Fennel’s syntax is documented here: https://fennel-lang.org/reference#syntax.

During make, the implementation code embedded in this document is turned into a Fennel source file by using a pre-processor that turns all lines into comments except non-indented fenced code blocks.

Parsing Command-line Arguments

(local root-ref (. arg 1))

(when (or (not root-ref)
          (= "-h" root-ref)
          (= "--help" root-ref))
  (print (.. "Usage: litmark <reference>\n"
             "See README.md for the manual."))
  (os.exit))

Syntax Rules for Parameters and References

The syntax is determined by the following variables.

(local param-syntax "[]")

(local ref-syntax
  {:fallback "<>"   ;<chunk name>
   :cpp      "@~"}) ;@chunk name~

Parameter syntax specifies the start and end delimiter. Reference syntax does the same for each language. :fallback is the default in case the syntax for a language is absent.

To change the syntax rules, simply edit the code block above and rerun make install with the same PREFIX that you used last time.

Collecting Code Chunks

Code chunks are stored in a table where the keys are chunk names and the values are tables with information about the chunks.

Escape Sequences

Before we can extract any information from the text, we need to deal with escape sequences. Lua patterns have an interesting feature that lets you easily match balanced pairs based on two characters. However, it does not let you escape those characters as easily. We will first build a function that lets you escape the characters for a balanced-pair match.

The principle is easy: if a delimiter is prepended by a backslash, it should not be considered a delimiter. This can be achieved by temporarily replacing those prepended delimiters by a different character before executing the match.

Because Lua is not so ‘batteries-included’ as Python, we also need to lay some groundwork before we can encode this function.

(fn string-replace [s i j r]
  (.. (string.sub s 1 (- i 1)) r (string.sub s (+ j 1))))

(assert (= (string-replace "xfy" 1 1 "z")
           "zfy"))
(assert (= (string-replace "xfy" 1 2 "z")
           "zy"))
(assert (= (string-replace "xfy" 3 3 "z")
           "xfz"))
(assert (= (string-replace "xfy" 2 2 "z")
           "xzy"))

(fn string-replace-ranges [s r t]
  (. (accumulate [result [s 0]
                  _ rng (ipairs t)]
       [(string-replace (. result 1)
                        (+ (. rng 1) (. result 2))
                        (+ (. rng 2) (. result 2))
                        r)
        (+ (. result 2) (- (length r) (+ 1 (- (. rng 2) (. rng 1)))))])
     1))

(assert (= (string-replace-ranges "xfy" "z" [[2 2]])
           "xzy"))

(assert (= (string-replace-ranges "xfy" "z" [[1 1] [3 3]])
           "zfz"))

(assert (= (string-replace-ranges "xfy" "ze" [[1 1] [3 3]])
           "zefze"))

(assert (= (string-replace-ranges "xfya" "z" [[1 2] [3 4]])
           "zz"))

(fn pattern-escape [char]
  (if (string.find "^$()%.[]*+-?" (.. "%" char))
    (.. "%" char)
    char))

(fn escaped-bp-gmatch [s pair]
  (let [d1 (pattern-escape (string.sub pair 1 1))
        d2 (pattern-escape (string.sub pair 2 2))
        matches-escaped-d1 (icollect [m (string.gmatch s (.. "\\()" d1))] [m m])
        matches-escaped-d2 (icollect [m (string.gmatch s (.. "\\()" d2))] [m m])
        altered-s (string-replace-ranges
                    (string-replace-ranges s " " matches-escaped-d1)
                    " " matches-escaped-d2)
        bp-matches (icollect [b e (string.gmatch altered-s
                                                 (.. "()%b" pair "()"))]
                     [b (- e 1)])]
    (table.unpack bp-matches)))

(let [s "(he\\(llo\\))"
      r (table.pack (escaped-bp-gmatch s "()"))]
  (assert (and (= (length r) 1)
               (= (length (. r 1)) 2)
               (= (string.sub s (. r 1 1) (. r 1 2))
                  "(he\\(llo\\))"))))

(let [s "(he(llo\\))"
      r (table.pack (escaped-bp-gmatch s "()"))]
  (assert (and (= (length r) 1)
               (= (length (. r 1)) 2)
               (= (string.sub s (. r 1 1) (. r 1 2))
                  "(llo\\))"))))

(let [s "(he\\(llo))"
      r (table.pack (escaped-bp-gmatch s "()"))]
  (assert (and (= (length r) 1)
               (= (length (. r 1)) 2)
               (= (string.sub s (. r 1 1) (. r 1 2))
                  "(he\\(llo)"))))

(let [s "<he\\<llo>>"
      r (table.pack (escaped-bp-gmatch s "<>"))]
  (assert (and (= (length r) 1)
               (= (length (. r 1)) 2)
               (= (string.sub s (. r 1 1) (. r 1 2))
                  "<he\\<llo>"))))

(let [s "<he\\<llo> <world>"
      r (table.pack (escaped-bp-gmatch s "<>"))]
  (assert (and (= (length r) 2)
               (= (length (. r 1)) 2)
               (= (string.sub s (. r 1 1) (. r 1 2))
                  "<he\\<llo>")
               (= (length (. r 2)) 2)
               (= (string.sub s (. r 2 1) (. r 2 2))
                  "<world>"))))

Chunk Names

Chunk names are stored as Lua patterns. When expanding chunk references, once a chunk name has been recognized, the corresponding chunk is found in the table by matching the recognized chunk name with the stored chunk-name patterns.

(fn extract-chunk-name? [line]
  (string.match line "^#+%s*(.+)$"))

(fn extract-chunk-name-pattern [chunk-name]
  (let [param-poses (table.pack (escaped-bp-gmatch chunk-name param-syntax))
        params-replaced (string-replace-ranges chunk-name
                           (.. "(%b" param-syntax ")")
                           param-poses)]
    (string.gsub
      (pick-values 1 (string.gsub params-replaced "\\%[" "\\%%["))
      "\\%]" "\\%%]")))

(assert (= (extract-chunk-name-pattern "Hello [x] world! \\[\\] [y]")
           "Hello (%b[]) world! \\%[\\%] (%b[])"))
(assert (= (extract-chunk-name-pattern "Hello world!")
           "Hello world!"))

The following information is stored in the table for each chunk name:

Information Per Chunk
Key Description
params The list of parameters for this chunk
lines The list of lines for this chunk

Chunk Parameters

Each item in the list of parameters is equal to the name of the parameter. We store this information separately because the parameters names are erased from the name of the chunk that is used as key.

(fn string-subs [s t]
  (icollect [_ rng (ipairs t)]
    (string.sub s (. rng 1) (. rng 2)))) 

(fn extract-chunk-params [chunk-name]
  (let [param-poses (table.pack (escaped-bp-gmatch chunk-name param-syntax))
        params (string-subs chunk-name param-poses)]
    (icollect [_ p (ipairs params)]
      (do
        (when (= 2 (length p))
          (error (.. "Empty parameter in: " chunk-name)))
        (string.sub p 2 -2)))))

(let [r (extract-chunk-params "Hello, world")]
  (assert (= (length r) 0)))

(let [r (extract-chunk-params "Hello, [x] world")]
  (assert (= (length r) 1))
  (assert (= (. r 1) "x")))

(let [r (extract-chunk-params "Hello, [x] world [y]")]
  (assert (= (length r) 2))
  (assert (= (. r 1) "x"))
  (assert (= (. r 2) "y")))

(let [r (extract-chunk-params "Hello, [\\[x] world [y]")]
  (assert (= (length r) 2))
  (assert (= (. r 1) "\\[x"))
  (assert (= (. r 2) "y")))

(let [r (extract-chunk-params "Hello, [<x>] world [y]")]
  (assert (= (length r) 2))
  (assert (= (. r 1) "<x>"))
  (assert (= (. r 2) "y")))

Each line is in turn a table with the following information:

Information Per Line
Key Description
language The language of the code block that the line appears in
text The text of the line

Language

Storing the language with each line enables us to append chunks with different languages.

(fn is-chunk-opening-fence? [line]
  (or (string.match line "^```+%s*{%.chunk[%s}]")
      (string.match line "^```+%s*{.*%s%.chunk[%s}]")))

(fn extract-language? [chunk-opening-fence]
  (let [class (string.match chunk-opening-fence "^```+%s*{%s*%.([%w%-_]+).*")]
    (or (and (not (= "chunk" class)) class)
        :fallback)))

Mode

(fn extract-mode [chunk-opening-fence]
  (or (string.match chunk-opening-fence "[{%s]mode=([aw])[%s}]")
      "a"))

Collecting

(local all-lines
  (icollect [line (io.lines)]
    line))

(local all-chunks
  (let [result {}]
    (var i 1)
    (var current-code-chunk nil)
    (while (<= i (length all-lines))

      ;; Consume documentation-chunk lines
      (while (and (<= i (length all-lines))
                  (not current-code-chunk))
        (if
          ;; Code-chunks are at least 3 lines long
          (> (+ i 2) (length all-lines))
          (set i (+ 1 (length all-lines)))

          (let [chunk-name (extract-chunk-name? (. all-lines i))
                opening-fence? (is-chunk-opening-fence? (. all-lines (+ 2 i)))]
            (if (and chunk-name opening-fence?)
              (let [chunk-name-pattern (extract-chunk-name-pattern chunk-name)
                    chunk-params (extract-chunk-params chunk-name)
                    language (extract-language? (. all-lines (+ 2 i)))
                    mode (extract-mode (. all-lines (+ 2 i)))
                    
                    existing-chunk-definition (. result chunk-name-pattern)]

                ;; Check that the existing params match the new params
                (when (and (= mode "a") existing-chunk-definition)
                  (let [existing-params (. existing-chunk-definition :params)]
                    (var param-list-different?
                         (not (= (length existing-params)
                                 (length chunk-params))))
                    (when (not param-list-different?)
                      (for [i 1 (length existing-params)]
                        (set param-list-different?
                             (or param-list-different?
                                 (not (= (. existing-params i)
                                         (. chunk-params i)))))))
                    (when param-list-different?
                      (error (.. "Line " (+ 2 i) ": param list different from"
                                 " previous definition: "
                                 "[" (table.concat existing-params ", ") "] "
                                 "[" (table.concat chunk-params ", ") "]")))))

                (set current-code-chunk {:language chunk-language
                                         :lines []
                                         :mode mode
                                         :name-pattern chunk-name-pattern
                                         :params chunk-params})
                (set i (+ i 3)))
              (set i (+ i 1))))))

      ;; Consume code-chunk lines
      (while current-code-chunk
        (when (> i (length all-lines))
          (error (.. "Line " i ": EOF. Code chunk not closed")))
        (if (string.match (. all-lines i) "^```")
          (do
            (tset result
                  (. current-code-chunk :name-pattern)

                  ;; If mode=w, replace the current lines.
                  ;; Otherwise, append current lines to existing lines
                  (let [existing-lines (or (?. result
                                               (. current-code-chunk
                                                  :name-pattern)
                                               :lines)
                                           [])
                        new-lines (. current-code-chunk :lines)]
                    {:lines (if (= (. current-code-chunk :mode) "w")
                              new-lines
                              (table.move new-lines 1 (length new-lines)
                                          (+ 1 (length existing-lines))
                                          existing-lines))
                     :params (. current-code-chunk :params)}))
            (set current-code-chunk nil))
          (table.insert (. current-code-chunk :lines)
                        {:language (. current-code-chunk :language)
                         :text (. all-lines i)}))
        (set i (+ i 1))))
    result))

Expanding References

Recognizing

Before we can code the algorithm for expanding code chunks, we need the algorithm that recognizes references correctly. We’ll need to write our own procedure. This procedure

Since references can be parameterized, they can contain other references. This procedure will not recognize such contained references. Instead they will simply become part of the reference string. Those references can be recognized later by simply calling this procedure again.

(fn extract-ref? [language line]
  (let [m (escaped-bp-gmatch line (or (. ref-syntax language)
                                      (. ref-syntax :fallback)))]
    (if m
      (values
        (. m 1)
        (string.sub line (. m 1) (. m 2))))))

(assert (= (extract-ref? :fallback "Hello")
           nil))

(let [r (table.pack (extract-ref? :fallback "Hello <world bro>"))]
  (assert (= (length r) 2))
  (assert (= (. r 1) 7))
  (assert (= (. r 2) "<world bro>")))

(let [r (table.pack (extract-ref? :fallback "Hello <world> <bro>"))]
  (assert (= (length r) 2))
  (assert (= (. r 1) 7))
  (assert (= (. r 2) "<world>")))

(let [r (table.pack (extract-ref? :fallback "Hello <world [<comp\\<licated>]> <bro>"))]
  (assert (= (length r) 2))
  (assert (= (. r 1) 7))
  (assert (= (. r 2) "<world [<comp\\<licated>]>")))

Parsing

Parsing a reference means finding out which name it is referencing and determining what the parameters are.

(fn parse-ref [ref]
  (let [ref-name (string.sub ref 2 -2)]
    (values
      (extract-chunk-name-pattern ref-name)
      (extract-chunk-params ref-name))))

Unescaping

(fn unescape-line [language line]
  (let [ref-s (or (. ref-syntax language)
                  (. ref-syntax :fallback))
        unescaped-1 (string.gsub line
                                 (.. "\\" (pattern-escape
                                            (string.sub param-syntax 1 1)))
                                 (string.sub param-syntax 1 1))
        unescaped-2 (string.gsub unescaped-1
                                 (.. "\\" (pattern-escape
                                            (string.sub param-syntax 2 2)))
                                 (string.sub param-syntax 2 2))
        unescaped-3 (string.gsub unescaped-2
                                 (.. "\\" (pattern-escape
                                            (string.sub ref-s 1 1)))
                                 (string.sub ref-s 1 1))
        unescaped-4 (string.gsub unescaped-3
                                 (.. "\\" (pattern-escape
                                            (string.sub ref-s 2 2)))
                                 (string.sub ref-s 2 2))]
    unescaped-4))

(assert (= (unescape-line :fallback "\\< \\> \\[ \\] <>[]")
           "< > [ ] <>[]"))

(assert (= (unescape-line :blabladaf "\\< \\> \\[ \\] <>[]")
           "< > [ ] <>[]"))

Environments

We will use an environment that maps references to text bodies. The critical operation is to, given an environment, derive a new environment with updated mappings. We will implement environments as tables. We can efficiently implement this operation without copying the table.

(fn kv-table [keys vals]
  (let [r {}]
    (for [i 1 (math.min (length keys) (length vals))]
      (tset r (. keys i) (. vals i)))
    r))

(let [t (kv-table [3 "foo" 2] [1 5 7])]
  (assert (= (. t 3) 1))
  (assert (= (. t "foo") 5))
  (assert (= (. t 2) 7)))

(fn update-env [old-env new-mappings]
  (let [mt (or (getmetatable new-mappings) {})]
    (tset mt :__index old-env)
    (setmetatable new-mappings mt)))

(let [e (update-env {3 1 "foo" "bar"} {"foo" "baz" 4 2})]
  (assert (= (. e "foo") "baz"))
  (assert (= (. e 3) 1))
  (assert (= (. e 4) 2)))

Expanding

With the information described in the previous section we can sketch the algorithm for expanding code chunks.

Given the following information:

  1. A line L
  2. Prefix R
  3. Environment E that maps names to bodies. A body is a list of lines.

The process of expanding L is as follows:

  1. Find the first reference C in L
  2. Split L into Pre and Suf (before and after C)
  3. Let Pre = unescape Pre
  4. If C is nil, return Pre
  5. Parse C into name N and parameter list P
  6. Let P’ be the replacement list of P i.e. each expanded element of P. Each element of P is expanded as a single line containing just the element and with an empty prefix. The results of each expansion is a list of lines!
  7. Copy the environment E to a new environment F
  8. Create mappings for the elements of P’ in F. That is, each parameter name of N is mapped to its corresponding value in P’.
  9. Obtain body B of N from E
  10. Let B’ be B expanded using (R + Pre) and F, the result is a list of lines
  11. Drop the last line L’ in B’
  12. Let S be the expansion of Suf with prefix L’ and environment E
  13. Return (B’ + S)

The major reason for the use of an environment is that we want to replace references as a whole instead of first replacing their parameters. That way we don’t accidentally destroy the references when those parameters happen to expand to multiple lines.

Expanding lines is achieved by two mutually-recursive functions: expand-lines and expand-line. One of them needs to be declared first otherwise the other one can’t find it.

(var expand-lines nil)

(fn expand-line [line prefix env]
  (let [text (. line :text)
        language (. line :language)
        (start ref) (extract-ref? language text)]
    (if (not start)
      [(.. prefix (unescape-line language text))]
      (let [chunk-prefix-1st-line (unescape-line language
                                    (string.sub text 1 (- start 1)))
            chunk-suffix-last-line (string.sub text (+ start (length ref)))

            (chunk-name ref-args) (parse-ref ref)
            expanded-args (icollect [_ arg (ipairs ref-args)]
                            {:lines (icollect [_ t (ipairs (expand-line
                                                             {:language language
                                                              :text arg}
                                                             "" env))]
                                      {:language language
                                       :text t})})
            ref-params (or (?. env chunk-name :params) [])
            ref-env (update-env env (kv-table ref-params expanded-args))
            expanded-chunk-lines (expand-lines chunk-name
                                               (.. prefix chunk-prefix-1st-line)
                                               ref-env)

            last-expanded-chunk-line (table.remove expanded-chunk-lines)
            expanded-suffix-lines (expand-line {:language language
                                                :text chunk-suffix-last-line}
                                               last-expanded-chunk-line
                                               env)]
        (table.move expanded-suffix-lines
                    1
                    (length expanded-suffix-lines)
                    (+ 1 (length expanded-chunk-lines))
                    expanded-chunk-lines)))))

(set expand-lines
     (fn [chunk-name prefix-1st-line env]
       (let [chunk-lines (?. env chunk-name :lines)
             prefix-rest (string.gsub prefix-1st-line "%g" " ")
             result []]
         (when (not chunk-lines)
           (error (.. "ERROR: chunk named '" chunk-name "' does not exist")))
         (each [i line (ipairs chunk-lines)]
           (let [prefix (if (= i 1)
                          prefix-1st-line
                          prefix-rest)
                 expanded-lines (expand-line line prefix env)]
             (each [_ expanded-line (ipairs expanded-lines)]
               (table.insert result expanded-line))))
         result)))

Main Loop

(each [_ line (ipairs (expand-line {:language :fallback
                                    :text root-ref}
                                   "" all-chunks))]
  (print line))

License: X11

Copyright (C) 1996 X Consortium

Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:

The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE X CONSORTIUM BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Except as contained in this notice, the name of the X Consortium shall
not be used in advertising or otherwise to promote the sale, use or
other dealings in this Software without prior written authorization
from the X Consortium.

X Window System is a trademark of X Consortium, Inc.