How to Work with Different Markup Styles

The generateSite task is often used to convert AsciiDoc to HTML. AsciiDoc is the default (and preferred) markup language for documentation written in docToolchain.

But because docToolchain is designed to be used by larger teams and organisations, its built-in site generator can render AsciiDoc, Markdown, and plain HTML.

Supported Markup Formats

The microsite (generateSite) accepts these input formats:

Format Status Notes

AsciiDoc (.adoc)

Built-in (default)

The preferred markup for docToolchain.

Markdown (.md)

Built-in

Rendered out of the box via flexmark; flavour configurable.

HTML (.html)

Built-in

The body is shown in the content area.

reStructuredText (.rst)

Supported, opt-in

Enabled with one additionalConverters entry (see below). Requires Python + docutils on the build host.
These formats are supported by the microsite pipeline (generateSite) only. The standalone generateHTML / generatePDF tasks process AsciiDoc. reStructuredText in particular is not available in generateHTML / generatePDF.

Meta-Data Header

In AsciiDoc, metadata is defined as attributes beginning with jbake-, such as :jbake-header:. For other markup languages, the metadata is defined in a block at the beginning of the document, delimited by ~~ [footnote: this can be configured in the jbake.properties file of the theme, and will be available in docToolchainConfig.groovy in the next version]. This jbake--prefixed metadata format is kept for backwards compatibility — docToolchain’s built-in generator reads the same headers.

Markdown

Out of the Box

docToolchain renders Markdown out of the box, so you can use it without any additional configuration. As illustrated by this sample repository, docToolchain utilises the convention-over-configuration principle to determine the menu (jbake-menu), the location within the menu (jbake-order), and the title entry of the menu (jbake-title) from the folder structure and the document’s first headline.

If you wish to override these defaults, you can use a metadata header.

Flavours

Here, the Markdown standard is relatively limited. For extended features, you’ll need to specify the flavour you want to use. docToolchain employs flexmark to render Markdown, and flexmark supports various flavours. These can be configured within the jbake.properties file within the theme. The default is markdown.extensions=GITHUB,EXTRA,TABLES,TOC,FENCED_CODE_BLOCKS (the names follow flexmark’s pegdown profile).

See the flexmark pegdown profile for the available extensions.

HTML

Plain HTML is supported in the same way as Markdown. The HTML body will be displayed in the content area of the microsite. Refer to src/docs/Demo/html.html in the sample repository.

restructuredText (.rst)

reStructuredText is already supported — you just have to switch it on. Since the built-in generator doesn’t render .rst directly, docToolchain ships a converter (scripts/rstToHtml.py) and runs it through the additionalConverters mechanism.

Prerequisite

The converter is a docutils script, so the build host needs:

  • python3

  • the docutils package (pip install docutils)

Enable .rst by adding this one entry to the microsite section of your docToolchainConfig.groovy (copy-paste ready):

docToolchainConfig.groovy
additionalConverters = [
    '.rst': [command: 'dtcw:rstToHtml.py', type: 'bash'],
]

dtcw:rstToHtml.py is the internal converter shipped with docToolchain; the dtcw: prefix tells docToolchain to resolve it from its own scripts/ folder.

Once an additional converter is configured, docToolchain will traverse all doc-files, check the extension, and invoke the configured script if the extension matches.

The script’s task is to convert the file to a markup format the site generator recognises (AsciiDoc, Markdown, or HTML). Afterwards, the built-in generator processes everything as usual. In this case, the Python docutils convert reStructuredText to HTML.

This applies to the microsite (generateSite) only — .rst is not processed by the standalone generateHTML / generatePDF tasks.

For reference, the full additionalConverters configuration block (with all supported converter types) looks like this:

template_config/Config.groovy
/**

if you need support for additional markup converters, you can configure them here
you have three different types of script you can define:

- groovy: just groovy code as string
- groovyFile: path to a groovy script
- bash: a bash command. It will receive the name of the file to be converted as first argument

`groovy` and `groovyFile` will have access to the file and config object

`dtcw:rstToHtml.py` is an internal script to convert restructuredText.
Needs `python3` and `docutils` installed.

**/
    additionalConverters = [
        //'.one': [command: 'println "test"+file.canonicalPath', type: 'groovy'],
        //'.two': [command: 'scripts/convert-md.groovy', type: 'groovyFile'],
        //'.rst': [command: 'dtcw:rstToHtml.py', type: 'bash'],
    ]

Additional Markup Languages

You can integrate additional markup languages in the same way as you added restructuredText. The only difference is that you will configure the script to render your files as a reference to your converter script, rather than using the internal script.

You can find an example of the internal script for restructured text here: https://github.com/docToolchain/docToolchain/blob/main-4.x/scripts/rstToHtml.py.

Special Cases

This mechanism also enables you to add new features to existing markup languages. For instance, you can use a script to replace all plantUML references in a Markdown file with a reference to a kroki.io server to render the file.

docToolchainConfig.groovy
...
    additionalConverters = [
            '.md': [
                    type: 'groovyFile',
                    command: 'scripts/markdown-kroki.groovy'
            ]
    ]
...
scripts/markdown-kroki.groovy
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.util.Base64;
import java.util.zip.Deflater;

public static byte[] encode(String decoded) throws IOException {
    return Base64.getUrlEncoder().encode(compress(decoded.getBytes()));
}

private static byte[] compress(byte[] source) throws IOException {
    Deflater deflater = new Deflater(Deflater.BEST_COMPRESSION);
    deflater.setInput(source);
    deflater.finish();

    byte[] buffer = new byte[2048];
    int compressedLength = deflater.deflate(buffer);
    byte[] result = new byte[compressedLength];
    System.arraycopy(buffer, 0, result, 0, compressedLength);
    return result;
}

def source = file.text
def krokiServer = "https://kroki.io/"

def newSource = source.replaceAll(/(?s)```(plantuml|mermaid)([^`]*)```/){all, type, diagramSource ->
    System.out.println file.canonicalPath
    System.out.println type
    System.out.println diagramSource
    imageUrl = krokiServer+type+'/png/'+new String(encode(diagramSource))
    System.out.println imageUrl
    return "![$type diagram]($imageUrl \"Image Title\")"
}
file.write(newSource)