docToolchain Manual


1. Installing docToolchain

Ralf D. Müller Jeremie Bresson Bennykillua Jérémie Bresson T8xi Michael Krauße Ralf D. Müller stehlih Dr. Stefan Pfeiffer Alexander Schwartz feinstaub Jakub J Jablonski jakubjab Jorge Dr. Gernot Starke Lars Francke

6 minutes to read

1.1. Getting the Wrapper

The way docToolchain is installed has changed since the v2.x release. It now uses a simple wrapper script which takes care of everything.

This wrapper helps by:

  • Ensuring that everyone using your repository uses the same docToolchain version.

  • Keeping all docToolchain technology out of your repository.

  • Wrapping your commands and directing them to your local installation.

  • Checking your docToolchain installation (if not installed, the wrapper will do it for you via direct download or Docker).

This guide explains how to set up docToolchain in the following environments:

  • A Linux/WSL2 with bash

  • Windows with PowerShell

  • Windows with cmd.exe

Please select the appropriate setup and follow these instructions to install docToolchain step by step.

Linux / WSL2 with bash
cd <your project>
curl -Lo dtcw doctoolchain.github.io/dtcw
chmod +x dtcw

if you don’t have curl installed, you can also use wget:

cd <your project>
wget doctoolchain.github.io/dtcw
chmod +x dtcw
Windows with Powershell
cd <your project>
Invoke-WebRequest doctoolchain.github.io/dtcw.ps1 -Outfile dtcw.ps1
Windows with cmd.exe
cd <your project>
curl -Lo dtcw.bat doctoolchain.github.io/dtcw.bat
dtcw.bat wraps the dtcw.ps1 script and executes it in powershell. This might be easier to use if you haven’t yet configured your powershell as a developer.
In order to support development for your project on both, Linux and Windows, it makes sense to download all three scripts to your project.
if you use cmd.exe, follow the PowerShell instructions but replace .ps1 with .bat.
dtcw uses bash as shell and thus tries to find java from within bash. Chances are that when you use dtcw from another shell like zsh, you shell knows about the right java version but your bash doesn’t. In that case, please switch to bash and make sure that the right java version is known. Afterwards, you can switch back to your shell.

1.2. Configure the Wrapper

There is no need to configure the wrapper itself if you have just downloaded it. But from time to time you should check github to see if there are new releases you want to use.

In this case, open dtcw in your favourite editor and look for the VERSION-line at the start of the file:

# find official release versions on github:
# https://github.com/docToolchain/docToolchain/releases
# set VERSION to "latest" to get the unreleased latest version
VERSION=2.0.5

Update it to match the latest release and and everything will be updated with the next run.

this will not update the wrapper (dtcw) itself. If you think you should update the wrapper, please re-install it.

There are some situation where you want to test a not-yet-released feature. In such a case, you can set the VERSION to latest and dtcw will clone or pull the current default branch of the project.

This only works with a local copy, not with a docker install.

You can also use latestdev as version. In this case, dtcw will try to clone the repository with the ssh-git protocoll. This way, you can easily point it to your fork and develop new features for docToolchain. You will find the fork in $HOME/.doctoolchain/docToolchain-latest.

latest and latestdev currently only work with the bash version of the wrapper.

1.3. Running your First Command

To get started with docToolchain, run tasks as the first command:

Linux / WSL2 with bash
./dtcw tasks --group=doctoolchain
Windows with Powershell
./dtcw.ps1 tasks --group=doctoolchain
if using cmd.exe replace .ps1 with .bat for all subsequent commands.

This command shows all docToolchain tasks and verifies that docToolchain is installed by doing the following:

  • Checking for a configuration file (and create one if none exist). Prefers a local installation.

  • Making use of Docker (if available) where there is no local installation.

If this is your first time installing docToolchain, the images download might take a while.
  • if docker is not available, it will download and install docToolchain to your home folder

    • it will then run a gradle build through the gradle wrapper. If this is your first time, it will download and install gradle.

    • the gradle build will also fetch some dependencies. This will also take a while if run for the first time..

    • build-script dependencies are fetched from a repository referenced by the property mavenRepository. By default the value https://plugins.gradle.org/m2/ is used. When a repository requiring credentials is used the properties mavenUsername and mavenPassword can be set as well.

Example command passing a custom maven repository with credentials from the command line
./gradlew generateHTML -PmavenRepository=your_maven_repo -PmavenUsername=your_username -PmavenPassword=your_pw --info

You’ll notice the installation is an incremental process and will take a while because of all the additional dependencies needed. After the first few commands, you’ll have all the required dependencies installed and things will run faster.

1.4. Existing Documents

If your project already has documents in AsciiDoc format, you’ll need to tell docToolchain where to find them.

To do so, take a look at the created docToolchainConfig.groovy and update it.

1.5. arc42 from Scratch

If you don’t have existing documents yet, or if you need a fresh start, you can get the arc42 template in AsciiDoc format.

Run the command below to tell docToolchain to download the template of your choice:

Linux / WSL2 with bash
./dtcw downloadTemplate
Windows
./dtcw.ps1 downloadTemplate

1.6. Building and Creating Templates.

By now, the docToolchain wrapper dtcw should be in your project folder along with the arc42 template.

Now Let’s render it to HTML and PDF. To do so, run the commands below:

Linux / WSL2 with bash
./dtcw generateHTML
./dtcw generatePDF
Windows
./dtcw.ps1 generateHTML
./dtcw.ps1 generatePDF

As a result, you will see the progress of your build together with some warnings which you can just ignore for the moment.

The first build generated some files within the build:

build
|-- html5
|   |-- arc42
|   |   `-- arc42.html
|   `-- images
|       |-- 05_building_blocks-EN.png
|       |-- 08-Crosscutting-Concepts-Structure-EN.png
|       `-- arc42-logo.png
`-- pdf
    |-- arc42
    |   `-- arc42.pdf
    `-- images
        |-- 05_building_blocks-EN.png
        |-- 08-Crosscutting-Concepts-Structure-EN.png
        `-- arc42-logo.png

Congratulations! If you see the same folder structure, you’ve just rendered the standard arc42 template as HTML and PDF!

Please raise an issue on github if you didn’t get the right output.


2. Using docToolchain to Build Docs

Bennykillua Ralf D. Müller Jeremie Bresson Michael Krauße Ralf D. Müller stehlih Dr. Stefan Pfeiffer Alexander Schwartz feinstaub Jakub J Jablonski jakubjab Jorge Dr. Gernot Starke Lars Francke

1 minute to read

docToolchain implements many features via scripts, which you call through the command line. These scripts are called tasks in this documentation.

Learn more about these scripts in the Tasks menu.


3. autobuildSite

Jody Winter Ralf D. Müller Jody Winter Ralf D. Müller Schrotti Tim Riemer Alexander Schwartz Julian Elve

1 minute to read

3.1. About This Task

This script starts an endless loop which checks for changes to your docs source then re-runs the generateSite-task whenever it detects changes. The output will be logged to build/generateSite.log.

3.2. Source

bin/autobuildSite.bash
#!/bin/bash
DIR_TO_WATCH='src/'
#COMMAND='rm -r build || true && mkdir -p build/microsite/output/images/ && ./dtcw generateSite 2>&1 | tee build/generateSite.log'
COMMAND='mkdir -p build/microsite/output/images/ && ./dtcw generateSite 2>&1 | tee build/generateSite.log'

#execute first time
cp src/docs/images/ready.png build/microsite/output/images/status.png
#eval $COMMAND

#wait for changes and execute
while true ; do
  watch --no-title --chgexit "ls -lR ${DIR_TO_WATCH} | sha1sum"
  cp src/docs/images/building.png build/microsite/output/images/status.png
  eval "$COMMAND"
  cp src/docs/images/ready.png build/microsite/output/images/status.png
  sleep 6
done

3.3. generateHTML

Jody Winter Jeremie Bresson Ralf D. Müller Jan Hendriks Alexander Schwartz Dr. Stefan Pfeiffer Lars Francke David M. Carr

3 minutes to read
generateHTML
About This Task

This is the standard Asciidoctor generator which is supported out of the box. The result is written to build/html5 (the HTML files need the images folder to be in the same directory to display correctly).

Generating Single-File HTML Output

If you would like the generator to produce a single-file HTML, you can configure Asciidoctor to store the images inline as data-uri by setting :data-uri: in the config of your AsciiDoc file. But be warned. The file can quickly become very large and some browsers might struggle to render it.

Creating Text-Based Diagrams

docToolchain is configured to use the asciidoctor-diagram plugin to create PlantUML diagrams. The plugin also supports many other text-based diagrams, but PlantUML is the most common. To use the plugin, specify your PlantUML code like this:

.example diagram
[plantuml, "{plantUMLDir}demoPlantUML", png] (1)
----
class BlockProcessor
class DiagramBlock
class DitaaBlock
class PlantUmlBlock

BlockProcessor <|-- DiagramBlock
DiagramBlock <|-- DitaaBlock
DiagramBlock <|-- PlantUmlBlock
----
1 The element of this list specifies the diagram tool plantuml to be used. The second element is the name of the image to be created, and the third specifies the image type.
{plantUMLDir} ensures that PlantUML also works for the generatePDF task. Without it, generateHTML works fine, but the PDF will not contain the generated images.
Be sure to specify a unique image name for each diagram, otherwise they will overwrite each other.

The above example renders as:

{plantUMLDir}demoPlantUML
Figure 1. example diagram
Controlling Diagram Size

If you want to control the size of the diagram in the output, configure the "width" attribute (in pixels) or the "scale" attribute (floating point ratio) passed to asciidoctor-diagram. The following example updates the diagram above by changing the declaration to one of the versions below:

[plantuml, target="{plantUMLDir}demoPlantUMLWidth", format=png, width=250]
# rest of the diagram definition

[plantuml, target="{plantUMLDir}demoPlantUMLScale", format=png, scale=0.75]
# rest of the diagram definition

The output will render like this:

{plantUMLDir}demoPlantUMLWidth
Figure 2. example diagram (with specified width)
{plantUMLDir}demoPlantUMLScale
Figure 3. example diagram (with specified scale)
To work correctly, PlantUML needs Graphviz dot installed. If you can’t install it, use the Java-based version of the dot library instead. Just add !pragma graphviz_dot smetana as the first line of your diagram definition. This is still an experimental feature, but already works well!
Further Reading and Resources
Source
AsciiDocBasics.gradle
task generateHTML (
        type: AsciidoctorTask,
        group: 'docToolchain',
        description: 'use html5 as asciidoc backend') {
        attributes (
            'plantUMLDir'         : file("${docDir}/${config.outputPath}/html5").toURI().relativize(new File("${docDir}/${config.outputPath}/html5/plantUML/").toURI()).getPath(),
        )

    // specify output folder explicitly to avoid cleaning targetDir from other generated content
    outputDir = file(targetDir + '/html5/')
    separateOutputDirs(false)

    def sourceFilesHTML = sourceFiles.findAll {
                                'html' in it.formats
                            }
//    onlyIf {
//        sourceFilesHTML
//    }

    sources {
        sourceFilesHTML.each {
            include it.file
            logger.info it.file
        }
    }
    resources {
        config.imageDirs.each { imageDir ->
            from(new File(file(srcDir),imageDir))
            logger.info ('imageDir: '+imageDir)
            into './images'
        }
        config.resourceDirs.each { resource ->
            from(new File(file(srcDir),resource.source))
            logger.info ('resource: '+resource.source)
            into resource.target
        }
    }
    backends = ['html5']
doFirst {
    if (sourceFilesHTML.size()==0) {
        throw new Exception ("""

>> No source files defined for type HTML.
>> Please specify at least one inputFile in your docToolchainConfig.groovy
""")
    }

}
}

Unresolved directive in <stdin> - include::../015_tasks/03_copy_themes.adoc[leveloffset=+2]


3.4. fixEncoding

Jody Winter Ralf D. Müller Alexander Schwartz

1 minute to read
About This Task

Whenever Asciidoctor has to process a file that is not UTF-8 encoded, Ruby tries to read it, then throws an error similar to this one:

asciidoctor: FAILED: /home/demo/test.adoc: Failed to load AsciiDoc document - invalid byte sequence in UTF-8

Unfortunately, finding the incorrectly encoded file is difficult if a lot of includes:: are used, and Asciidoctor will only show the name of the main document. This is not Asciidoctor’s fault. The fault lies with the Ruby interpreter that sits underneath.

The fixEncoding task crawls through all *.ad and *.adoc files and checks their encoding. If it comes across a file which is not UTF-8 encoded, it will rewrite it with the UTF-8 encoding.

Source
scripts/fixEncoding.gradle
import groovy.util.*
import static groovy.io.FileType.*

task fixEncoding(
        description: 'finds and converts non UTF-8 adoc files to UTF-8',
        group: 'docToolchain helper',
) {
    doLast {
        File sourceFolder = new File("${docDir}/${inputPath}")
        println("sourceFolder: " + sourceFolder.canonicalPath)
        sourceFolder.traverse(type: FILES) { file ->
            if (file.name ==~ '^.*(ad|adoc|asciidoc)$') {
                CharsetToolkit toolkit = new CharsetToolkit(file);
                // guess the encoding
                def guessedCharset = toolkit.getCharset().toString().toUpperCase();
                if (guessedCharset!='UTF-8') {
                    def text = file.text
                    file.write(text, "utf-8")
                    println("  converted ${file.name} from '${guessedCharset}' to 'UFT-8'")
                }
            }
        }
    }
}

3.5. prependFilename

Jody Winter Ralf D. Müller Alexander Schwartz Ralf D. Müller

1 minute to read
About This Task

When Asciidoctor renders a file, the file context only knows the name of the top-level AsciiDoc file. But an include file doesn’t know that it is being included. It simply gets the name of the master file and has no chance to get its own name as an attribute. This task crawls through all AsciiDoc files and prepends the name of the current file like this:

:filename: 015_tasks/03_task_prependFilename.adoc

This way, each file gets its own filename. This enables features like the inclusion of file contributors (see exportContributors-task).

This task skips all files named config.*, _config.*, feedback.* and _feedback.*.
Source
scripts/prependFilename.gradle
import static groovy.io.FileType.*

task prependFilename(
        description: 'crawls through all AsciiDoc files and prepends the name of the current file',
        group: 'docToolchain helper',
) {
    doLast {
        File sourceFolder = new File("${docDir}/${inputPath}")
        println("sourceFolder: " + sourceFolder.canonicalPath)
        sourceFolder.traverse(type: FILES) { file ->
            if (file.name ==~ '^.*(ad|adoc|asciidoc)$') {
                if (file.name.split('[.]')[0] in ["feedback", "_feedback", "config", "_config"]) {
                    println "skipped "+file.name
                } else {
                    def text = file.getText('utf-8')
                    def name = file.canonicalPath - sourceFolder.canonicalPath
                    name = name.replace("\\", "/").replaceAll("^/", "")
                    if (text.contains(":filename:")) {
                        text = text.replaceAll(":filename:.*", ":filename: $name")
                        println "updated "+name
                    } else {
                        text = ":filename: $name\n" + text
                        println "added   "+name
                    }
                    file.write(text,'utf-8')
                }
            }
        }
    }
}

3.6. collectIncludes

rmi2hi Jody Winter Jody Winter Ralf D. Müller Schrotti Tim Riemer Alexander Schwartz Julian Elve Ralf D. Mueller

2 minutes to read
About This Task

This task crawls through your entire project looking for AsciiDoc files with a specific name pattern, then creates a single AsciiDoc file which includes only those files.

When you create modular documentation, most includes are static. For example, the arc42-template has 12 chapters and a master template that includes those 12 chapters.

Normally when you work with dynamic modules like ADRs (Architecture Decision Records) you create those files on the fly. Maybe not within your /src/docs folder, but alongside the code file for which you wrote the ADR. In order to include these files in your documentation, you have to add the file with its whole relative path to one of your AsciiDoc files.

This task will handle it for you!

Just stick to this file-naming pattern ^[A-Z]{3,}[-_].* (begin with at least three uppercase letters and a dash/underscore) and this task will collect the file and write it to your build folder. You only have to include this generated file from within your documentation. If you provide templates for the documents, those templates are skipped if the name matches the pattern ^.\*[-\_][tT]emplate [-\_].*.

The Optional Parameter Configurations

You can configure which files are found by the script be setting the paramters in the Config.groovy file.

collectIncludes = [:]

collectIncludes.with {

fileFilter = "adoc" // define which files are considered. default: "ad|adoc|asciidoc"
minPrefixLength = "3" // define what minimum length the prefix. default: "3"
maxPrefixLength = "3" // define what maximum length the prefix. default: ""
separatorChar = "_" // define the allowed separators after prefix. default: "-_"
    cleanOutputFolder = true // should the output folder be emptied before generation? defailt: false
}
Example

You have a file called:

/src/java/yourCompany/domain/books/ADR-1-whyWeUseTheAISINInsteadOFISBN.adoc

The task will collect this file and write another file called:

/build/docs/_includes/ADR_includes.adoc

…​which will look like this:

include::../../../src/java/yourCompany/domain/books/ADR-1-whyWeUseTheAISINInsteadOFISBN.adoc[]

Obviously, you’ll reap the most benefits if the task has several ADR files to collect. 😎

You can then include these files in your main documentation by using a single include:

include::{targetDir}/docs/_includes/ADR_includes.adoc[]
Source
scripts/collectIncludes.gradle
import static groovy.io.FileType.*
import java.security.MessageDigest

task collectIncludes(
        description: 'collect all ADRs as includes in one file',
        group: 'docToolchain'
) {
    doFirst {
        boolean cleanOutputFolder = config.collectIncludes.cleanOutputFolder?:false
        String outputFolder = targetDir + '/_includes'
        if (cleanOutputFolder){
            delete fileTree(outputFolder)
        }

        new File(outputFolder).mkdirs()
    }
    doLast {
        //let's search the whole project for files, not only the docs folder
        //could be a problem with node projects :-)

        //running as subproject? set scandir to main project
        if (project.name!=rootProject.name && scanDir=='.') {
            scanDir = project(':').projectDir.path
        }
        if (docDir.startsWith('.')) {
            docDir = file(new File(projectDir, docDir).canonicalPath)
        }
        logger.info "docToolchain> docDir: ${docDir}"
        logger.info "docToolchain> scanDir: ${scanDir}"
        if (scanDir.startsWith('.')) {
            scanDir = file(new File(docDir, scanDir).canonicalPath)
        } else {
            scanDir = file(new File(scanDir, "").canonicalPath)
        }
        logger.info "docToolchain> scanDir: ${scanDir}"

        logger.info "docToolchain> includeRoot: ${includeRoot}"

        if (includeRoot.startsWith('.')) {
            includeRoot = file(new File(docDir, includeRoot).canonicalPath)
        }
        logger.info "docToolchain> includeRoot: ${includeRoot}"

        File sourceFolder = scanDir
        println "sourceFolder: " + sourceFolder.canonicalPath
        def collections = [:]

        String fileFilter = config.collectIncludes.fileFilter?:"ad|adoc|asciidoc"
        String minPrefixLength = config.collectIncludes.minPrefixLength?:"3"
        String maxPrefixLength = config.collectIncludes.maxPrefixLength?:""
        String separatorChar = config.collectIncludes.separatorChar?:"-_"

        String prefixRegEx = "[A-Z]{" + minPrefixLength + "," + maxPrefixLength + "}"
        String separatorCharRegEx = "[" + separatorChar + "]"
        String fileFilterRegEx = "^" + prefixRegEx + separatorCharRegEx + ".*[.](" + fileFilter + ")\$"

        logger.info "considering files with this pattern: " + fileFilterRegEx

        sourceFolder.traverse(type: FILES) { file ->
            if (file.name ==~ fileFilterRegEx) {
                String typeRegEx = "^(" + prefixRegEx + ")" + separatorCharRegEx + ".*\$"
                def type = file.name.replaceAll(typeRegEx,'\$1')
                if (!collections[type]) {
                    collections[type] = []
                }
                logger.info "file: " + file.canonicalPath
                def fileName = (file.canonicalPath - scanDir.canonicalPath)[1..-1]
                if (file.name ==~ '^.*[Tt]emplate.*$') {
                    logger.info "ignore template file: " + fileName
                } else {
                    String includeFileRegEx = "^.*" + prefixRegEx + "_includes.adoc\$"
                    if (file.name ==~ includeFileRegEx) {
                        logger.info "ignore generated _includes files: " + fileName
                    } else {
                        if ( fileName.startsWith('docToolchain') || fileName.replace("\\", "/").matches('^.*/docToolchain/.*$')) {
                            //ignore docToolchain as submodule
                        } else {
                            logger.info "include corrected file: " + fileName
                            collections[type] << fileName
                        }
                    }
                }
            }
        }
        println "targetFolder: " + (targetDir - docDir)
        logger.info "targetDir - includeRoot: " + (targetDir - includeRoot)
        def pathDiff = '../' * ((targetDir - docDir)
                                    .replaceAll('^/','')
                                    .replaceAll('/$','')
                                    .replaceAll("[^/]",'').size()+1)

        logger.info "pathDiff: " + pathDiff
        collections.each { type, fileNames ->
            if (fileNames) {
                def outFile = new File(targetDir + '/_includes', type + '_includes.adoc')
                logger.info outFile.canonicalPath-sourceFolder.canonicalPath
                outFile.write("// this is autogenerated\n")
                fileNames.sort().each { fileName ->
                    outFile.append ("include::../" + pathDiff + fileName.replace("\\", "/")+"[]\n\n")
                }
            }
        }
    }
}

3.7. generatePDF

Ralf D. Müller Jody Winter stehlih Ralf D. Müller Alexander Schwartz Dr. Stefan Pfeiffer Lars Francke

2 minutes to read
At a Glance
About This Task

This task makes use of the asciidoctor-pdf plugin to render your documents as pretty PDF files. Files are written to build/pdf. The PDF is generated directly from your AsciiDoc sources. There is no need for an intermediate format or other tools. The result looks more like a nicely rendered book than a print-to-PDF HTML page.

For a file to be rendered, it has to be configured in the doctoolchainConfig.groovy file. There you will find a section that looks like this:

inputFiles = [
        [file: 'manual.adoc',       formats: ['html','pdf']],
        /** inputFiles **/
]

Add the files that you want to be rendered, along with the desired format. In this case pdf.

Hint

Why do you need to configure the files to be rendered?

Asciidoctor renders all .adoc files by default. It doesn’t matter if they are the main documents or chapters you want to include. Most people only want to convert selected files to PDF, so that’s why you need to configure which ones.

Creating a Custom PDF Theme

If you want to change colors, fonts or page headers and footers, you can do so by creating a custom-theme.yml file. Copy the file src/docs/pdfTheme/custom-theme.yml from docToolchain to your project and reference it from your main .adoc`file by setting the `:pdf-themesdir:. In addition, set the :pdf-theme: to the name of your theme. In this case custom. For example, insert the following at the top of your document to reference custom-theme.yml from the /pdfTheme folder.

:pdf-themesdir: ../pdfTheme
:pdf-theme: custom
Further Reading and Resources
Source
AsciiDocBasics.gradle
task generatePDF (
        type: AsciidoctorTask,
        group: 'docToolchain',
        description: 'use pdf as asciidoc backend') {

    attributes (
        'plantUMLDir'         : file("${docDir}/${config.outputPath}/pdf/images/plantUML/").path,
    )

    attributes (
        'data-uri': 'true',
        'plantUMLDir'         : file("${docDir}/${config.outputPath}/images/").path,
        'imagesoutdir'        : file("${docDir}/${config.outputPath}/images/").path
    )

    def sourceFilesPDF = sourceFiles.findAll {
        'pdf' in it.formats
    }
//    onlyIf {
//        sourceFilesPDF
//    }

    sources {
        sourceFilesPDF.each {
            include it.file
            logger.info it.file
        }
    }

    backends = ['pdf']
    /**
    //check if a remote pdfTheme is defined
    def pdfTheme = System.getenv('DTC_PDFTHEME')
    def themeFolder = pdfTheme.md5()
    if (pdfTheme) {
        //check if it is already installed
        //TODO: finish this...
    }
    **/
    doFirst {
            if (sourceFilesPDF.size()==0) {
                throw new Exception ("""

    >> No source files defined for type PDF.
    >> Please specify at least one inputFile in your docToolchainConfig.groovy
    """)
            }

    }
}

3.8. generateSite

Jeremie Bresson Jody Winter Ralf D. Müller Schrotti Ralf D. Müller

8 minutes to read
About This Task

When you have only one document, the output of generateHTML might meet your requirements. But as your documentation grows, and you have multiple documents, you will need a microsite which bundles all of the information. The generateSite task uses jBake to create a static site with a landing page, a blog and search.

Pages

The microsite is page-oriented, not document-oriented. If you have already organized your documents by chapter, use them as pages to create a great user experience. The arc42-template sources are a good example.

To include a page in the microsite, add a metadata header to it.

page metadata
:jbake-menu: arc42
:jbake-title: Solution Strategy
:jbake-order: 4
:jbake-type: page_toc
:jbake-status: published
:filename: 015_tasks/03_task_generateSite.adoc

:toc:

[[section-solution-strategy]]
=== Solution Strategy

Here is an overview of each element:

jbake-menu

The top-level menu’s code for this page. Defaults to the top-level folder name (without the order prefix) of the .adoc file within the docDir. Example: if the top-level folder name is 10_news it will default to the value news.

For each code, the display text and the order in the top-level menu can be configured.

jbake-title

The title to be displayed in the drop-down top-level menu. Defaults to the first headline of the file.

jbake-order

Applies a sort order to drop-down entries. Defaults to a prefixed file number, such as 04_filename.adoc or to the prefixed number of the second-level folder name. When nothing is defined the default value is -1 or -987654321 for index pages.

jbake-type

The page type. Controls which template is used to render the page. You will mostly use page for a full-width page or page_toc for a page with a table of contents (toc) rendered on the left. Defaults to page_toc.

jbake-status

Either draft or published. Only published pages will be rendered. Defaults to published for files with a jbake-order and draft for files without jbake-order or files prefixed with _.

filename

Required for edit and feedback links (coming soon). Defaults to the filename :-).

ifndef

Fixes the imagesdir according to the nesting level of your docs folder. Defaults to the main docDir/images.

toc

For :jbake-type: page_toc, you need this line to generate the toc.

Start your pages with a == level headline. You can fix the level offset when you include the page in a larger document with include::chapter.adoc[leveloffset=+1].
Configuration

The configuration follows the convention-over-configuration approach. If you follow the conventions, you don’t have to configure anything. But if you want to, you can override the convention behaviour with a configuration.

Menu
Navigation elements

The navigation is organized with following elements:

  • A top level menu.

  • For each item of this top level menu, a section sidebar on the left.

The location of a page in the top-level menu and in the section sidebar depends from:

  • Its location in the folder structure

  • Page attributes

  • Site configurations

Example:

src/docs/
├── 10_foo
│   ├── 10_first.adoc
│   └── 20_second.adoc
└── 20_bar
    ├── 10_lorem.adoc
    ├── 20_ipsum
    │   ├── 10_topic-A.adoc
    │   └── 20_topic-B.adoc
    └── 30_delis
        ├── 10_one.adoc
        ├── 20_two.adoc
        └── index.adoc

The top level folders (10_foo and 20_bar) are used to determine to which menu-code the page belongs (foo and bar, unless overridden by the :jbake-menu: inside each page).

In the section sidebar, the navigation tree is determined by the folder structure. Folders are nodes in the sidebar tree. Each node can contains pages (leafs) or other folders (child nodes). The order is controlled by the prefix of the file or folder name (unless overridden by the :jbake-order: inside each page).

When an index page is present (like 20_bar/30_delis/index.adoc in the example) then the navigation tree node corresponds to this index page (you can click on it and the title is taken from the page). When this index.adoc does not declare a specific order with :jbake-order: then the order of the parent folder (for the example: 30 because the folder is named 30_delis).

When the index page is absent (like there is no 20_bar/20_ipsum/index.adoc in the example) then the name of the folder is used to create the node, and you can not click on the node because no pages is associated with this node. You can still define the order with the name (for the example 20 because the folder is named 20_ipsum).

When there is no sub-folder, only a flat list of pages is created.

When an index.adoc page is defined inside the top level folder (like: 10_foo/index.adoc or 20_bar/index.adoc) then the page will listed in the section navigation tree in the sidebar as any other regular page. By default it will be the first element of the tree, unless the value is overridden by a :jbake-order: attribute.
Configuring the top level menu

The :jbake-menu: is only the code for the menu entry to be created. You can map these codes to menu entries through the configuration (microsite-Section) in the following way:

menu = [code1: 'Some Title 1', code2: 'Other Title 2', code3: '-']

When no mapping is defined in the configuration file, the code is used as title.

The menu configuration is also impacting the display order.

If you have four files, located in following folder structure:

src/docs
├── code1
│   ├── demo1.adoc
│   └── demo2.adoc
└── code3
    ├── demo3.adoc
    └── _demo4.adoc

Where demo1.adoc and demo3.adoc contain no :jbake-menu: header, demo2.adoc contains :jbake-menu: code2, then:

  • demo1.adoc will have a menu-code of code1 because it is located in the folder code1. This code is translated through the configuration to the menu named Some Title 1.

  • demo2.adoc is in the same folder, but the :jbake-menu: attribute has a higher precedence which results in menu-code code2. This code is translated through the configuration to the menu named Other Title 2.

  • demo3.adoc will have a menu-code code3 because it is located in the folder code3. This code is translated through the configuration to the special menu - which will not be displayed. This is an easy way to hide a menu in the rendered microsite.

  • _demo4.adoc starts with an underscore _ and thus will be handled as draft (:jbake-status: draft). It will not be rendered as part of any menu, but it will be available in the microsite as "hidden" _demo4-draft.html. Feel free to remove these draft renderings before you publish your microsite.

In the column on the right, links are driven by the values defined in docToolchainConfig.groovy.

  • "Improve this doc": displayed when gitRepoUrl is set.

  • "Create an issue": displayed when issueUrl is set.

Configuring the JBake plugin

Behind the scene the generateSite task is relying on Jbake.

In the docToolchainConfig.groovy it is possible to amend the configuration of the jbake gradle plugin:

  • Add additional asciidoctorj plugins (add dependencies to the jbake configuration)

  • Add additional asciidoctor attributes

jbake configuration
//customization of the Jbake gradle plugin used by the generateSite task
jbake.with {
    // possibility to configure additional asciidoctorj plugins used by jbake
    plugins = [ ]

    // possibiltiy to configure additional asciidoctor attributes passed to the jbake task
    asciidoctorAttributes = [ ]
}

The plugins are retrieved from a repository (by default maven-central) configured with project property depsMavenRepository. When a repository requiring credentials is used the properties depsMavenUsername and depsMavenPassword can be set as well.

Templates and Style

The jBake templates and CSS are hidden for convenience.

The basic template uses Twitter Bootstrap 5 as its CSS framework. Use the copyThemes task to copy all hidden jBake resources to your project. You can then remove the resources you don’t need, and change those you want to change.

copyThemes overwrites existing files, but because your code is safely managed using version control, this shouldn’t be a problem.
Landing Page

Place an index.gsp page as your landing page in src/site/templates. This landing page is plain HTML5 styled with Twitter Bootstrap. The page header and footer are added by docToolchain. An example can be found at copyThemes or on GitHub.

Blog

The microsite also contains a simple but powerful blog. Use it to inform your team about changes, as well as architecture decision records (ADRs).

To create a new blog post, create a new file in src/docs/blog/<year>/<post-name>.adoc with the following template:

blog post template
:jbake-title: <title-of your post>
:jbake-date: <date formatted as 2021-02-28>
:jbake-type: post
:jbake-tags: <blog, asciidoc>
:jbake-status: published

:imagesdir: ../../images

== {jbake-title}
{jbake-author}
{jbake-date}

<insert your text here>

The microsite does not have its own local search. But it does have a search input field which can be used to link to another search engine.

CI/CD

When running in an automated build, set the environment variable DTC_HEADLESS to true or 1. This stops docToolchain from asking to install the configured theme, and it will simply assume that you do want to install it. You can also avoid the theme downloading with every build by copying the themes folder from $HOME/.doctoolchain/themes to the corresponding folder in your build container.

Further Reading and Resources

Read about the previewSite task here.

Source
scripts/generateSite.gradle
import groovy.util.*
import static groovy.io.FileType.*

buildscript {
    repositories {
        maven {
            credentials {
                username mavenUsername
                password mavenPassword
            }
            url mavenRepository
        }
    }
    dependencies {
        classpath 'org.asciidoctor:asciidoctorj-diagram:2.0.2'
    }
}
repositories {
    maven {
        credentials {
            username depsMavenUsername
            password depsMavenPassword
        }
        url depsMavenRepository
    }
}
dependencies {
    jbake 'org.asciidoctor:asciidoctorj-diagram:2.0.2'
    jbake 'io.pebbletemplates:pebble:3.1.2'
    config.jbake.plugins.each { plugin ->
        jbake plugin
    }
}

apply plugin: 'org.jbake.site'
apply plugin: 'org.gretty'

def color = { color, text ->
    def colors = [black: 30, red: 31, green: 32, yellow: 33, blue: 34, magenta: 35, cyan: 36, white: 37]
    return new String((char) 27) + "[${colors[color]}m${text}" + new String((char) 27) + "[0m"
}

jbake {
    version = '2.6.7'
    srcDirName = "${targetDir}/microsite/tmp/site"
    destDirName = "${targetDir}/microsite/output"
    configuration['asciidoctor.option.requires'] = "asciidoctor-diagram"
    config.microsite.each { key, value ->
        configuration['site.'+key-'config.microsite.'] = value?:''
        //println 'site.'+key-'config.microsite.' +" = "+ value
    }

    configuration['asciidoctor.attributes'] = [
        "sourceDir=${targetDir}",
        'source-highlighter=prettify@',
        //'imagesDir=../images@',
        "imagesoutDir=${targetDir}/microsite/output/images@",
        "imagesDir=${config.microsite.contextPath}/images@",
        "targetDir=${targetDir}",
        "docDir=${docDir}",
        "projectRootDir=${new File(docDir).canonicalPath}@",
    ]
    if(config.jbake.asciidoctorAttributes) {
        config.jbake.asciidoctorAttributes.each { entry ->
            configuration['asciidoctor.attributes'] << entry
        }
    }

}
bakePreview {
    port = '8046'
}
gretty {
    httpPort = "${config.microsite.previewPort?:8042}" as Integer
    contextPath = "${config.microsite.contextPath}"
    extraResourceBases = ["${targetDir}/microsite/output"]
}


def prepareAndCopyTheme = {
    //copy internal theme
    println "copy internal theme ${new File(projectDir, 'src/site').canonicalPath}"
    copy {
        from('src/site')
        into("${targetDir}/microsite/tmp/site")
    }
    //check if a remote pdfTheme is defined
    def siteTheme = System.getenv('DTC_SITETHEME')?:""
    def themeFolder = new File(projectDir, "../themes/" + siteTheme.md5())
    try {
        if (siteTheme) {
            println "use siteTheme $siteTheme"
            //check if it is already installed
            if (!themeFolder.exists()) {
                if (System.getenv('DTC_HEADLESS')) {
                    ant.yesno = "y"
                } else {
                    println "${color 'green', """\nTheme '$siteTheme' is not installed yet. """}"
                    def input = ant.input(message: """
${color 'green', 'do you want me to download and install it to '}
${color 'green', '   ' + themeFolder.canonicalPath}
${color 'green', 'for you?'}\n""",
                        validargs: 'y,n', addproperty: 'yesno')
                }
                if (ant.yesno == "y") {
                    themeFolder.mkdirs()
                    download.run {
                        src siteTheme
                        dest new File(themeFolder, 'siteTheme.zip')
                        overwrite true
                    }
                    copy {
                        from zipTree(new File(themeFolder, 'siteTheme.zip'))
                        into themeFolder
                    }
                    delete {
                        delete new File(themeFolder, 'siteTheme.zip')
                    }

                } else {
                    println "${color 'green', """\nI will continue without the theme for now... """}"
                    siteTheme = ""
                }

            }
            //copy external theme
            if (siteTheme) {
                copy {
                    from(themeFolder) {}
                    into("${targetDir}/microsite/tmp/")
                }
                //check if the config has to be updated
                // check if config still contains /** microsite **/
                def configFile = new File(docDir, mainConfigFile)
                def configFileText = configFile.text
                if (configFileText.contains("/** start:microsite **/")) {
                    def configFragment = new File(targetDir,'/microsite/tmp/site/configFragment.groovy')
                    if (configFragment.exists()) {
                        println "${color 'green', """
    It seems that this theme is used for the first time in this project.
    Let's configure it!
    If you are unsure, change these settings later in your config file
    $configFile.canonicalPath
    """}"
                        def comment = ""
                        def conf = ""
                        def example = ""
                        def i = 0
                        configFragment.eachLine { line ->
                            if (line.trim()) {
                                if (line.startsWith("//")) {
                                    conf += "    " + line + "\n"
                                    def tmp = line[2..-1].trim()
                                    comment += color('green', tmp) + "\n"
                                    if (tmp.toLowerCase().startsWith("example")) {
                                        example = tmp.replaceAll("[^ ]* ", "")
                                    }
                                } else {
                                    //only prompt if there is something to prompt
                                    if (line.contains("##")) {
                                        def property = line.replaceAll("[ =].*", "")
                                        if (!example) {
                                            example = config.microsite[property]
                                        }
                                        comment = color('blue', "$property") + "\n" + comment
                                        if (example) {
                                            ant.input(message: comment,
                                                addproperty: 'res' + i, defaultvalue: example)
                                        } else {
                                            ant.input(message: comment,
                                                addproperty: 'res' + i)
                                        }
                                        (comment, example) = ["", ""]
                                        line = line.replaceAll("##.+##", ant['res' + i])
                                        conf += "    " + line + "\n"
                                        i++
                                    } else {
                                        conf += "    " + line + "\n"
                                    }
                                }
                            } else {
                                conf += "\n"
                            }
                        }
                        configFile.write(configFileText.replaceAll("(?sm)/[*][*] start:microsite [*][*]/.*/[*][*] end:microsite [*][*]/", "%%marker%%").replace("%%marker%%", conf))
                        println color('green', "config written\ntry\n ./dtcw generateSite previewSite\nto see your microsite!")
                    }
                    //copy the dummy docs (blog, landing page) to the project repository
                    copy {
                        from(new File(themeFolder, 'site/doc')) {}
                        into(new File(docDir, inputPath))
                    }
                }
            }
        }
    } catch (Exception e) {
        println color('red', e.message)
        if (e.message.startsWith("Not Found")) {
            themeFolder.deleteDir()
            throw new GradleException("Couldn't find theme. Did you specify the right URL?\n"+e.message)
        } else {
            throw new GradleException(e.message)
        }
    }
    //copy project theme
    if (config.microsite.siteFolder) {
        def projectTheme = new File(new File(docDir, inputPath), config.microsite.siteFolder)
        println "copy project theme ${projectTheme.canonicalPath}"
        copy {
            from(projectTheme) {}
            into("${targetDir}/microsite/tmp/site")
        }
    }

}

def convertAdditionalFormats = {
    if (config.microsite.additionalConverters) {
        File sourceFolder = new File(targetDir, '/microsite/tmp/site/doc')
        sourceFolder.traverse(type: FILES) { file ->
            def extension = '.' + file.name.split("[.]")[-1]
            if (config.microsite.additionalConverters[extension]) {
                def command = config.microsite.additionalConverters[extension].command
                def type = config.microsite.additionalConverters[extension].type
                def binding = new Binding([
                    file  : file,
                    config: config
                ])
                def shell = new GroovyShell(getClass().getClassLoader(), binding)
                switch (type) {
                    case 'groovy':
                        shell.evaluate(command)
                        break
                    case 'groovyFile':
                        shell.evaluate(new File(docDir, command).text)
                        break
                    case 'bash':
                        if (command=='dtcw:rstToHtml.py') {
                            // this is an internal script
                            command = projectDir.canonicalPath+'/scripts/rstToHtml.py'
                        }
                        command = ['bash', '-c', command + ' ' + file]
                        def process = command.execute([], new File(docDir))
                        process.waitFor()
                        if (process.exitValue()) {
                            def error = process.err.text
                            throw new Exception("""
can't convert '${file.canonicalPath-docDir-'/build/microsite/tmp/site/doc'}':
${error}
""")
                        }
                }
            }
        }
    }

}

def parseAsciiDocAttribs = { beforeToc, origText, jbake ->
    def parseAttribs = true
    def text = ""
    origText.eachLine { line ->
        if (parseAttribs && line.startsWith(":jbake")) {
            line = (line - ":jbake-").split(": +", 2)
            jbake[line[0]] = line[1]
        } else {
            if (line.startsWith("[")) {
                // stop parsing jBake-attribs when a [source] - block starts which might contain those attribs as example
                parseAttribs = false
            }
            text += line+"\n"
            //there are some attributes which have to be set before the toc
            if (line.startsWith(":toc") ) {
                beforeToc += line+"\n"
            }
        }
    }
    return text
}

def parseOtherAttribs = { origText, jbake ->
    if (origText.contains('~~~~~~')) {
        def parseAttribs = true
        def text = ""
        origText.eachLine { line ->
            if (parseAttribs && line.contains("=")) {
                line = (line - "jbake-").split("=", 2)
                jbake[line[0]] = line[1]
            } else {
                if (line.startsWith("~~~~~~")) {
                    // stop parsing jBake-attribs when delimiter shows up
                    parseAttribs = false
                } else {
                    text += line + "\n"
                }
            }
        }
        return text
    } else {
        return origText
    }
}

def renderHeader = { fileName, jbake ->
    def header = ''
    if (fileName.toLowerCase() ==~ '^.*(html|md)$') {
        jbake.each { key, value ->
            if (key == 'order') {
                header += "jbake-${key}=${(value ?: '1') as Integer}\n"
            } else {
                if (key in ['type', 'status']) {
                    header += "${key}=${value}\n"
                } else {
                    header += "jbake-${key}=${value}\n"
                }
            }
        }
        header += "~~~~~~\n\n"

    } else {
        jbake.each { key, value ->
            if (key == 'order') {
                header += ":jbake-${key}: ${(value ?: '1') as Integer}\n"
            } else {
                header += ":jbake-${key}: ${value}\n"
            }
        }
    }
    return header
}
def fixMetaDataHeader = {
    //fix MetaData-Header
    File sourceFolder = new File(targetDir, '/microsite/tmp/site/doc')
    logger.info("sourceFolder: " + sourceFolder.canonicalPath)
    sourceFolder.traverse(type: FILES) { file ->
        if (file.name.toLowerCase() ==~ '^.*(ad|adoc|asciidoc|html|md)$') {
            if (file.name.startsWith("_") || file.name.startsWith(".")) {
                //ignore
            } else {
                def origText = file.text
                //parse jbake attributes
                def text = ""
                def jbake = [
                    status: "published",
                    order: -1,
                    type: 'page_toc'
                ]
                if (file.name.toLowerCase() ==~ '^.*(md|html)$') {
                    // we don't have a toc for md or html
                    jbake.type = 'page'
                }
                def beforeToc = ""
                if (file.name.toLowerCase() ==~ '^.*(ad|adoc|asciidoc)$') {
                    text = parseAsciiDocAttribs(beforeToc, origText, jbake)
                } else {
                    text = parseOtherAttribs(origText, jbake)
                }
                def name = file.canonicalPath - (sourceFolder.canonicalPath+File.separator)
                if (File.separator=='\\') {
                    name = name.split("\\\\")
                } else {
                    name = name.split("/")
                }
                if (name.size()>1) {
                    if (!jbake.menu) {
                        jbake.menu = name[0]
                        if (jbake.menu ==~ /[0-9]+[-_].*/) {
                            jbake.menu = jbake.menu.split("[-_]", 2)[1]
                        }
                    }
                    def docname = name[-1]
                    if (docname ==~ /[0-9]+[-_].*/) {
                        jbake.order = docname.split("[-_]",2)[0]
                        docname     = docname.split("[-_]",2)[1]
                    }
                    if (name.size() > 2) {
                        if ((jbake.order as Integer)==0) {
                            // let's take the order from the second level dir or file and not the file
                            def secondLevel = name[1]
                            if (secondLevel ==~ /[0-9]+[-_].*/) {
                                jbake.order = secondLevel.split("[-_]",2)[0]
                            }
                        } else {
                            if (((jbake.order?:'1') as Integer) > 0) {
                                //
                            } else {
                                jbake.status = "draft"
                            }
                        }
                    }
                    if (jbake.order==-1 && docname.startsWith('index')) {
                        jbake.order = -987654321 // special 'magic value' given to index pages.
                        jbake.status = "published"
                    }
                    // news blog
                    if (jbake.order==-1 && jbake.type=='post') {
                        jbake.order = 0
                        try {
                            jbake.order = Date.parse("yyyy-MM-dd", jbake.date).time / 100000
                        } catch ( Exception e) {
                            System.out.println "unparsable date ${jbake.date} in $name"
                        }
                        jbake.status = "published"
                    }
                    def leveloffset = 0
                    if (file.name.toLowerCase() ==~ '^.*(ad|adoc|asciidoc)$') {
                        text.eachLine { line ->
                            if (!jbake.title && line ==~ "^=+ .*") {
                                jbake.title = (line =~ "^=+ (.*)")[0][1]
                                def level = (line =~ "^(=+) .*")[0][1]
                                if (level == "=") {
                                    leveloffset = 1
                                }
                            }
                        }
                    } else {
                        if (file.name.toLowerCase() ==~ '^.*(html)$') {
                            if (!jbake.title) {
                                text.eachLine { line ->
                                    if (!jbake.title && line ==~ "^<h[1-9]>.*</h.*") {
                                        jbake.title = (line =~ "^<h[1-9]>(.*)</h.*")[0][1]
                                    }
                                }
                            }
                        } else {
                            // md
                            if (!jbake.title) {
                                text.eachLine { line ->
                                    if (!jbake.title && line ==~ "^#+ .*") {
                                        jbake.title = (line =~ "^#+ (.*)")[0][1]
                                    }
                                }
                            }
                        }
                    }
                    if (!jbake.title) {
                        jbake.title = docname
                    }
                    if (leveloffset==1) {
                        //leveloffset needed
                        // we always start with "==" not with "="
                        // only used for adoc
                        text = text.replaceAll("(?ms)^(=+) ", '$1= ')
                    }
                    if (config.microsite.customConvention) {
                        def binding = new Binding([
                            file  : file,
                            sourceFolder : sourceFolder,
                            config: config,
                            headers : jbake
                        ])
                        def shell = new GroovyShell(getClass().getClassLoader(), binding)
                        shell.evaluate(config.microsite.customConvention)
                        System.out.println jbake

                    }
                    def header = renderHeader(file.name, jbake)
                    if (file.name.toLowerCase() ==~ '^.*(ad|adoc|asciidoc)$') {
                        file.write(header + "\nifndef::dtc-magic-toc[]\n:dtc-magic-toc:\n$beforeToc\n\n:toc: left\n\n++++\n<!-- endtoc -->\n++++\nendif::[]\n" + text, "utf-8")
                    } else {
                        file.write(header + "\n" + text, "utf-8")
                    }
                }
            }
        }
    }

}

task generateSite(
    group: 'docToolchain',
    description: 'generate a microsite using jBake.') {
    doLast {
        new File("${targetDir}/microsite/tmp").mkdirs()
        println new File("${targetDir}/microsite/tmp/").canonicalPath

        prepareAndCopyTheme()

        //copy docs
        copy {
            from(new File(docDir, inputPath)) {}
            into("${targetDir}/microsite/tmp/site/doc")
        }

        // if configured, convert restructuredText or anything else
        convertAdditionalFormats()

        // convention over configuration
        fixMetaDataHeader()

    }
}
task previewSite(
    group: 'docToolchain',
    dependsOn: [],
    description: 'start a little webserver to preview your Microsite',
) {
    if (new File("${targetDir}/microsite/output").exists()) {
        finalizedBy 'jettyRun'
    }
    doLast {
        if (new File("${targetDir}/microsite/output").exists()) {
            // everything is fine
        } else {
            throw new GradleException(""">
> Microsite not built yet, please run './dtcw generateSite' first
>""")
        }
    }
}
task copyImages(type: Copy) {
    config.imageDirs.each { imageDir ->
        from(new File (new File(docDir, inputPath),imageDir)) {}
        logger.info ('imageDir: '+imageDir)
        into("${targetDir}/microsite/output/images")
    }
    config.resourceDirs.each { resource ->
        from(new File(file(srcDir),resource.source))
        logger.info ('resource: '+resource.source)
        into("${targetDir}/microsite/output/" + resource.target)
    }
}

bake.dependsOn copyImages
generateSite.finalizedBy bake

3.9. generateDocbook

Jody Winter Jeremie Bresson Ralf D. Müller

1 minute to read
At a Glance
generateDocBook
About This Task

A helper task, generateDocbook generates the intermediate format for convertToDocx<<>> and convertToEpub.

Source
AsciiDocBasics.gradle
task generateDocbook (
        type: AsciidoctorTask,
        group: 'docToolchain',
        description: 'use docbook as asciidoc backend') {

    def sourceFilesDOCBOOK = sourceFiles.findAll {
        'docbook' in it.formats
    }
//    onlyIf {
//        sourceFilesDOCBOOK
//    }

    sources {
        sourceFilesDOCBOOK.each {
            include it.file
            logger.info it.file
        }
    }

    backends = ['docbook']
    doFirst {
        if (sourceFilesDOCBOOK.size()==0) {
            throw new Exception ("""

    >> No source files defined for type docbook.
    >> Please specify at least one inputFile in your docToolchainConfig.groovy
    """)
        }

    }
}

3.10. generateDeck

Jody Winter Jeremie Bresson Ralf D. Müller Dr. Stefan Pfeiffer

1 minute to read
At a Glance
generateDeck
About This Task

This task makes use of the asciidoctor-reveal.js backend to render your documents into a HTML-based presentation. It creates a PowerPoint presentation, then enriches it by adding reveal.js slide definitions in AsciiDoc to the speaker notes. For best results, use this task with the exportPPT task.

Source
AsciiDocBasics.gradle
task generateDeck (
        type: AsciidoctorTask,
        group: 'docToolchain',
        description: 'use revealJs as asciidoc backend to create a presentation') {

    attributes (
            'idprefix': 'slide-',
            'idseparator': '-',
            'docinfo1': '',
            'revealjs_theme': 'black@',
            'revealjs_progress': 'true@',
            'revealjs_touch': 'true@',
            'revealjs_hideAddressBar': 'true@',
            'revealjs_transition': 'linear@',
            'revealjs_history': 'true@',
            'revealjs_slideNumber': 'true@'
    )
    options template_dirs : [new File(new File (projectDir,'/resources/asciidoctor-reveal.js'),'templates').absolutePath ]

    def sourceFilesREVEAL = sourceFiles.findAll {
        'revealjs' in it.formats
    }
//    onlyIf {
//        sourceFilesREVEAL
//    }

    sources {
        sourceFilesREVEAL.each {
            include it.file
            logger.info it.file
        }
    }

    outputDir = file(targetDir+'/decks/')

    resources {
        from('resources') {
            include 'reveal.js/**'
        }
        from(sourceDir) {
            include 'images/**'
        }
        into("")
        logger.info "${docDir}/${config.outputPath}/images"
    }
    doFirst {
        if (sourceFilesREVEAL.size()==0) {
            throw new Exception ("""
    >> No source files defined for type 'revealjs'.
    >> Please specify at least one inputFile in your docToolchainConfig.groovy
    """)
        }

    }
}

3.11. publishToConfluence

Rosi Martin Ralf D. Müller Jan-Niklas Vierheller Jody Winter Denny Israel Jeremie Bresson Nikolay Orozov Ralf D. Müller Vladi Joseph Staub Peter Stange oliver drewes Nils Mahlstaedt sabatmonk Heiko Stehli Alexander Herrmann Tim Riemer Luis Muniz Dierk Höppner bit-jkraushaar Alexander Schwartz Constantin Alfon Thomas Grabietz Lars Francke David M. Carr

7 minutes to read
At a Glance
publishToConfluence
About This Task

This task takes a generated HTML file, splits it by headline, and pushes it to Confluence. This lets you use the docs-as-code approach even if your organisation insists on using Confluence as its main document repo.

Special Features
Easy Code Block Conversion

[source]-blocks are converted to code-macro blocks in Confluence.

Code-macro blocks in Confluence throw an error if the language attribute contains an unknown language. Get a list of valid languages (and learn how to add others) here.
Minimal Impact on Non-Techie Confluence Users

Only pages and images that changed between task runs are published, and only those changes are notified to page watchers, cutting down on 'spam'.

Keywords Automatically Attached as Labels

:keywords: are attached as labels to every Confluence page generated using the publishToConfluence task. See Atlassian’s own guidelines on labels. Several keywords are allowed, and they must be separated by commas. For example: :keywords: label_1, label-2, label3, …​. Labels (keywords) must not contain a space character. Use either '_' or '-'.

Configuration

We tried to make the configuration self-explanatory, but below is some more information about each config option.

input

is an array of files to upload to Confluence with the ability to configure a different parent page for each file.

Attributes
  • file: absolute or relative path to the asciidoc generated html file to be exported

  • url: absolute URL to an asciidoc generated html file to be exported

  • ancestorName (optional): the name of the parent page in Confluence as string; this attribute has priority over ancestorId, but if page with given name doesn’t exist, ancestorId will be used as a fallback

  • ancestorId (optional): the id of the parent page in Confluence as string; leave this empty if a new parent shall be created in the space

  • preambleTitle (optional): the title of the page containing the preamble (everything before the first second level heading). Default is 'arc42'

The following four keys can also be used in the global section below

  • spaceKey (optional): page specific variable for the key of the confluence space to write to

  • 'allInOnePage' (optional): page specific variable to determine whether documentation is written to just one Confluence page or separate ones per chapter (default)

  • createSubpages (optional): page specific variable to determine whether ".sect2" sections shall be split from the current page into subpages

  • pagePrefix (optional): page specific variable, the pagePrefix will be a prefix for the page title and it’s sub-pages use this if you only have access to one confluence space but need to store several pages with the same title - a different pagePrefix will make them unique

  • pageSuffix (optional): same usage as prefix but appended to the title and it’s subpages

only 'file' or 'url' is allowed. If both are given, 'url' is ignored

ancestorId

The page ID of the parent page where you want your docs to be published. Go to this page, click Edit and the required ID will show up in the URL. Specify the ID as a string within the config file.

api

For cloud instances, [context] is wiki.

preambleTitle

The page title. Contains the preamble (everything except the first second-level heading). Default is 'arc42'.

disableToC

This boolean configuration determines whether or not the table of contents (ToC) is disabled on the page once uploaded to Confluence. false by default, so the ToC is active.

pagePrefix/pageSuffix

Confluence can’t handle two pages with the same name - even with different casing (lowercase, UPPERCASE, a mix). This script matches pages regardless of case and refuses to replace a page whose name differs from an existing page only by casing. Ideally, you should create a new Confluence space for each piece of larger documentation. If you are restricted and can’t create new spaces, you can use pagePrefix/pageSuffix to define a prefix/suffix for the doc so that it doesn’t conflict with other page names.

credentials

Use username and password, or even better - username and api-token. You can create new API-tokens in your profile. To avoid having your password or API-token versioned through Git, you can store them outside of this configuration either as an environment variable or in another file. The key here is that the config file is a groovy script. For example, you can do things like credentials = "user:${new File("/home/me/apitoken").text}".bytes.encodeBase64().toString()

There is a fallback if you need to simplify the injection of credentials from external sources. If you leave the credentials field empty, the variables confluenceUser and confluencePass from the build environment will be used for authentication. You can set these through any means allowed by gradle, such as the gradle.properties file in the project or your home directory, environment variables or command-line flags. For all the ways to set these variables, take a look at the gradle user guide.

apikey

In situations where you have to use full user authorisation because of internal Confluence permission handling, you’ll need to add the API-token in addition to the credentials. The API-token cannot be added to the credentials because it’s used for user and password exchange. Therefore the API-token can be added as parameter apikey, which makes the addition of the token a separate header field with key: keyId and value of apikey. An example (including storing of the real value outside this configuration) is: apikey = "${new File("/home/me/apitoken").text}".

extraPageContent

If you need to prefix your pages with a warning stating that 'this is generated content', this is where you do it.

enableAttachments

If value is set to true, any links to local file references will be uploaded as attachments. The current implementation only supports a single folder, the name of which will be used as a prefix to validate whether or not your file should be uploaded. If you enable this feature, and use a folder which starts with 'attachment', an adaption of this prefix is required.

pageLimit

Limits the number of pages retrieved from the server to check if a page with this name already exists.

jiraServerId Stores the Jira server ID that your Confluence instance is connected to. If a value is set, all anchors pointing to a Jira ticket will be replaced by the Confluence Jira macro. To function properly, jiraRoot must be configured (see exportJiraIssues). Here’s an example:

All files to attach will need to be linked inside the document: link:attachment/myfolder/myfile.json[My API definition]

attachmentPrefix

Stores the expected foldername of your output directory. Default is attachment.

proxy

If you need to provide proxy to access Confluence, you can set a map with the keys host (e.g. 'my.proxy.com'), port (e.g. '1234') and schema (e.g. 'http') of your proxy.

useOpenapiMacro

If this option is present and equal to confluence-open-api or swagger-open-api then any source block marked with class openapi will be wrapped in the Elitesoft Swagger Editor macro (see Elitesoft Swagger Editor). The key depends on the version of the macro.

For backward compatibility, if this option is present and equal to true, then again the Elitesoft Swagger Editor macro will be used.

If this option is present and equal to "open-api" then any source block marked with class openapi will be wrapped in Open API Documentation for Confluence macro: (see Open API Documentation for Confluence). A download source (yaml) button is shown by default.
Using the plugin can be handled on different ways.

  • copy/paste the content of the YAML file to the plugin without linking to the origin source by using the url to the YAML file

    [source.openapi,yaml]
    ----
    \include::https://my-domain.com/path-to-yaml[]
    ----
  • copy/paste the content of the YAML file to the plugin without linking to the origin source by using a YAML file in your project structure:

    [source.openapi,yaml]
    ----
    \include::my-yaml-file.yaml[]
    ----
  • create a link between the plugin and the YAML file without copying the content into the plugin. The advantage following this way is that even in case the API specification is changed without re-generating the documentation, the new version of the configuration is used in Confluence.

    [source.openapi,yaml,role="url:https://my-domain.com/path-to-yaml"]
    ----
    \include::https://my-domain.com/path-to-yaml[]
    ----
publishToConfluence.gradle
//Configureation for publishToConfluence

confluence = [:]

// 'input' is an array of files to upload to Confluence with the ability
//          to configure a different parent page for each file.
//
// Attributes
// - 'file': absolute or relative path to the asciidoc generated html file to be exported
// - 'url': absolute URL to an asciidoc generated html file to be exported
// - 'ancestorName' (optional): the name of the parent page in Confluence as string;
//                             this attribute has priority over ancestorId, but if page with given name doesn't exist,
//                             ancestorId will be used as a fallback
// - 'ancestorId' (optional): the id of the parent page in Confluence as string; leave this empty
//                            if a new parent shall be created in the space
//                            Set it for every file so the page scanning is done only for the given ancestor page trees.
// - 'preambleTitle' (optional): the title of the page containing the preamble (everything
//                            before the first second level heading). Default is 'arc42'
//
// The following four keys can also be used in the global section below
// - 'spaceKey' (optional): page specific variable for the key of the confluence space to write to
// - 'createSubpages' (optional): page specific variable to determine whether ".sect2" sections shall be split from the current page into subpages
// - 'pagePrefix' (optional): page specific variable, the pagePrefix will be a prefix for the page title and it's sub-pages
//                            use this if you only have access to one confluence space but need to store several
//                            pages with the same title - a different pagePrefix will make them unique
// - 'pageSuffix' (optional): same usage as prefix but appended to the title and it's subpages
// only 'file' or 'url' is allowed. If both are given, 'url' is ignored
confluence.with {
    input = [
            [ file: "build/docs/html5/arc42-template-de.html" ],
    ]

    // endpoint of the confluenceAPI (REST) to be used
    // verfiy that you got the correct endpoint by browsing to
    // https://[yourServer]/[context]/rest/api/user/current
    // you should get a valid json which describes your current user
    // a working example is https://arc42-template.atlassian.net/wiki/rest/api/user/current
    api = 'https://[yourServer]/[context]/rest/api/'

//    Additionally, spaceKey, createSubpages, pagePrefix and pageSuffix can be globally defined here. The assignment in the input array has precedence

    // the key of the confluence space to write to
    spaceKey = 'asciidoc'

    // the title of the page containing the preamble (everything the first second level heading). Default is 'arc42'
    preambleTitle = ''

    // variable to determine whether the whole document should be uploaded as just one page or split into separate
    // pages per chapter
    allInOnePage = false

    // variable to determine whether ".sect2" sections shall be split from the current page into subpages
    createSubpages = false

    // the pagePrefix will be a prefix for each page title
    // use this if you only have access to one confluence space but need to store several
    // pages with the same title - a different pagePrefix will make them unique
    pagePrefix = ''

    pageSuffix = ''

    /*
    WARNING: It is strongly recommended to store credentials securely instead of commiting plain text values to your git repository!!!

    Tool expects credentials that belong to an account which has the right permissions to to create and edit confluence pages in the given space.
    Credentials can be used in a form of:
     - passed parameters when calling script (-PconfluenceUser=myUsername -PconfluencePass=myPassword) which can be fetched as a secrets on CI/CD or
     - gradle variables set through gradle properties (uses the 'confluenceUser' and 'confluencePass' keys)
    Often, same credentials are used for Jira & Confluence, in which case it is recommended to pass CLI parameters for both entities as
    -Pusername=myUser -Ppassword=myPassword
    */

    //optional API-token to be added in case the credentials are needed for user and password exchange.
    //apikey = "[API-token]"

    // HTML Content that will be included with every page published
    // directly after the TOC. If left empty no additional content will be
    // added
    // extraPageContent = '<ac:structured-macro ac:name="warning"><ac:parameter ac:name="title" /><ac:rich-text-body>This is a generated page, do not edit!</ac:rich-text-body></ac:structured-macro>
    extraPageContent = ''

    // enable or disable attachment uploads for local file references
    enableAttachments = false

    // default attachmentPrefix = attachment - All files to attach will require to be linked inside the document.
    // attachmentPrefix = "attachment"


    // Optional proxy configuration, only used to access Confluence
    // schema supports http and https
    // proxy = [host: 'my.proxy.com', port: 1234, schema: 'http']

    // Optional: specify which Confluence OpenAPI Macro should be used to render OpenAPI definitions
    // possible values: ["confluence-open-api", "open-api", "swagger-open-api", true]. true is the same as "confluence-open-api" for backward compatibility
    // useOpenapiMacro = "confluence-open-api"
}
CSS Styling

Some AsciiDoctor features depend on specific CSS style definitions. Unless these styles are defined, some formatting that is present in the HTML version will not be represented when published to Confluence. To configure Confluence to include additional style definitions:

  1. Log in to Confluence as a space admin.

  2. Go to the desired space.

  3. Select Space tools > Look and Feel > Stylesheet.

  4. Click Edit then enter the desired style definitions.

  5. Click Save.

The default style definitions can be found in the AsciiDoc project as asciidoctor-default.css. You will most likely NOT want to include the entire thing, as some of the definitions are likely to disrupt Confluence’s layout.

The following style definitions are Confluence-compatible, and will enable the use of the built-in roles (big/small, underline/overline/line-through, COLOR/COLOR-background for the sixteen HTML color names):

.big{font-size:larger}
.small{font-size:smaller}
.underline{text-decoration:underline}
.overline{text-decoration:overline}
.line-through{text-decoration:line-through}
.aqua{color:#00bfbf}
.aqua-background{background-color:#00fafa}
.black{color:#000}
.black-background{background-color:#000}
.blue{color:#0000bf}
.blue-background{background-color:#0000fa}
.fuchsia{color:#bf00bf}
.fuchsia-background{background-color:#fa00fa}
.gray{color:#606060}
.gray-background{background-color:#7d7d7d}
.green{color:#006000}
.green-background{background-color:#007d00}
.lime{color:#00bf00}
.lime-background{background-color:#00fa00}
.maroon{color:#600000}
.maroon-background{background-color:#7d0000}
.navy{color:#000060}
.navy-background{background-color:#00007d}
.olive{color:#606000}
.olive-background{background-color:#7d7d00}
.purple{color:#600060}
.purple-background{background-color:#7d007d}
.red{color:#bf0000}
.red-background{background-color:#fa0000}
.silver{color:#909090}
.silver-background{background-color:#bcbcbc}
.teal{color:#006060}
.teal-background{background-color:#007d7d}
.white{color:#bfbfbf}
.white-background{background-color:#fafafa}
.yellow{color:#bfbf00}
.yellow-background{background-color:#fafa00}
Source
publishToConfluence.gradle
task publishToConfluence(
        description: 'publishes the HTML rendered output to confluence',
        group: 'docToolchain'
) {
    doLast {
        logger.info("docToolchain> docDir: "+docDir)

        binding.setProperty('config',config)
        binding.setProperty('docDir',docDir)
        evaluate(new File(projectDir, 'scripts/asciidoc2confluence.groovy'))
    }
}
scripts/asciidoc2confluence.groovy
/**
 * Created by Ralf D. Mueller and Alexander Heusingfeld
 * https://github.com/rdmueller/asciidoc2confluence
 *
 * this script expects an HTML document created with AsciiDoctor
 * in the following style (default AsciiDoctor output)
 * <div class="sect1">
 *     <h2>Page Title</h2>
 *     <div class="sectionbody">
 *         <div class="sect2">
 *            <h3>Sub-Page Title</h3>
 *         </div>
 *         <div class="sect2">
 *            <h3>Sub-Page Title</h3>
 *         </div>
 *     </div>
 * </div>
 * <div class="sect1">
 *     <h2>Page Title</h2>
 *     ...
 * </div>
 *
 */

/*
    Additions for issue #342 marked as #342-dierk42
    ;-)
*/

// some dependencies
/**
@Grapes(
        [@Grab('org.jsoup:jsoup:1.8.2'),
         @Grab('org.codehaus.groovy.modules.http-builder:http-builder:0.6' ),
         @Grab('org.apache.httpcomponents:httpmime:4.5.1')]
)
**/
import org.jsoup.Jsoup
import org.jsoup.parser.Parser
import org.jsoup.nodes.Entities.EscapeMode
import org.jsoup.nodes.Document
import org.jsoup.nodes.Document.OutputSettings
import org.jsoup.nodes.Element
import org.jsoup.nodes.TextNode
import org.jsoup.select.Elements
import groovyx.net.http.RESTClient
import groovyx.net.http.HttpResponseException
import groovyx.net.http.HTTPBuilder
import groovyx.net.http.EncoderRegistry
import groovyx.net.http.ContentType
import java.security.MessageDigest
import static groovy.io.FileType.FILES
//to upload attachments:
import org.apache.http.entity.mime.MultipartEntity
import org.apache.http.entity.mime.content.StringBody
import org.apache.http.entity.mime.content.InputStreamBody
import org.apache.http.entity.mime.HttpMultipartMode
import groovyx.net.http.Method

def CDATA_PLACEHOLDER_START = '<cdata-placeholder>'
def CDATA_PLACEHOLDER_END = '</cdata-placeholder>'

def baseUrl
def allPages
// configuration

def confluenceSpaceKey
def confluenceCreateSubpages
def confluenceAllInOnePage
def confluencePagePrefix
def baseApiPath = new URI(config.confluence.api).path
// helper functions

def MD5(String s) {
    MessageDigest.getInstance("MD5").digest(s.bytes).encodeHex().toString()
}

// for getting better error message from the REST-API
// LuisMuniz: return the action's result, if successful.
def trythis(Closure action) {
    try {
        action.call()
    } catch (HttpResponseException error) {
        println "something went wrong - got an http response code "+error.response.status+":"
        switch (error.response.status) {
            case '401':
                println (error.response.data.toString().replaceAll("^.*Reason","Reason"))
                println "please check your confluence credentials in config file or passed parameters"
                throw new Exception("missing authentication credentials")
                break
            case '400':
                println error.response.data.message
                println "please check the ancestorId in your config file"
                throw new Exception("Parent does not exist")
                break
            default:
                println error.response.data
        }
        null
    }
}


def parseAdmonitionBlock(block, String type) {
    content = block.select(".content").first()
    titleElement = content.select(".title")
    titleText = ''
    if(titleElement != null) {
        titleText = "<ac:parameter ac:name=\"title\">${titleElement.text()}</ac:parameter>"
        titleElement.remove()
    }
    block.after("<ac:structured-macro ac:name=\"${type}\">${titleText}<ac:rich-text-body>${content}</ac:rich-text-body></ac:structured-macro>")
    block.remove()
}

/*  #342-dierk42

    add labels to a Confluence page. Labels are taken from :keywords: which
    are converted as meta tags in HTML. Building the array: see below

    Confluence allows adding labels only after creation of a page.
    Therefore we need extra API calls.

    Currently the labels are added one by one. Suggestion for improvement:
    Build a label structure of all labels an place them with one call.

    Replaces exisiting labels. No harm
    Does not check for deleted labels when keywords are deleted from source
    document!
*/
def addLabels = { def pageId, def labelsArray ->
    //https://docs.atlassian.com/confluence/REST/latest/
    def api = new RESTClient(config.confluence.api)
    //this fixes the encoding (dierk42: Is this needed here? Don't know)
    api.encoderRegistry = new EncoderRegistry( charset: 'utf-8' )
    def headers = getHeaders()
    // Attach each label in a API call of its own. The only prefix possible
    // in our own Confluence is 'global'
    labelsArray.each { label ->
        label_data = [
            prefix : 'global',
            name : label
        ]
        trythis {
            // attach label to page pageId
            // https://developer.atlassian.com/display/CONFDEV/Confluence+REST+API+Examples#ConfluenceRESTAPIExamples-Updatingapage
            def res = api.post(contentType: ContentType.JSON,
                              path: 'content/' + pageId + "/label", body: label_data, headers: headers)
            }
        println "added label " + label + " to page ID " + pageId
    }
}


def uploadAttachment = { def pageId, String url, String fileName, String note ->
    def is
    def localHash
    if (url.startsWith('http')) {
        is = new URL(url).openStream()
        //build a hash of the attachment
        localHash = MD5(new URL(url).openStream().text)
    } else {
        is = new File(url).newDataInputStream()
        //build a hash of the attachment
        localHash = MD5(new File(url).newDataInputStream().text)
    }

    //https://docs.atlassian.com/confluence/REST/latest/
    def api = new RESTClient(config.confluence.api)
    //this fixes the encoding
    api.encoderRegistry = new EncoderRegistry( charset: 'utf-8' )
    if (config.confluence.proxy) {
        api.setProxy(config.confluence.proxy.host, config.confluence.proxy.port, config.confluence.proxy.schema ?: 'http')
    }
   def headers = getHeaders()
    //check if attachment already exists
    def result = "nothing"
    def attachment = api.get(path: 'content/' + pageId + '/child/attachment',
            query: [
                    'filename': fileName,
            ], headers: headers).data
    def http
    if (attachment.size==1) {
        // attachment exists. need an update?
        def remoteHash = attachment.results[0].extensions.comment.replaceAll("(?sm).*#([^#]+)#.*",'$1')
        if (remoteHash!=localHash) {
            //hash is different -> attachment needs to be updated
            http = new HTTPBuilder(config.confluence.api + 'content/' + pageId + '/child/attachment/' + attachment.results[0].id + '/data')
            println "    updated attachment"
        }
    } else {
        http = new HTTPBuilder(config.confluence.api + 'content/' + pageId + '/child/attachment')

    }
    if (http) {
		if (config.confluence.proxy) {
            http.setProxy(config.confluence.proxy.host, config.confluence.proxy.port, config.confluence.proxy.schema ?: 'http')
        }

        http.request(Method.POST) { req ->
            requestContentType: "multipart/form-data"
            MultipartEntity multiPartContent = new MultipartEntity(HttpMultipartMode.BROWSER_COMPATIBLE)
            // Adding Multi-part file parameter "file"
            multiPartContent.addPart("file", new InputStreamBody(is, fileName))
            // Adding another string parameter "comment"
            multiPartContent.addPart("comment", new StringBody(note + "\r\n#" + localHash + "#"))
            req.setEntity(multiPartContent)
            headers.each { key, value ->
                req.addHeader(key, value)
            }
        }
    }
}


def realTitle = { pageTitle ->
    confluencePagePrefix + pageTitle + confluencePageSuffix
}

def rewriteMarks = { body ->
    // Confluence strips out mark elements.  Replace them with default formatting.
    body.select('mark').wrap('<span style="background:#ff0;color:#000"></style>').unwrap()
}

def retrieveAllPagesByAncestorId(RESTClient api, Map headers, List<String> pageIds, String baseApiPath, int pageLimit) {
    def allPages = [:]
    def request = [
        'type' : 'page',
        'limit': pageLimit
    ]

    int start = 0
    def ids = []
    def pageId = pageIds.remove(0)
    boolean morePages = true
    while (morePages) {
        def results = trythis {
            request.start = start
            def args = [
                'headers': headers,
                'path'   : "${baseApiPath}content/${pageId}/child/page",
                'query'  : request,
            ]
            api.get(args).data
        } ?: []

        results = results.results ?: []


        results.inject(allPages) { Map acc, Map match ->
            //unique page names in confluence, so we can get away with indexing by title
            ids.add(match.id)
            acc[match.title.toLowerCase()] = [
                title   : match.title,
                id      : match.id,
                parentId: pageId
            ]
            acc
        }

        if (results.empty && ids.isEmpty()) {
            if(pageIds.isEmpty()) {
                morePages = false
            } else {
                pageId = pageIds.remove(0)
            }
        } else if (!results.empty) {
            start += results.size
        } else {
            start = 0
            pageId = ids.remove(0);
        }
    }
    allPages
}

def retrieveAllPagesBySpace(RESTClient api, Map headers, String spaceKey, String baseApiPath, int pageLimit) {
    boolean morePages = true
    int start = 0
    def request = [
        'type'    : 'page',
        'spaceKey': spaceKey,
        'expand'  : 'ancestors',
        'limit'   : pageLimit
    ]

    def allPages = [:]
    while (morePages) {
        def results = trythis {
            request.start = start
            def args = [
                'headers': headers,
                'path'   : "${baseApiPath}content",
                'query'  : request,
            ]
            api.get(args).data
        } ?: []
        results = results.results ?: []
        if (results.empty) {
            morePages = false
        } else {
            start += results.size
        }
        results.inject(allPages) { Map acc, Map match ->
            //unique page names in confluence, so we can get away with indexing by title
            def ancestors = match.ancestors.collect { it.id }
            acc[match.title.toLowerCase()] = [
                title   : match.title,
                id      : match.id,
                parentId: ancestors.isEmpty() ? null : ancestors.last()
            ]
            acc
        }
    }
    allPages
}

// #352-LuisMuniz: Helper methods
// Fetch all pages of the defined config ancestorsIds. Only keep relevant info in the pages Map
// The map is indexed by lower-case title
def retrieveAllPages = { RESTClient api, Map headers, String spaceKey ->
    if (allPages != null) {
        println "allPages already retrieved"
        allPages
    } else {
        def pageIds = []
        def checkSpace = false
        int pageLimit = config.confluence.pageLimit ? config.confluence.pageLimit : 100
        config.confluence.input.each { input ->
            if (!input.ancestorId) {
                // if one ancestorId is missing we should scan the whole space
                checkSpace = true;
                return
            }
            pageIds.add(input.ancestorId)
        }
        println (".")

        if(checkSpace) {
            allPages = retrieveAllPagesBySpace(api, headers, spaceKey, baseApiPath, pageLimit)
        } else {
            allPages = retrieveAllPagesByAncestorId(api, headers, pageIds, baseApiPath, pageLimit)
        }
        allPages
    }
}


// Retrieve a page by id with contents and version
def retrieveFullPage = { RESTClient api, Map headers, String id ->
    trythis {
        api.get(
                [
                        'headers': headers,
                        'path'   : "${baseApiPath}content/${id}",
                        'query'  : ['expand': 'body.storage,version'],
                ]
        ).data


    } ?: [:]
}

//if a parent has been specified, check whether a page has the same parent.
boolean hasRequestedParent(Map existingPage, String requestedParentId) {
    if (requestedParentId) {
        existingPage.parentId == requestedParentId
    } else {
        true
    }
}

def rewriteDescriptionLists = { body ->
    def TAGS = [ dt: 'th', dd: 'td' ]
    body.select('dl').each { dl ->
        // WHATWG allows wrapping dt/dd in divs, simply unwrap them
        dl.select('div').each { it.unwrap() }

        // group dts and dds that belong together, usually it will be a 1:1 relation
        // but HTML allows for different constellations
        def rows = []
        def current = [dt: [], dd: []]
        rows << current
        dl.select('dt, dd').each { child ->
            def tagName = child.tagName()
            if (tagName == 'dt' && current.dd.size() > 0) {
                // dt follows dd, start a new group
                current = [dt: [], dd: []]
                rows << current
            }
            current[tagName] << child.tagName(TAGS[tagName])
            child.remove()
        }

        rows.each { row ->
            def sizes = [dt: row.dt.size(), dd: row.dd.size()]
            def rowspanIdx = [dt: -1, dd: sizes.dd - 1]
            def rowspan = Math.abs(sizes.dt - sizes.dd) + 1
            def max = sizes.dt
            if (sizes.dt < sizes.dd) {
                max = sizes.dd
                rowspanIdx = [dt: sizes.dt - 1, dd: -1]
            }
            (0..<max).each { idx ->
                def tr = dl.appendElement('tr')
                ['dt', 'dd'].each { type ->
                    if (sizes[type] > idx) {
                        tr.appendChild(row[type][idx])
                        if (idx == rowspanIdx[type] && rowspan > 1) {
                            row[type][idx].attr('rowspan', "${rowspan}")
                        }
                    } else if (idx == 0) {
                        tr.appendElement(TAGS[type]).attr('rowspan', "${rowspan}")
                    }
                }
            }
        }

        dl.wrap('<table></table>')
            .unwrap()
    }
}

def rewriteInternalLinks = { body, anchors, pageAnchors ->
    // find internal cross-references and replace them with link macros
    body.select('a[href]').each { a ->
        def href = a.attr('href')
        if (href.startsWith('#')) {
            def anchor = href.substring(1)
            def pageTitle = anchors[anchor] ?: pageAnchors[anchor]
            if (pageTitle && a.text()) {
                // as Confluence insists on link texts to be contained
                // inside CDATA, we have to strip all HTML and
                // potentially loose styling that way.
                a.html(a.text())
                a.wrap("<ac:link${anchors.containsKey(anchor) ? ' ac:anchor="' + anchor + '"' : ''}></ac:link>")
                   .before("<ri:page ri:content-title=\"${realTitle pageTitle}\"/>")
                   .wrap("<ac:plain-text-link-body>${CDATA_PLACEHOLDER_START}${CDATA_PLACEHOLDER_END}</ac:plain-text-link-body>")
                   .unwrap()
            }
        }
    }
}

def rewriteJiraLinks = { body ->
    // find links to jira tickets and replace them with jira macros
    body.select('a[href]').each { a ->
        def href = a.attr('href')
        if (href.startsWith(config.jira.api + "/browse/")) {
                def ticketId = a.text()
                a.before("""<ac:structured-macro ac:name=\"jira\" ac:schema-version=\"1\">
                     <ac:parameter ac:name=\"key\">${ticketId}</ac:parameter>
                     <ac:parameter ac:name=\"serverId\">${config.confluence.jiraServerId}</ac:parameter>
                     </ac:structured-macro>""")
                a.remove()
        }
    }
}


def rewriteCodeblocks = { body ->
    body.select('pre > code').each { code ->
        if (code.attr('data-lang')) {
            code.select('span[class]').each { span ->
                span.unwrap()
            }
            code.select('i[class]').each { i ->
                i.unwrap()
            }
            code.select('b').each { b ->
                b.before(" // ")
                b.unwrap()
            }
            code.before("<ac:parameter ac:name=\"language\">${code.attr('data-lang')}</ac:parameter>")
        }
        code.parent() // pre now
            .wrap('<ac:structured-macro ac:name="code"></ac:structured-macro>')
            .unwrap()
        code.wrap("<ac:plain-text-body>${CDATA_PLACEHOLDER_START}${CDATA_PLACEHOLDER_END}</ac:plain-text-body>")
            .unwrap()
    }
}

def rewriteOpenAPI = { org.jsoup.nodes.Element body ->
    if (config.confluence.useOpenapiMacro == true || config.confluence.useOpenapiMacro == 'confluence-open-api') {
        body.select('div.openapi  pre > code').each { code ->
            def parent=code.parent()
            def rawYaml=code.wholeText()
            code.parent()
                    .wrap('<ac:structured-macro ac:name="confluence-open-api" ac:schema-version="1" ac:macro-id="1dfde21b-6111-4535-928a-470fa8ae3e7d"></ac:structured-macro>')
                    .unwrap()
            code.wrap("<ac:plain-text-body>${CDATA_PLACEHOLDER_START}${CDATA_PLACEHOLDER_END}</ac:plain-text-body>")
                    .replaceWith(new TextNode(rawYaml))
        }
    } else if (config.confluence.useOpenapiMacro == 'swagger-open-api') {
        body.select('div.openapi  pre > code').each { code ->
            def parent=code.parent()
            def rawYaml=code.wholeText()
            code.parent()
                    .wrap('<ac:structured-macro ac:name="swagger-open-api" ac:schema-version="1" ac:macro-id="f9deda8a-1375-4488-8ca5-3e10e2e4ee70"></ac:structured-macro>')
                    .unwrap()
            code.wrap("<ac:plain-text-body>${CDATA_PLACEHOLDER_START}${CDATA_PLACEHOLDER_END}</ac:plain-text-body>")
                    .replaceWith(new TextNode(rawYaml))
        }
    } else if (config.confluence.useOpenapiMacro == 'open-api') {

              def includeURL=null

              for (Element e : body.select('div .listingblock.openapi')) {
                  for (String s : e.className().split(" ")) {
                      if (s.startsWith("url")) {
                          //include the link to the URL for the macro
                          includeURL = s.replace('url:', '')
                     }
                  }
              }

              body.select('div.openapi  pre > code').each { code ->
                  def parent=code.parent()
                  def rawYaml=code.wholeText()

                  code.parent()
                      .wrap('<ac:structured-macro ac:name="open-api" ac:schema-version="1" data-layout="default" ac:macro-id="4302c9d8-fca4-4f14-99a9-9885128870fa"></ac:structured-macro>')
                      .unwrap()

                  if (includeURL!=null)
                  {
                      code.before('<ac:parameter ac:name="url">'+includeURL+'</ac:parameter>')
                  }
                  else {
                      //default: show download button
                      code.before('<ac:parameter ac:name="showDownloadButton">true</ac:parameter>')
                      code.wrap("<ac:plain-text-body>${CDATA_PLACEHOLDER_START}${CDATA_PLACEHOLDER_END}</ac:plain-text-body>")
                          .replaceWith(new TextNode(rawYaml))
                  }
              }
    }
}

def unescapeCDATASections = { html ->
    def start = html.indexOf(CDATA_PLACEHOLDER_START)
    while (start > -1) {
        def end = html.indexOf(CDATA_PLACEHOLDER_END, start)
        if (end > -1) {
            def prefix = html.substring(0, start) + CDATA_PLACEHOLDER_START
            def suffix = html.substring(end)
            def unescaped = html.substring(start + CDATA_PLACEHOLDER_START.length(), end)
                    .replaceAll('&lt;', '<').replaceAll('&gt;', '>')
                    .replaceAll('&amp;', '&')
            html = prefix + unescaped + suffix
        }
        start = html.indexOf(CDATA_PLACEHOLDER_START, start + 1)
    }
    html
}

//modify local page in order to match the internal confluence storage representation a bit better
//definition lists are not displayed by confluence, so turn them into tables
//body can be of type Element or Elements
def deferredUpload = []
def parseBody =  { body, anchors, pageAnchors ->
    rewriteOpenAPI body

    body.select('div.paragraph').unwrap()
    body.select('div.ulist').unwrap()
    body.select('div.sect3').unwrap()
    [   'note':'info',
        'warning':'warning',
        'important':'warning',
        'caution':'note',
        'tip':'tip'            ].each { adType, cType ->
        body.select('.admonitionblock.'+adType).each { block ->
            parseAdmonitionBlock(block, cType)
        }
    }
    //special for the arc42-template
    body.select('div.arc42help').select('.content')
            .wrap('<ac:structured-macro ac:name="expand"></ac:structured-macro>')
            .wrap('<ac:rich-text-body></ac:rich-text-body>')
            .wrap('<ac:structured-macro ac:name="info"></ac:structured-macro>')
            .before('<ac:parameter ac:name="title">arc42</ac:parameter>')
            .wrap('<ac:rich-text-body><p></p></ac:rich-text-body>')
    body.select('div.arc42help').unwrap()
    body.select('div.title').wrap("<strong></strong>").before("<br />").wrap("<div></div>")
    body.select('div.listingblock').wrap("<p></p>").unwrap()
    // see if we can find referenced images and fetch them
    new File("tmp/images/.").mkdirs()
    // find images, extract their URLs for later uploading (after we know the pageId) and replace them with this macro:
    // <ac:image ac:align="center" ac:width="500">
    // <ri:attachment ri:filename="deployment-context.png"/>
    // </ac:image>
    body.select('img').each { img ->
        img.attributes().each { attribute ->
            //println attribute.dump()
        }
        def src = img.attr('src')
        def imgWidth = img.attr('width')?:500
        def imgAlign = img.attr('align')?:"center"
        println "    image: "+src

        //it is not an online image, so upload it to confluence and use the ri:attachment tag
        if(!src.startsWith("http")) {
          def newUrl = baseUrl.toString().replaceAll('\\\\','/').replaceAll('/[^/]*$','/')+src
          def fileName = java.net.URLDecoder.decode((src.tokenize('/')[-1]),"UTF-8")
          newUrl = java.net.URLDecoder.decode(newUrl,"UTF-8")

          trythis {
              deferredUpload <<  [0,newUrl,fileName,"automatically uploaded"]
          }
          img.after("<ac:image ac:align=\"${imgAlign}\" ac:width=\"${imgWidth}\"><ri:attachment ri:filename=\"${fileName}\"/></ac:image>")
        }
        // it is an online image, so we have to use the ri:url tag
        else {
          img.after("<ac:image ac:align=\"imgAlign\" ac:width=\"${imgWidth}\"><ri:url ri:value=\"${src}\"/></ac:image>")
        }
        img.remove()
    }


    if(config.confluence.enableAttachments){
        attachmentPrefix = config.confluence.attachmentPrefix ? config.confluence.attachmentPrefix : 'attachment'
        body.select('a').each { link ->

            def src = link.attr('href')
            println "    attachment src: "+src

            //upload it to confluence and use the ri:attachment tag
            if(src.startsWith(attachmentPrefix)) {
                def newUrl = baseUrl.toString().replaceAll('\\\\','/').replaceAll('/[^/]*$','/')+src
                def fileName = java.net.URLDecoder.decode((src.tokenize('/')[-1]),"UTF-8")
                newUrl = java.net.URLDecoder.decode(newUrl,"UTF-8")

                trythis {
                    deferredUpload <<  [0,newUrl,fileName,"automatically uploaded non-image attachment by docToolchain"]
                }
                def uriArray=fileName.split("/")
                def pureFilename = uriArray[uriArray.length-1]
                def innerhtml = link.html()
                link.after("<ac:structured-macro ac:name=\"view-file\" ac:schema-version=\"1\"><ac:parameter ac:name=\"name\"><ri:attachment ri:filename=\"${pureFilename}\"/></ac:parameter></ac:structured-macro>")
                link.after("<ac:link><ri:attachment ri:filename=\"${pureFilename}\"/><ac:plain-text-link-body> <![CDATA[\"${innerhtml}\"]]></ac:plain-text-link-body></ac:link>")
                link.remove()

            }
        }
    }

    if(config.confluence.jiraServerId){
        rewriteJiraLinks body
    }

    rewriteMarks body
    rewriteDescriptionLists body
    rewriteInternalLinks body, anchors, pageAnchors
    //sanitize code inside code tags
    rewriteCodeblocks body
    def pageString = unescapeCDATASections body.html().trim()

    //change some html elements through simple substitutions
    pageString = pageString
            .replaceAll('<br>','<br />')
            .replaceAll('</br>','<br />')
            .replaceAll('<a([^>]*)></a>','')
            .replaceAll(CDATA_PLACEHOLDER_START,'<![CDATA[')
            .replaceAll(CDATA_PLACEHOLDER_END,']]>')

    return pageString
}

// the create-or-update functionality for confluence pages
// #342-dierk42: added parameter 'keywords'
def pushToConfluence = { pageTitle, pageBody, String parentId, anchors, pageAnchors, keywords ->
    def api = new RESTClient(config.confluence.api)
    def headers = getHeaders()
    String realTitleLC = realTitle(pageTitle).toLowerCase()

    //this fixes the encoding
    api.encoderRegistry = new EncoderRegistry( charset: 'utf-8' )
    if (config.confluence.proxy) {
        api.setProxy(config.confluence.proxy.host, config.confluence.proxy.port, config.confluence.proxy.schema ?: 'http')
    }
    //try to get an existing page
    localPage = parseBody(pageBody, anchors, pageAnchors)
    def localHash = MD5(localPage)
    if(config.confluence.disableToC){
        def prefix = (config.confluence.extraPageContent?:'')
        localPage  = prefix+localPage
        localHash = MD5(localPage)
        localPage += '<p style="display:none">hash: #'+localHash+'#</p>'
    }else{
        def default_toc = '<p><ac:structured-macro ac:name="toc"/></p>'
        def prefix = (config.confluence.tableOfContents?:default_toc)+(config.confluence.extraPageContent?:'')
        localPage  = prefix+localPage
        def default_children = '<p><ac:structured-macro ac:name="children"><ac:parameter ac:name="sort">creation</ac:parameter></ac:structured-macro></p>'
        localPage += (config.confluence.tableOfChildren?:default_children)
        localHash = MD5(localPage)
        localPage += '<p style="display:none">hash: #'+localHash+'#</p>'
    }


    def request = [
            type : 'page',
            title: realTitle(pageTitle),
            space: [
                    key: confluenceSpaceKey
            ],
            body : [
                    storage: [
                            value         : localPage,
                            representation: 'storage'
                    ]
            ]
    ]
    if (parentId) {
        request.ancestors = [
                [ type: 'page', id: parentId]
        ]
    }

    def pages = retrieveAllPages(api, headers, config.confluence.spaceKey)

    // println "Suche nach vorhandener Seite: " + pageTitle
    Map existingPage = pages[realTitleLC]
    def page

    if (existingPage) {
        if (hasRequestedParent(existingPage, parentId)) {
            page = retrieveFullPage(api, headers, existingPage.id)
        } else {
            page = null
        }
    } else {
        page = null
    }
    // println "Gefunden: " + page.id + " Titel: " + page.title

    if (page) {
        println "found existing page: " + page.id +" version "+page.version.number

        //extract hash from remote page to see if it is different from local one

        def remotePage = page.body.storage.value.toString().trim()

        def remoteHash = remotePage =~ /(?ms)hash: #([^#]+)#/
        remoteHash = remoteHash.size()==0?"":remoteHash[0][1]

        // println "remoteHash: " + remoteHash
        // println "localHash:  " + localHash

        if (remoteHash == localHash) {
            println "page hasn't changed!"
            deferredUpload.each {
                uploadAttachment(page?.id, it[1], it[2], it[3])
            }
            deferredUpload = []
            // #324-dierk42: Add keywords as labels to page.
            if (keywords) {
                addLabels(page.id, keywords)
            }
            return page.id
        } else {
            trythis {
                // update page
                // https://developer.atlassian.com/display/CONFDEV/Confluence+REST+API+Examples#ConfluenceRESTAPIExamples-Updatingapage
                request.id      = page.id
                request.version = [number: (page.version.number as Integer) + 1]
                def res = api.put(contentType: ContentType.JSON,
                                  requestContentType : ContentType.JSON,
                                  path: 'content/' + page.id, body: request, headers: headers)
            }
            println "> updated page "+page.id
            deferredUpload.each {
                uploadAttachment(page.id, it[1], it[2], it[3])
            }
            deferredUpload = []
            // #324-dierk42: Add keywords as labels to page.
            if (keywords) {
                addLabels(page.id, keywords)
            }
            return page.id
        }
    } else {
        //#352-LuisMuniz if the existing page's parent does not match the requested parentId, fail
        if (existingPage && !hasRequestedParent(existingPage, parentId)) {
            throw new IllegalArgumentException("Cannot create page, page with the same "
                    + "title=${existingPage.title} "
                    + "with id=${existingPage.id} already exists in the space. "
                    + "A Confluence page title must be unique within a space, consider specifying a 'confluencePagePrefix' in ConfluenceConfig.groovy")
        }

        //create a page
        trythis {
            page = api.post(contentType: ContentType.JSON,
                            requestContentType: ContentType.JSON,
                            path: 'content', body: request, headers: headers)
        }
        println "> created page "+page?.data?.id
        deferredUpload.each {
            uploadAttachment(page?.data?.id, it[1], it[2], it[3])
        }
        deferredUpload = []
        // #324-dierk42: Add keywords as labels to page.
        if (keywords) {
            addLabels(page?.data?.id, keywords)
        }
        return page?.data?.id
    }
}

def parseAnchors = { page ->
    def anchors = [:]
    page.body.select('[id]').each { anchor ->
        def name = anchor.attr('id')
        anchors[name] = page.title
        anchor.before("<ac:structured-macro ac:name=\"anchor\"><ac:parameter ac:name=\"\">${name}</ac:parameter></ac:structured-macro>")
    }
    anchors
}

def pushPages
pushPages = { pages, anchors, pageAnchors, labels ->
    pages.each { page ->
        println page.title
        def id = pushToConfluence page.title, page.body, page.parent, anchors, pageAnchors, labels
        page.children*.parent = id
        // println "Push children von id " + id
        pushPages page.children, anchors, pageAnchors, labels
        // println "Ende Push children von id " + id
    }
}

def recordPageAnchor = { head ->
    def a = [:]
    if (head.attr('id')) {
        a[head.attr('id')] = head.text()
    }
    a
}

def promoteHeaders = { tree, start, offset ->
    (start..7).each { i ->
        tree.select("h${i}").tagName("h${i-offset}").before('<br />')
    }
}

def retrievePageIdByName = { String name ->
    def api = new RESTClient(config.confluence.api)
    def headers = getHeaders()

        trythis {
            def request = [
                'title'    : name,
                'spaceKey' : confluenceSpaceKey
            ]
            api.get(
                [
                        'headers': headers,
                        'path'   : "${baseApiPath}content",
                        'query'  : request,
                ]
            ).data.results?.getAt(0)?.id
        } ?: null
}

def getHeaders(){
    println 'Start getting headers'
    def headers
    if(config.confluence.bearerToken){
        headers = [
                'Authorization': 'Bearer ' + config.confluence.bearerToken,
                'X-Atlassian-Token':'no-check'
        ]
         println 'Start using bearer auth'
    } else {
        headers = [
                'Authorization': 'Basic ' + config.confluence.credentials,
                'X-Atlassian-Token':'no-check'
        ]
        //Add api key and value to REST API request header if configured - required for authentification.
        if (config.confluence.apikey){
            headers.keyid = config.confluence.apikey
        }
    }
    return headers
}

if(config.confluence.inputHtmlFolder) {
    htmlFolder = "${docDir}/${config.confluence.inputHtmlFolder}"
    println "Starting processing files in folder: " + config.confluence.inputHtmlFolder
    def dir = new File(htmlFolder)

    dir.eachFileRecurse (FILES) { fileName ->
        if (fileName.isFile()){
            def map = [file: config.confluence.inputHtmlFolder+fileName.getName()]
            config.confluence.input.add(map)
         }
    }
}

config.confluence.input.each { input ->
    if(input.file) {
        input.file = "${docDir}/${input.file}"

        println "publish ${input.file}"

        if (input.file ==~ /.*[.](ad|adoc|asciidoc)$/) {
            println "HINT:"
            println "please first convert ${input.file} to html by executing generateHTML"
            println "the generated file will be found in build/html5/. and has to be referenced instead of the .adoc file"
            throw new RuntimeException("config problem")
        }
    //  assignend, but never used in pushToConfluence(...) (fixed here)
        confluenceSpaceKey = input.spaceKey ?: config.confluence.spaceKey
        confluenceCreateSubpages = (input.createSubpages != null) ? input.createSubpages : config.confluence.createSubpages
        confluenceAllInOnePage = (input.allInOnePage != null) ? input.allInOnePage : config.confluence.allInOnePage
        if (confluenceAllInOnePage && confluenceCreateSubpages) {
            println "ERROR:"
            println "Conflicting config: One one of confluenceAllInOnePage or confluenceCreateSubpages can be true."
            throw new RuntimeException("config problem")
        }
    //  hard to read in case of using :sectnums: -> so we add a suffix
        confluencePagePrefix = input.pagePrefix ?: config.confluence.pagePrefix
    //  added
        confluencePageSuffix = input.pageSuffix ?: config.confluence.pageSuffix
        confluencePreambleTitle = input.preambleTitle ?: config.confluence.preambleTitle

        def html = input.file ? new File(input.file).getText('utf-8') : new URL(input.url).getText()
        baseUrl = input.file ? new File(input.file) : new URL(input.url)
        Document dom = Jsoup.parse(html, 'utf-8', Parser.xmlParser())
        dom.outputSettings().prettyPrint(false);//makes html() preserve linebreaks and spacing
        dom.outputSettings().escapeMode(org.jsoup.nodes.Entities.EscapeMode.xhtml); //This will ensure xhtml validity regarding entities
        dom.outputSettings().charset("UTF-8"); //does no harm :-)

        // if ancestorName is defined try to find machingAncestorId in confluence
        def retrievedAncestorId
        if (input.ancestorName) {
            // Retrieve a page id by name
            retrievedAncestorId = retrievePageIdByName(input.ancestorName)
            println("Retrieved pageId for given ancestorName '${input.ancestorName}' is ${retrievedAncestorId}")
        }

        // if input does not contain an ancestorName, check if there is ancestorId, otherwise check if there is a global one
        def parentId = retrievedAncestorId ?: input.ancestorId ?: config.confluence.ancestorId

        // if parentId is still not set, create a new parent page (parentId = null)
        parentId = parentId ?: null
        //println("ancestorName: '${input.ancestorName}', ancestorId: ${input.ancestorId} ---> final parentId: ${parentId}")

        def anchors = [:]
        def pageAnchors = [:]
        def sections = pages = []
        // #342-dierk42: get the keywords from the meta tags
        def keywords = []
        dom.select('meta[name=keywords]').each { kw ->
            kws = kw.attr('content').split(',')
            kws.each { skw ->
                keywords << skw.trim()
            }
            println "Keywords:" + keywords
        }
        if (confluenceAllInOnePage) {
            dom.select('div#content').each { pageBody ->
                pageBody.select('div.sect2').unwrap()
                def page = [title   : confluencePreambleTitle ?: "arc42",
                            body    : pageBody,
                            children: [],
                            parent  : parentId]
                pages << page
                sections = page.children
                parentId = null
                anchors.putAll(parseAnchors(page))
            }
        } else {
            // let's try to select the "first page" and push it to confluence
            dom.select('div#preamble div.sectionbody').each { pageBody ->
                pageBody.select('div.sect2').unwrap()
                def preamble = [
                    title: confluencePreambleTitle ?: "arc42",
                    body: pageBody,
                    children: [],
                    parent: parentId
                ]
                pages << preamble
                sections = preamble.children
                parentId = null
                anchors.putAll(parseAnchors(preamble))
            }
            // <div class="sect1"> are the main headings
            // let's extract these
            dom.select('div.sect1').each { sect1 ->
                Elements pageBody = sect1.select('div.sectionbody')
                def currentPage = [
                    title: sect1.select('h2').text(),
                    body: pageBody,
                    children: [],
                    parent: parentId
                ]
                pageAnchors.putAll(recordPageAnchor(sect1.select('h2')))

                if (confluenceCreateSubpages) {
                    pageBody.select('div.sect2').each { sect2 ->
                        def title = sect2.select('h3').text()
                        pageAnchors.putAll(recordPageAnchor(sect2.select('h3')))
                        sect2.select('h3').remove()
                        def body = Jsoup.parse(sect2.toString(),'utf-8', Parser.xmlParser())
                        body.outputSettings(new Document.OutputSettings().prettyPrint(false))
                        def subPage = [
                            title: title,
                            body: body
                        ]
                        currentPage.children << subPage
                        promoteHeaders sect2, 4, 3
                        anchors.putAll(parseAnchors(subPage))
                    }
                    pageBody.select('div.sect2').remove()
                } else {
                    pageBody.select('div.sect2').unwrap()
                    promoteHeaders sect1, 3, 2
                }
                sections << currentPage
                anchors.putAll(parseAnchors(currentPage))
            }
        }

        pushPages pages, anchors, pageAnchors, keywords
        if (parentId) {
            println "published to ${config.confluence.api - "rest/api/"}spaces/${confluenceSpaceKey}/pages/${parentId}"
        } else {
            println "published to ${config.confluence.api - "rest/api/"}spaces/${confluenceSpaceKey}"
        }
    }
}
""

3.12. convertToDocx

Jody Winter Jody Winter Jeremie Bresson Ralf D. Müller Heiko Stehli ghp-dev Lars Francke

1 minute to read
At a Glance
convertToDocx
Before You Begin

Before using this task:

  • Install pandoc.

  • Ensure that 'docbook' and 'docx' are added to the inputFiles formats in Config.groovy.

  • As an optional step, specify a reference doc file with custom stylesheets (see task createReferenceDoc).

Further Reading and Resources

Read the Render AsciiDoc to docx (MS Word) blog post.

Source
pandoc.gradle
task convertToDocx (
        group: 'docToolchain',
        description: 'converts file to .docx via pandoc. Needs pandoc installed.',
        type: Exec
) {
    // All files with option `docx` in config.groovy is converted to docbook and then to docx.
    def sourceFilesDocx = sourceFiles.findAll { 'docx' in it.formats }
    sourceFilesDocx.each {
        def sourceFile = it.file.replace('.adoc', '.xml')
        def targetFile = sourceFile.replace('.xml', '.docx')

        workingDir "$targetDir/docbook"
        executable = "pandoc"

        if(referenceDocFile?.trim()) {
            args = ["-r","docbook",
                    "-t","docx",
                    "-o","../docx/$targetFile",
                    "--reference-doc=${docDir}/${referenceDocFile}",
                    sourceFile]
        } else {
            args = ["-r","docbook",
                    "-t","docx",
                    "-o","../docx/$targetFile",
                    sourceFile]
        }
    }
    doFirst {
        new File("$targetDir/docx/").mkdirs()
    }
}

3.13. createReferenceDoc

Before You Begin

Install pandoc.

Jody Winter Jeremie Bresson Ralf D. Müller Heiko Stehli

1 minute to read
About This Task

This task creates a reference docx file used by pandoc during docbook-to-docx conversion. Use task convertToDocx to edit this file so it uses your preferred styles.

The contents of the reference docx are ignored, but its stylesheets and document properties (including margins, page size, header and footer) are used in the new docx. For more information, see Pandoc User’s Guide: Options affecting specific writers (--reference-doc) And if you have problems with changing the default table style: see https://github.com/jgm/pandoc/issues/3275.

Config.groovy Notes

The 'referenceDocFile' property must be set to your custom reference file in Config.groovy:

inputPath = '.'

// use a style reference file in the input path for conversion from docbook to docx
referenceDocFile = "${inputPath}/my-ref-file.docx"
Source
pandoc.gradle
task createReferenceDoc (
        group: 'docToolchain helper',
        description: 'creates a docx file to be used as a format style reference in task convertToDocx. Needs pandoc installed.',
        type: Exec
) {
    workingDir "$docDir"
    executable = "pandoc"
    args = ["-o", "${docDir}/${referenceDocFile}",
            "--print-default-data-file",
            "reference.docx"]
    
    doFirst {
        if(!(referenceDocFile?.trim())) {
            throw new GradleException("Option `referenceDocFile` is not defined in config.groovy or has an empty value.")
        }
    }   
}

3.14. convertToEpub

Jody Winter Jody Winter Jeremie Bresson Ralf D. Müller Dr. Stefan Pfeiffer Lars Francke

1 minute to read
At a Glance
convertToEpub
Dependency
About This Task

This task uses pandoc to convert the DocBook output from AsciiDoctor to ePub. This publishes the output as an eBook which can be read using any eBook reader. The resulting file can be found in build/docs/epub.

Further Reading and Resources
Source
pandoc.gradle
task convertToEpub (
        group: 'docToolchain',
        description: 'converts file to .epub via pandoc. Needs pandoc installed.',
        type: Exec
) {
    // All files with option `epub` in config.groovy is converted to docbook and then to epub.
    def sourceFilesEpub = sourceFiles.findAll { 'epub' in it.formats }
    sourceFilesEpub.each {
        def sourceFile = it.file.replace('.adoc', '.xml')
        def targetFile = sourceFile.replace('.xml', '.epub')

        workingDir "$targetDir/docbook"
        executable = "pandoc"
        args = ['-r','docbook',
                '-t','epub',
                '-o',"../epub/$targetFile",
                sourceFile]
    }
    doFirst {
        new File("$targetDir/epub/").mkdirs()
    }
}

3.15. exportEA

Jody Winter Schrotti Jody Winter Ralf D. Müller Peter Stange Peter Stange Alexander Schwartz ghp-dev Lars Francke

4 minutes to read
At a Glance
exportEA
About This Task

By default, no special configuration is necessary. However, several optional parameter configurations are available to support a project and packages to be used for export. These parameters can be used independently from one another. A sample of how to edit your projects' Config.groovy is provided in the 'Config.groovy' of the docToolchain project itself.

Important

Currently this feature is WINDOWS-only. See this related issue.

The Optional Parameter Configurations
connection

Either set the connection to a certain project, or comment it out to use all project files inside the src folder or its child folder.

packageFilter

Add one or multiple packageGUIDs to be used for export. All packages are analysed, if no packageFilter is set.

exportPath

Relative path to base 'docDir' to which the diagrams and notes are to be exported. Default: "src/docs". Example: docDir = 'D:\work\mydoc\' ; exportPath = 'src/pdocs' ; Images will be exported to 'D:\work\mydoc\src\pdocs\images\ea', Notes will be exported to 'D:\work\mydoc\src\pdocs\ea',

searchPath

Relative path to base 'docDir', in which Enterprise Architect project files are searched Default: "src/docs". Example: docDir = 'D:\work\mydoc\' ; exportPath = 'src/projects' ; Lookup for eap and eapx files starts in 'D:\work\mydoc\src\projects' and goes down the folder structure. Note: In case parameter 'connection' is already defined, the searchPath value is also used. exportEA starts opening the database parameter 'connection' first then looks for further project files either in the searchPath (if set) or in the docDir folder of the project.

glossaryAsciiDocFormat

Whether or not the EA project glossary is exported depends on this parameter. If not set or an empty string, no glossary is exported. The glossaryAsciiDocFormat string is used to format each glossary entry in a certain AsciiDoc format.

The following placeholders are defined for the format string: ID, TERM, MEANING, TYPE. One or more can be used by the output format. For example:

A valid output format is to include the glossary as a flat list. The file can be included where needed in the documentation.

glossaryAsciiDocFormat = "TERM:: MEANING"

Other format strings can be used to include it as a table row. The glossary terms are sorted in alphabetical order.

glossaryTypes

This parameter is used in case a glossaryAsciiDocFormat is defined, otherwise it is not evaluated. It’s used to filter for certain types. If the glossaryTypes list is empty, all entries will be used. For example:

glossaryTypes = ["Business", "Technical"]

diagramAttributes

If set, the string is used to create and store diagram attributes to be included in the document alongside a diagram. These placeholders are defined and populated with the diagram attributes, if used in the diagramAttributes string:

%DIAGRAM_AUTHOR%,
%DIAGRAM_CREATED%,
%DIAGRAM_GUID%,
%DIAGRAM_MODIFIED%,
%DIAGRAM_NAME%,
%DIAGRAM_NOTES%,
%DIAGRAM_DIAGRAM_TYPE%,
%DIAGRAM_VERSION%,
%NEWLINE%

Example: diagramAttributes = "Last modification: %DIAGRAM_MODIFIED%%NEWLINE%Version: %DIAGRAM_VERSION%"

You can add the string %NEWLINE% where a line break will be added. The resulting text is stored next to the diagram image using the same path and file name, but a different file extension (.ad). This can be included in the document if required. If diagramAttributes is not set or an empty string, no file is written.

additionalOptions

This parameter is used to define the specific behavior of the export. Currently these options are supported:

KeepFirstDiagram

If diagrams are not uniquely named, the last diagram will be saved. If you want to prevent diagrams from being overwritten, add this parameter to additionalOptions.

Glossary export

By setting the glossaryAsciiDocFormat, the glossary terms stored in the EA project will be exported into a folder named 'glossary' below the configured exportPath. In case multiple EA projects are found for export, one glossary per project is exported - each named using the project’s GUID plus extension '.ad'.

Each individual file will be filtered (see glossaryTypes) and sorted in alphabetical order. In addition, a global glossary is created by using all single glossary files. This global file is named 'glossary.ad' and is also placed in the glossary folder. The global glossary is also filtered and sorted. If there is only one EA project, only the global glossary is written.

Further Reading and Resources
Source
build.gradle
task exportEA(
        dependsOn: [streamingExecute],
        description: 'exports all diagrams and some texts from EA files',
        group: 'docToolchain'
) {
    doFirst {
    }
    doLast {
        logger.info("docToolchain > exportEA: " + docDir)
        logger.info("docToolchain > exportEA: " + mainConfigFile)
        def configFile = new File(docDir, mainConfigFile)
        def config = new ConfigSlurper().parse(configFile.text)
        def scriptParameterString = ""
        def exportPath = ""
        def searchPath = ""
        def glossaryPath = ""
        def readme = """This folder contains exported diagrams or notes from Enterprise Architect.

Please note that these are generated files but reside in the `src`-folder in order to be versioned.

This is to make sure that they can be used from environments other than windows.

# Warning!

**The contents of this folder	will be overwritten with each re-export!**

use `gradle exportEA` to re-export files
"""

        if (!config.exportEA.connection.isEmpty()) {
            logger.info("docToolchain > exportEA: found " + config.exportEA.connection)
            scriptParameterString = scriptParameterString + "-c \"${config.exportEA.connection}\""
        }
        if (!config.exportEA.packageFilter.isEmpty()) {
            def packageFilterToCreate = config.exportEA.packageFilter as List
            logger.info("docToolchain > exportEA: package filter list size: " + packageFilterToCreate.size())
            packageFilterToCreate.each { packageFilter ->
                scriptParameterString = scriptParameterString + " -p \"${packageFilter}\""
            }
        }
        if (!config.exportEA.exportPath.isEmpty()) {
            exportPath = new File(docDir, config.exportEA.exportPath).getAbsolutePath()
        } else {
            exportPath = new File(docDir, 'src/docs').getAbsolutePath()
        }
        if (!config.exportEA.searchPath.isEmpty()) {
            searchPath = new File(docDir, config.exportEA.searchPath).getAbsolutePath()
        }
        else if (!config.exportEA.absoluteSearchPath.isEmpty()) {
        searchPath = new File(config.exportEA.absoluteSearchPath).getAbsolutePath()
        }
        else {
            searchPath = new File(docDir, 'src').getAbsolutePath()
        }
        scriptParameterString = scriptParameterString + " -d \"$exportPath\""
        scriptParameterString = scriptParameterString + " -s \"$searchPath\""
        logger.info("docToolchain > exportEA: exportPath: " + exportPath)

        //remove old glossary files/folder if exist
        new File(exportPath, 'glossary').deleteDir()
        //set the glossary file path in case an output format is configured, other no glossary is written
        if (!config.exportEA.glossaryAsciiDocFormat.isEmpty()) {
            //create folder to store glossaries
            new File(exportPath, 'glossary/.').mkdirs()
            glossaryPath = new File(exportPath, 'glossary').getAbsolutePath()
            scriptParameterString = scriptParameterString + " -g \"$glossaryPath\""
        }
        //configure additional diagram attributes to be exported
        if (!config.exportEA.diagramAttributes.isEmpty()) {
            scriptParameterString = scriptParameterString + " -da \"$config.exportEA.diagramAttributes\""
        }
        //configure additional diagram attributes to be exported
        if (!config.exportEA.additionalOptions.isEmpty()) {
            scriptParameterString = scriptParameterString + " -ao \"$config.exportEA.additionalOptions\""
        }
        //make sure path for notes exists
        //and remove old notes
        new File(exportPath, 'ea').deleteDir()
        //also remove old diagrams
        new File(exportPath, 'images/ea').deleteDir()

        //create a readme to clarify things
        new File(exportPath, 'images/ea/.').mkdirs()
        new File(exportPath, 'images/ea/readme.ad').write(readme)
        new File(exportPath, 'ea/.').mkdirs()
        new File(exportPath, 'ea/readme.ad').write(readme)

        //execute through cscript in order to make sure that we get WScript.echo right
        logger.info("docToolchain > exportEA: parameters: " + scriptParameterString)
        "%SystemRoot%\\System32\\cscript.exe //nologo ${projectDir}/scripts/exportEAP.vbs ${scriptParameterString}".executeCmd()
        //the VB Script is only capable of writing iso-8859-1-Files.
        //we now have to convert them to UTF-8
        new File(exportPath, 'ea/.').eachFileRecurse { file ->
            if (file.isFile()) {
                println "exported notes " + file.canonicalPath
                file.write(file.getText('iso-8859-1'), 'utf-8')
            }
        }

        //sort, filter and reformat a glossary if an output format is configured
        if (!config.exportEA.glossaryAsciiDocFormat.isEmpty()) {
            def glossaryTypes

            if (!config.exportEA.glossaryTypes.isEmpty()) {
                glossaryTypes = config.exportEA.glossaryTypes as List
            }
            new GlossaryHandler().execute(glossaryPath, config.exportEA.glossaryAsciiDocFormat, glossaryTypes);
        }
    }
}
scripts/exportEAP.vbs
    ' based on the "Project Interface Example" which comes with EA
    ' http://stackoverflow.com/questions/1441479/automated-method-to-export-enterprise-architect-diagrams



    Dim EAapp 'As EA.App
    Dim Repository 'As EA.Repository
    Dim FS 'As Scripting.FileSystemObject

    Dim projectInterface 'As EA.Project

    Const   ForAppending = 8
    Const   ForWriting = 2

    ' Helper
    ' http://windowsitpro.com/windows/jsi-tip-10441-how-can-vbscript-create-multiple-folders-path-mkdir-command
    Function MakeDir (strPath)
      Dim strParentPath, objFSO
      Set objFSO = CreateObject("Scripting.FileSystemObject")
      On Error Resume Next
      strParentPath = objFSO.GetParentFolderName(strPath)

      If Not objFSO.FolderExists(strParentPath) Then MakeDir strParentPath
      If Not objFSO.FolderExists(strPath) Then objFSO.CreateFolder strPath
      On Error Goto 0
      MakeDir = objFSO.FolderExists(strPath)
    End Function

    ' Replaces certain characters with '_' to avoid unwanted file or folder names causing errors or structure failures.
    ' Regular expression can easily be extended with further characters to be replaced.
    Function NormalizeName(theName)
       dim re : Set re = new regexp
       re.Pattern = "[\\/\[\]\s]"
       re.Global = True
       NormalizeName = re.Replace(theName, "_")
    End Function

    Sub WriteNote(currentModel, currentElement, notes, prefix)
        If (Left(notes, 6) = "{adoc:") Then
            strFileName = Mid(notes,7,InStr(notes,"}")-7)
            strNotes = Right(notes,Len(notes)-InStr(notes,"}"))
            set objFSO = CreateObject("Scripting.FileSystemObject")
            If (currentModel.Name="Model") Then
              ' When we work with the default model, we don't need a sub directory
              path = objFSO.BuildPath(exportDestination,"ea/")
            Else
              path = objFSO.BuildPath(exportDestination,"ea/"&NormalizeName(currentModel.Name)&"/")
            End If
            MakeDir(path)

            post = ""
            If (prefix<>"") Then
                post = "_"
            End If
            MakeDir(path&prefix&post)

            set objFile = objFSO.OpenTextFile(path&prefix&post&strFileName&".ad",ForAppending, True)
            name = currentElement.Name
            name = Replace(name,vbCr,"")
            name = Replace(name,vbLf,"")

            if (Left(strNotes, 3) = vbCRLF&"|") Then
                ' content should be rendered as table - so don't interfere with it
                objFile.WriteLine(vbCRLF)
            else
                'let's add the name of the object
                objFile.WriteLine(vbCRLF&vbCRLF&"."&name)
            End If
            objFile.WriteLine(vbCRLF&strNotes)
            objFile.Close
            if (prefix<>"") Then
                ' write the same to a second file
                set objFile = objFSO.OpenTextFile(path&prefix&".ad",ForAppending, True)
                objFile.WriteLine(vbCRLF&vbCRLF&"."&name&vbCRLF&strNotes)
                objFile.Close
            End If
        End If
    End Sub

    Sub SyncJira(currentModel, currentDiagram)
        notes = currentDiagram.notes
        set currentPackage = Repository.GetPackageByID(currentDiagram.PackageID)
        updated = 0
        created = 0
        If (Left(notes, 6) = "{jira:") Then
            WScript.echo " >>>> Diagram jira tag found"
            strSearch = Mid(notes,7,InStr(notes,"}")-7)
            Set objShell = CreateObject("WScript.Shell")
            'objShell.CurrentDirectory = fso.GetFolder("./scripts")
            Set objExecObject = objShell.Exec ("cmd /K  groovy ./scripts/exportEAPJiraPrintHelper.groovy """ & strSearch &""" & exit")
            strReturn = ""
            x = 0
            y = 0
            Do While Not objExecObject.StdOut.AtEndOfStream
                output = objExecObject.StdOut.ReadLine()
                ' WScript.echo output
                jiraElement = Split(output,"|")
                name = jiraElement(0)&":"&vbCR&vbLF&jiraElement(4)
                On Error Resume Next
                Set requirement = currentPackage.Elements.GetByName(name)
                On Error Goto 0
                if (IsObject(requirement)) then
                    ' element already exists
                    requirement.notes = ""
                    requirement.notes = requirement.notes&"<a href='"&jiraElement(5)&"'>"&jiraElement(0)&"</a>"&vbCR&vbLF
                    requirement.notes = requirement.notes&"Priority: "&jiraElement(1)&vbCR&vbLF
                    requirement.notes = requirement.notes&"Created: "&jiraElement(2)&vbCR&vbLF
                    requirement.notes = requirement.notes&"Assignee: "&jiraElement(3)&vbCR&vbLF
                    requirement.Update()
                    updated = updated + 1
                else
                    Set requirement = currentPackage.Elements.AddNew(name,"Requirement")
                    requirement.notes = ""
                    requirement.notes = requirement.notes&"<a href='"&jiraElement(5)&"'>"&jiraElement(0)&"</a>"&vbCR&vbLF
                    requirement.notes = requirement.notes&"Priority: "&jiraElement(1)&vbCR&vbLF
                    requirement.notes = requirement.notes&"Created: "&jiraElement(2)&vbCR&vbLF
                    requirement.notes = requirement.notes&"Assignee: "&jiraElement(3)&vbCR&vbLF
                    requirement.Update()
                    currentPackage.Elements.Refresh()
                    Set dia_obj = currentDiagram.DiagramObjects.AddNew("l="&(10+x*200)&";t="&(10+y*50)&";b="&(10+y*50+44)&";r="&(10+x*200+180),"")
                    x = x + 1
                    if (x>3) then
                      x = 0
                      y = y + 1
                    end if
                    dia_obj.ElementID = requirement.ElementID
                    dia_obj.Update()
                    created = created + 1
                end if
            Loop
            Set objShell = Nothing
            WScript.echo "created "&created&" requirements"
            WScript.echo "updated "&updated&" requirements"
        End If
    End Sub

    ' This sub routine checks if the format string defined in diagramAttributes
    ' does contain any characters. It replaces the known placeholders:
    ' %DIAGRAM_AUTHOR%, %DIAGRAM_CREATED%, %DIAGRAM_GUID%, %DIAGRAM_MODIFIED%,
    ' %DIAGRAM_NAME%, %DIAGRAM_NOTES%, %DIAGRAM_DIAGRAM_TYPE%, %DIAGRAM_VERSION%
    ' with the attribute values read from the EA diagram object.
    ' None, one or multiple number of placeholders can be used to create a diagram attribute
    ' to be added to the document. The attribute string is stored as a file with the same
    ' path and name as the diagram image, but with suffix .ad. So, it can
    ' easily be included in an asciidoc file.
    Sub SaveDiagramAttribute(currentDiagram, path, diagramName)
        If Len(diagramAttributes) > 0 Then
            filledDiagAttr = diagramAttributes
            set objFSO = CreateObject("Scripting.FileSystemObject")
            filename = objFSO.BuildPath(path, diagramName & ".ad")
            set objFile = objFSO.OpenTextFile(filename, ForWriting, True)
            filledDiagAttr = Replace(filledDiagAttr, "%DIAGRAM_AUTHOR%",       currentDiagram.Author)
            filledDiagAttr = Replace(filledDiagAttr, "%DIAGRAM_CREATED%",      currentDiagram.CreatedDate)
            filledDiagAttr = Replace(filledDiagAttr, "%DIAGRAM_GUID%",         currentDiagram.DiagramGUID)
            filledDiagAttr = Replace(filledDiagAttr, "%DIAGRAM_MODIFIED%",     currentDiagram.ModifiedDate)
            filledDiagAttr = Replace(filledDiagAttr, "%DIAGRAM_NAME%",         currentDiagram.Name)
            filledDiagAttr = Replace(filledDiagAttr, "%DIAGRAM_NOTES%",        currentDiagram.Notes)
            filledDiagAttr = Replace(filledDiagAttr, "%DIAGRAM_DIAGRAM_TYPE%", currentDiagram.Type)
            filledDiagAttr = Replace(filledDiagAttr, "%DIAGRAM_VERSION%",      currentDiagram.Version)
            filledDiagAttr = Replace(filledDiagAttr, "%NEWLINE%",              vbCrLf)
            objFile.WriteLine(filledDiagAttr)
            objFile.Close
        End If
    End Sub
    Sub SaveDiagram(currentModel, currentDiagram)
        Dim exportDiagram ' As Boolean

        ' Open the diagram
        Repository.OpenDiagram(currentDiagram.DiagramID)

        ' Save and close the diagram
        set objFSO = CreateObject("Scripting.FileSystemObject")
        If (currentModel.Name="Model") Then
            ' When we work with the default model, we don't need a sub directory
            path = objFSO.BuildPath(exportDestination,"/images/ea/")
        Else
            path = objFSO.BuildPath(exportDestination,"/images/ea/" & NormalizeName(currentModel.Name) & "/")
        End If
        path = objFSO.GetAbsolutePathName(path)
        MakeDir(path)

        diagramName = currentDiagram.Name
        diagramName = Replace(diagramName,vbCr,"")
        diagramName = Replace(diagramName,vbLf,"")
        diagramName = NormalizeName(diagramName)
        filename = objFSO.BuildPath(path, diagramName & ".png")

        exportDiagram = True
        If objFSO.FileExists(filename) Then
            WScript.echo " --- " & filename & " already exists."
            If Len(additionalOptions) > 0 Then
                If InStr(additionalOptions, "KeepFirstDiagram") > 0 Then
                    WScript.echo " --- Skipping export -- parameter 'KeepFirstDiagram' set."
                Else
                    WScript.echo " --- Overwriting -- parameter 'KeepFirstDiagram' not set."
                    exportDiagram = False
                End If
            Else
                WScript.echo " --- Overwriting -- parameter 'KeepFirstDiagram' not set."
            End If
        End If
        If exportDiagram Then
          projectInterface.SaveDiagramImageToFile(filename)
          WScript.echo " extracted image to " & filename
          If Not IsEmpty(diagramAttributes) Then
              SaveDiagramAttribute currentDiagram, path, diagramName
          End If
        End If
        Repository.CloseDiagram(currentDiagram.DiagramID)

        ' Write the note of the diagram
        WriteNote currentModel, currentDiagram, currentDiagram.Notes, diagramName&"_notes"

        For Each diagramElement In currentDiagram.DiagramObjects
            Set currentElement = Repository.GetElementByID(diagramElement.ElementID)
            WriteNote currentModel, currentElement, currentElement.Notes, diagramName&"_notes"
        Next
        For Each diagramLink In currentDiagram.DiagramLinks
            set currentConnector = Repository.GetConnectorByID(diagramLink.ConnectorID)
            WriteNote currentModel, currentConnector, currentConnector.Notes, diagramName&"_links"
        Next
    End Sub
    '
    ' Recursively saves all diagrams under the provided package and its children
    '
    Sub DumpDiagrams(thePackage,currentModel)
        Set currentPackage = thePackage

        ' export element notes
        For Each currentElement In currentPackage.Elements
            WriteNote currentModel, currentElement, currentElement.Notes, ""
            ' export connector notes
            For Each currentConnector In currentElement.Connectors
                ' WScript.echo currentConnector.ConnectorGUID
                if (currentConnector.ClientID=currentElement.ElementID) Then
                    WriteNote currentModel, currentConnector, currentConnector.Notes, ""
                End If
            Next
            if (Not currentElement.CompositeDiagram Is Nothing) Then
                SyncJira currentModel, currentElement.CompositeDiagram
                SaveDiagram currentModel, currentElement.CompositeDiagram
            End If
            if (Not currentElement.Elements Is Nothing) Then
                DumpDiagrams currentElement,currentModel
            End If
        Next


        ' Iterate through all diagrams in the current package
        For Each currentDiagram In currentPackage.Diagrams
            SyncJira currentModel, currentDiagram
            SaveDiagram currentModel, currentDiagram
        Next

        ' Process child packages
        Dim childPackage 'as EA.Package
        ' otPackage = 5
        if (currentPackage.ObjectType = 5) Then
            For Each childPackage In currentPackage.Packages
                call DumpDiagrams(childPackage, currentModel)
            Next
        End If
    End Sub

    Function SearchEAProjects(path)
      For Each folder In path.SubFolders
        SearchEAProjects folder
      Next

      For Each file In path.Files
            If fso.GetExtensionName (file.Path) = "eap" OR fso.GetExtensionName (file.Path) = "eapx" Then
                WScript.echo "found "&file.path
                If (Left(file.name, 1) = "_") Then
                    WScript.echo "skipping, because it start with `_` (replication)"
                Else
                    OpenProject(file.Path)
                End If
            End If
      Next
    End Function

    'Gets the package object as referenced by its GUID from the Enterprise Architect project.
    'Looks for the model node, the package is a child of as it is required for the diagram export.
    'Calls the Sub routine DumpDiagrams for the model and package found.
    'An error is printed to console only if the packageGUID is not found in the project.
    Function DumpPackageDiagrams(EAapp, packageGUID)
        WScript.echo "DumpPackageDiagrams"
        WScript.echo packageGUID
        Dim package
        Set package = EAapp.Repository.GetPackageByGuid(packageGUID)
        If (package Is Nothing) Then
          WScript.echo "invalid package - as package is not part of the project"
        Else
          Dim currentModel
          Set currentModel = package
          while currentModel.IsModel = false
            Set currentModel = EAapp.Repository.GetPackageByID(currentModel.parentID)
          wend
          ' Iterate through all child packages and save out their diagrams
          ' save all diagrams of package itself
          call DumpDiagrams(package, currentModel)
        End If
    End Function

    Function FormatStringToJSONString(inputString)
        outputString = Replace(inputString, "\", "\\")
        outputString = Replace(outputString, """", "\""")
        outputString = Replace(outputString, vbCrLf, "\n")
        outputString = Replace(outputString, vbLf, "\n")
        outputString = Replace(outputString, vbCr, "\n")
        FormatStringToJSONString = outputString
    End Function

    'If a valid file path is set, the glossary terms are read from EA repository,
    'formatted in a JSON compatible format and written into file.
    'The file is read and reformatted by the exportEA gradle task afterwards.
    Function ExportGlossaryTermsAsJSONFile(EArepo)
        If (Len(glossaryFilePath) > 0) Then
            set objFSO = CreateObject("Scripting.FileSystemObject")
            GUID = Replace(EArepo.ProjectGUID,"{","")
            GUID = Replace(GUID,"}","")
            currentGlossaryFile = objFSO.BuildPath(glossaryFilePath,"/"&GUID&".ad")
            set objFile = objFSO.OpenTextFile(currentGlossaryFile,ForAppending, True)

            Set glossary = EArepo.Terms()
            objFile.WriteLine("[")
            dim counter
            counter = 0
            For Each term In glossary
                if (counter > 0) Then
                    objFile.Write(",")
                end if
                objFile.Write("{ ""term"" : """&FormatStringToJSONString(term.term)&""", ""meaning"" : """&FormatStringToJSONString(term.Meaning)&""",")
                objFile.WriteLine(" ""termID"" : """&FormatStringToJSONString(term.termID)&""", ""type"" : """&FormatStringToJSONString(term.type)&""" }")
                counter = counter + 1
            Next
            objFile.WriteLine("]")

            objFile.Close
        End If
    End Function

    Sub OpenProject(file)
      ' open Enterprise Architect
      Set EAapp = CreateObject("EA.App")
      WScript.echo "opening Enterprise Architect. This might take a moment..."
      ' load project
      EAapp.Repository.OpenFile(file)
      ' make Enterprise Architect to not appear on screen
      EAapp.Visible = False

      ' get repository object
      Set Repository = EAapp.Repository
      ' Show the script output window
      ' Repository.EnsureOutputVisible("Script")
      call ExportGlossaryTermsAsJSONFile(Repository)

      Set projectInterface = Repository.GetProjectInterface()

      Dim childPackage 'As EA.Package
      ' Iterate through all model nodes
      Dim currentModel 'As EA.Package
      If (InStrRev(file,"{") > 0) Then
        ' the filename references a GUID
        ' like {04C44F80-8DA1-4a6f-ECB8-982349872349}
        WScript.echo file
        GUID = Mid(file, InStrRev(file,"{")+0,38)
        WScript.echo GUID
        ' Iterate through all child packages and save out their diagrams
        call DumpPackageDiagrams(EAapp, GUID)
      Else
        If packageFilter.Count = 0 Then
          WScript.echo "done"
        ' Iterate through all model nodes
        For Each currentModel In Repository.Models
            ' Iterate through all child packages and save out their diagrams
            For Each childPackage In currentModel.Packages
                call DumpDiagrams(childPackage,currentModel)
            Next
        Next
        Else
          ' Iterate through all packages found in the package filter given by script parameter.
          For Each packageGUID In packageFilter
            call DumpPackageDiagrams(EAapp, packageGUID)
          Next
        End If
      End If
      EAapp.Repository.CloseFile()
      ' Since EA 15.2 the Enterprise Architect background process hangs without calling Exit explicitly
      EAapp.Repository.Exit()
    End Sub

  Private connectionString
  Private packageFilter
  Private exportDestination
  Private searchPath
  Private glossaryFilePath
  Private diagramAttributes
  Private additionalOptions

  exportDestination = "./src/docs"
  searchPath = "./src"
  Set packageFilter = CreateObject("System.Collections.ArrayList")
  Set objArguments = WScript.Arguments

  Dim argCount
  argCount = 0
  While objArguments.Count > argCount+1
    Select Case objArguments(argCount)
      Case "-c"
        connectionString = objArguments(argCount+1)
      Case "-p"
        packageFilter.Add objArguments(argCount+1)
      Case "-d"
        exportDestination = objArguments(argCount+1)
      Case "-s"
        searchPath = objArguments(argCount+1)
      Case "-g"
        glossaryFilePath = objArguments(argCount+1)
      Case "-da"
        diagramAttributes = objArguments(argCount+1)
      Case "-ao"
        additionalOptions = objArguments(argCount+1)
    End Select
    argCount = argCount + 2
  WEnd
  set fso = CreateObject("Scripting.fileSystemObject")
  WScript.echo "Image extractor"

  ' Check both types in parallel - 1st check Enterprise Architect database connection, 2nd look for local project files
  If Not IsEmpty(connectionString) Then
     WScript.echo "opening database connection now"
     OpenProject(connectionString)
  End If
  WScript.echo "looking for .eap(x) files in " & fso.GetAbsolutePathName(searchPath)
  ' Dim f As Scripting.Files
  SearchEAProjects fso.GetFolder(searchPath)

  WScript.echo "finished exporting images"

3.16. exportVisio

Jody Winter Jody Winter Ralf D. Müller Alexander Schwartz Lars Francke jakubjab

1 minute to read
At a Glance
exportVisio
About This Task

This task searches for Visio files in the /src/docs folder then exports all diagrams and element notes to /src/docs/images/visio and /src/docs/visio.

Images are stored as /images/visio/[filename]-[pagename].png. Notes are stored as /visio/[filename]-[pagename].adoc

You can specify a filename to export notes to by starting any comment with {adoc:[filename].adoc}. It will then be written to /visio/[filename].adoc.

Important Information About This Task
  • Currently, only Visio files stored directly in /src/docs are supported. All others will export to the wrong location.

  • Before running this task, close any open Visio instance.

Further Reading and Resources
Source
exportVisio.gradle
task exportVisio(
        dependsOn: [streamingExecute],
        description: 'exports all diagrams and notes from visio files',
        group: 'docToolchain'
) {
    doLast {
        //make sure path for notes exists
        //and remove old notes
        new File(docDir, 'src/docs/visio').deleteDir()
        //also remove old diagrams
        new File(docDir, 'src/docs/images/visio').deleteDir()
        //create a readme to clarify things
        def readme = """This folder contains exported diagrams and notes from visio files.
		
Please note that these are generated files but reside in the `src`-folder in order to be versioned.

This is to make sure that they can be used from environments other than windows.

# Warning!

**The contents of this folder	will be overwritten with each re-export!**

use `gradle exportVisio` to re-export files
"""
        new File(docDir, 'src/docs/images/visio/.').mkdirs()
        new File(docDir, 'src/docs/images/visio/readme.ad').write(readme)
        new File(docDir, 'src/docs/visio/.').mkdirs()
        new File(docDir, 'src/docs/visio/readme.ad').write(readme)
        def sourcePath = new File(docDir, 'src/docs/.').canonicalPath
        def scriptPath = new File(projectDir, 'scripts/VisioPageToPngConverter.ps1').canonicalPath
        "powershell ${scriptPath} -SourcePath ${sourcePath}".executeCmd()
    }
}
scripts/VisioPageToPngConverter.ps1
# Convert all pages in all visio files in the given directory to png files.
# A Visio windows might flash shortly.
# The converted png files are stored in the same directory
# The name of the png file is concatenated from the Visio file name and the page name.
# In addtion all the comments are stored in adoc files.
# If the Viso file is named "MyVisio.vsdx" and the page is called "FirstPage"
# the name of the png file will be "MyVisio-FirstPage.png" and the comment will
# be stored in "MyVisio-FirstPage.adoc".
# But for the name of the adoc files there is an alternative. It can be given in the first 
# line of the comment. If it is given in the comment it has to be given in curly brackes 
# with the prefix "adoc:", e.g. {adoc:MyCommentFile.adoc}
# Prerequisites: Viso and PowerShell has to be installed on the computer.
# Parameter: SourcePath where visio files can be found
# Example powershell VisoPageToPngConverter.ps1 -SourcePath c:\convertertest\

Param
(
    [Parameter(Mandatory=$true,ValueFromPipeline=$true,Position=0)]
    [Alias('p')][String]$SourcePath
)

Write-Output "starting to export visio"

If (!(Test-Path -Path $SourcePath))
{
    Write-Warning "The path ""$SourcePath"" does not exist or is not accessible, please input the correct path."
    Exit
}

# Extend the source path to get only Visio files of the given directory and not in subdircetories
If ($SourcePath.EndsWith("\"))
{
    $SourcePath  = "$SourcePath"
}
Else
{
    $SourcePath  = "$SourcePath\"
}

$VisioFiles = Get-ChildItem -Path "$SourcePath*" -Recurse -Include *.vsdx,*.vssx,*.vstx,*.vxdm,*.vssm,*.vstm,*.vsd,*.vdw,*.vss,*.vst

If(!($VisioFiles))
{
    Write-Warning "There are no Visio files in the path ""$SourcePath""."
    Exit
}

$VisioApp = New-Object -ComObject Visio.Application
$VisioApp.Visible = $false
    
# Extract the png from all the files in the folder
Foreach($File in $VisioFiles)
{
    $FilePath = $File.FullName
    Write-Output "found ""$FilePath"" ."
    $FileDirectory = $File.DirectoryName   # Get the folder containing the Visio file. Will be used to store the png and adoc files
    $FileBaseName = $File.BaseName -replace '[ :/\\*?|<>]','-'   # Get the filename to be used as part of the name of the png and adoc files
    
    Try
    {
        $Document = $VisioApp.Documents.Open($FilePath)
        $Pages = $VisioApp.ActiveDocument.Pages
        Foreach($Page in $Pages)
        {
            # Create valid filenames for the png and adoc files
            $PngFileName = $Page.Name -replace '[ :/\\*?|<>]','-'
            $PngFileName = "$FileBaseName-$PngFileName.png"
            $AdocFileName = $PngFileName.Replace(".png", ".adoc")

            #TODO: this needs better logic
            Write-Output("$SourcePath\images\visio\$PngFileName")
            $Page.Export("$SourcePath\images\visio\$PngFileName")
            
            $AllPageComments = ""
            ForEach($PageComment in $Page.Comments)
            {
                # Extract adoc filename from comment text if the syntax is valid
                # Remove the filename from the text and save the comment in a file with a valid name
                $EofStringIndex = $PageComment.Text.IndexOf(".adoc}")
                if ($PageComment.Text.StartsWith("{adoc") -And ($EofStringIndex -gt 6))
                {
                    $AdocFileName = $PageComment.Text.Substring(6, $EofStringIndex -1)
                    $AllPageComments += $PageComment.Text.Substring($EofStringIndex + 6)
                }
                else
                {
                    $AllPageComments += $PageComment.Text+"`n"
                }
            }
            If ($AllPageComments)
            {

                $AdocFileName = $AdocFileName -replace '[:/\\*?|<>]','-'
                #TODO: this needs better logic
                $stream = [System.IO.StreamWriter] "$SourcePath\visio\$AdocFileName"
                $stream.WriteLine($AllPageComments)
                $stream.close()
            }                    
        }
        $Document.Close()
    }
    Catch
    {
        if ($Document)
        {
            $Document.Close()
        }
        Write-Warning "One or more visio page(s) in file ""$FilePath"" have been lost in this converting."
        Write-Warning "Error was: $_"
    }
}
$VisioApp.Quit()

3.17. exportDrawIo

Jody Winter Ralf D. Müller Ralf D. Müller Alexander Schwartz Jared Morgan

2 minutes to read
About This Task

There is no exportDrawIo task available in docToolchain because such a task is not required. You can continue to use diagrams.net (formerly known as draw.io) to edit your diagrams simply by making a change to your diagram-authoring workflow.

About diagrams.net

diagrams.net offers free and open source desktop editors for all major operating system platforms. Visit https://www.diagrams.net/integrations to find a desktop editor application compatible with your operating system. When you use the desktop version, just create your diagram with the .png (or even better, .dio.png) extension and diagrams.net will always save your diagram as a PNG with the source as metadata.

They have also launched a free plugin for VS Code and IntelliJ so you can edit your diagrams offline!

How to Change Your Workflow to Use diagrams.net

Export your diagrams.net/draw.io diagrams as a PNG with the source embedded in the file metadata. This allows you to embed your diagrams into AsciiDoc source as you normally would (using the image:: macro) with the added advantage of storing the diagram source with the image itself.

How to Convert a Confluence Page to AsciiDoc

If you are converting a Confluence page with embedded draw.io diagrams to AsciiDoc, use this export workflow to continue using diagrams.net:

  1. Export an editable PNG diagram from Confluence.

  2. Load the diagram you want to export from Confluence.

  3. Click File  Export as  PNG…​.

  4. In the Image modal, make sure that Include a copy of my diagram is selected.

  5. Click Export to save the PNG file with the pattern [file].dio.png.

  6. Commit the exported PNG file to source control.

Your diagram can now be managed in source control, added to your documentation source and edited using a diagrams.net desktop version.

Specifying .dio (short for "drawio") in the name will help you identify PNG files containing an embedded XML diagram source.

3.18. exportChangeLog

Jody Winter Jody Winter Ralf D. Müller Lars Francke Jakub J Jablonski

2 minutes to read
At a Glance
exportChangeLog
About This Task

As the name suggests, this task exports the changelog to be referenced from within your documentation, if needed. The changelog is written to build/docs/changelog.adoc.

This task can be configured to use a different source control system or a different directory. To configure this task, copy template_config/scripts/ChangelogConfig.groovy to your directory and modify to suit your needs. Then use -PchangelogConfigFile=<your config file> to add the path to your configuration file to the task. See the description inside the template for more details.

By default, the source is the Git changelog for the path src/docs and only contains the commit messages for changes made to the documentation. All changes to the build or other sources in the repository will not show up. By default, the changelog contains changes made to date, author and commit message already formatted as AsciiDoc table content:

| 09.04.2017
| Ralf D. Mueller
| fix #24 template updated to V7.0

| 08.04.2017
| Ralf D. Mueller
| fixed typo

You simply include it like this:

.Changes
[options="header",cols="1,2,6"]
|====
| Date
| Author
| Comment

include::../../build/docs/changelog.adoc[]

|====

By excluding the table definition, you can easily translate the table headings through different text snippets.

In a future docToolchain release, you will have the ability to include only certain commit messages from the changelog and exclude others (starting with # or //?). This feature is not available just yet.
Further Reading and Resources
Source
exportChangelog.gradle
task exportChangeLog(
        description: 'exports the change log from a git subpath',
        group: 'docToolchain'
) {
    doFirst {
        new File(targetDir).mkdirs()
    }
    doLast {
        logger.info("docToolchain> docDir: "+docDir)
        logger.info("docToolchain> mainConfigFile: "+mainConfigFile)
        def config = new ConfigSlurper().parse(new File(docDir, mainConfigFile).text)

        def cmd = "${config.changelog.cmd} ."
        def changes = cmd.execute(null, new File(docDir, config.changelog.dir)).text
        def changelog = new File(targetDir, 'changelog.adoc')
        logger.info "> changelog exported ${changelog.canonicalPath}"
        changelog.write(changes)
    }
}

3.19. exportContributors

Schrotti Ralf D. Müller Jody Winter Ralf D. Müller Alexander Schwartz

3 minutes to read
About This Task

This task crawls through all Asciidoctor source files and extracts a list of contributors, which is then rendered as AsciiDoc images of each contributor’s gravatar picture. The extracted list is stored in /home/runner/work/docToolchain/docToolchain/build/contributors/015_tasks/03_task_exportContributors.adoc so it can be easily included in your documents.

How to Use This Task

The best way to use this task is to create a feedback.adoc file similar to this:

feedback.adoc
ifndef::backend-pdf[] (1)

    image::https://img.shields.io/badge/improve-this%20doc-orange.svg[link={manualdir}{filename}, float=right] (2)
    image::https://img.shields.io/badge/create-an%20issue-blue.svg[link="https://github.com/docToolchain/documentation/issues/new?title=&body=%0A%0A%5BEnter%20feedback%20here%5D%0A%0A%0A---%0A%23page:{filename}", float=right] (3)

endif::[]

include::{targetDir}/contributors/{filename}[] (4)
Key:
1 Do not show this section when docs are rendered as PDF.
2 Create an Improve This Doc button which links to your GitHub sources.
3 Create a Create an Issue button which links to your issue tracker.
4 Include the list of contributors created by this task.

(The task automatically adds the estimated reading time to the list of contributors.)

About the Avatar-Icons

It seems not to be possible to extract a link to the github avatar icons from the log. So, the solution is to use Gravatar icons. For this to work, the contributors email address is hashed and an icon link is generated from that hash.

http://www.gravatar.com/avatar/cc5f3bf8b3cb91c985ed4fd046aa451d?d=identicon

This result at least in an icon which has a distinct color.

Contributors can setup their own image through Gravatar.com. For this to work, the git commits need to use an email address which can be verified by Gravatar.com. Unfortunately, this is not the case if a contributor decided to make his email address private in the email sections of her github account.

File Attributes

This task also exports some GitHub file attributes. The extracted attributes are stored in /home/runner/work/docToolchain/docToolchain/build/fileattribs/015_tasks/03_task_exportContributors.adoc.

:lastUpdated: 16.05.2019 06:22
:lastAuthorName: Ralf D. Müller
:lastAuthorEmail: ralf.d.mueller@gmail.com
:lastAuthorAvatar: http://www.gravatar.com/avatar/cc5f3bf8b3cb91c985ed4fd046aa451d?d=identicon[32,32,role='gravatar',alt='Ralf D. Müller',title='Ralf D. Müller']
:lastMessage: #310 started to document config options

You can import and use these attributes in the same way as you import the contributors list.

please make sure that you do not accidentally publish the email address if your contributors do not want it.

For example:

feedback.adoc
include::{targetDir}/fileattribs/{filename}[]

Last updated {lastUpdated} by {lastAuthorName}

3.20. exportJiraIssues

Jody Winter Jody Winter Ralf D. Müller Vladi Vladi Alexander Schwartz

2 minutes to read
At a Glance
exportJiraIssues
About This Task

This task exports all issues for a given query or queries from Jira as either an AsciiDoc table or an Excel file.

The configuration for this task can be found within Config.gradle (gradle.properties can be used as a fallback). Username/password is deprecated, so you need to use username/API-token instead.

An API-token can be created through https://id.atlassian.com/manage/api-tokens. We recommend that you keep username and API-token out of your GitHub repository, and instead pass them as environment variables to docToolchain.

Configuration

Jira configuration support list requests to Jira where results of each requests will be saved in a file with specifies filename. Flags saveAsciidoc & saveExcel allow you to easily configure the format in which results should be saved.

The old configuration was based on the single Jira query is deprecated (single 'jql' parameter). Support for it will be removed in the near future. Please migrate to the new configuration which allows multiple Jira queries.
Config.groovy
// Configuration for Jira related tasks
jira = [:]

jira.with {

    // endpoint of the JiraAPI (REST) to be used
    api = 'https://your-jira-instance'

    /*
    WARNING: It is strongly recommended to store credentials securely instead of commiting plain text values to your git repository!!!

    Tool expects credentials that belong to an account which has the right permissions to read the JIRA issues for a given project.
    Credentials can be used in a form of:
     - passed parameters when calling script (-PjiraUser=myUsername -PjiraPass=myPassword) which can be fetched as a secrets on CI/CD or
     - gradle variables set through gradle properties (uses the 'jiraUser' and 'jiraPass' keys)
    Often, Jira & Confluence credentials are the same, in which case it is recommended to pass CLI parameters for both entities as
    -Pusername=myUser -Ppassword=myPassword
    */

    // the key of the Jira project
    project = 'PROJECTKEY'

    // the format of the received date time values to parse
    dateTimeFormatParse = "yyyy-MM-dd'T'H:m:s.SSSz" // i.e. 2020-07-24'T'9:12:40.999 CEST

    // the format in which the date time should be saved to output
    dateTimeFormatOutput = "dd.MM.yyyy HH:mm:ss z" // i.e. 24.07.2020 09:02:40 CEST

    // the label to restrict search to
    label = 'label1'

    // Legacy settings for Jira query. This setting is deprecated & support for it will soon be completely removed. Please use JiraRequests settings
    jql = "project='%jiraProject%' AND labels='%jiraLabel%' ORDER BY priority DESC, duedate ASC"

    // Base filename in which Jira query results should be stored
    resultsFilename = 'JiraTicketsContent'

    saveAsciidoc = true // if true, asciidoc file will be created with *.adoc extension
    saveExcel = true // if true, Excel file will be created with *.xlsx extension

    // Output folder for this task inside main outputPath
    resultsFolder = 'JiraRequests'

    /*
    List of requests to Jira API:
    These are basically JQL expressions bundled with a filename in which results will be saved.
    User can configure custom fields IDs and name those for column header,
    i.e. customfield_10026:'Story Points' for Jira instance that has custom field with that name and will be saved in a coloumn named "Story Points"
    */
    requests = [
        new JiraRequest(
            filename:"File1_Done_issues",
            jql:"project='%jiraProject%' AND status='Done' ORDER BY duedate ASC",
            customfields: [customfield_10026:'Story Points']
        ),
        new JiraRequest(
            filename:'CurrentSprint',
            jql:"project='%jiraProject%' AND Sprint in openSprints() ORDER BY priority DESC, duedate ASC",
            customfields: [customfield_10026:'Story Points']
        ),
    ]
}

@groovy.transform.Immutable
class JiraRequest {
    String filename  //filename (without extension) of the file in which JQL results will be saved. Extension will be determined automatically for Asciidoc or Excel file
    String jql // Jira Query Language syntax
    Map<String,String> customfields // map of customFieldId:displayName values for Jira fields which don't have default names, i.e. customfield_10026:StoryPoints
}
Further Reading and Resources
Source
exportJiraIssues.gradle
task exportJiraIssues(
        description: 'exports all jira issues from a given search',
        group: 'docToolchain'
) {
    doLast {
        final String taskSubfolderName = config.jira.resultsFolder
        final File targetFolder = new File(targetDir + File.separator + taskSubfolderName)
        if (!targetFolder.exists()) targetFolder.mkdirs()
        logger.debug("Output folder for 'exportJiraIssues' task is: '${targetFolder}'")

        // map configuration from Config.groovy to existing variables for compatibility with naming of Jira settings in gradle.properties
        def jiraRoot = config.jira.api
        def jiraProject = config.jira.project
        def jiraLabel = config.jira.label
        def jiraResultsFilename = config.jira.resultsFilename
        def jiraDateTimeFormatParse = config.jira.dateTimeFormatParse
        def jiraDateTimeOutput = config.jira.dateTimeFormatOutput
        def defaultFields = 'priority,created,resolutiondate,summary,assignee,status'

        def jira = new groovyx.net.http.RESTClient(jiraRoot + '/rest/api/2/')
        jira.encoderRegistry = new groovyx.net.http.EncoderRegistry(charset: 'utf-8')
        def headers = [
                'Authorization': "Basic " + config.jira.credentials,
                'Content-Type' : 'application/json; charset=utf-8'
        ]

        def jiraRequests = config.jira.requests

        if (config.jira.jql) {
            logger.warn(">>>Found legacy Jira requests. Please migrate to the new Jira configuration ASAP. Old config with jql will be removed soon")
            writeAsciiDocFileForLegacyConfiguration(targetFolder, jira, headers, config.jira)
        }

        jiraRequests.each {rq ->
            logger.quiet("Request to Jira API for '${rq.filename}' with query: '${rq.jql}'")

            def allHeaders = "${defaultFields},${rq.customfields.values().join(",")}"
            def allFieldIds = "${defaultFields},${rq.customfields.keySet().join(",")}"
            logger.quiet("Preparing headers for default & custom fields: ${allHeaders}")
            logger.quiet("Preparing field IDs for default & custom fields: ${allFieldIds}")

            // Save AsciiDoc file
            if (config.jira.saveAsciidoc) {
                def extension = 'adoc'
                jiraResultsFilename = "${rq.filename}.${extension}"
                logger.info("Results will be saved in '${rq.filename}.${extension}' file")

                def jiraDataAsciidoc = new File(targetFolder, "${rq.filename}.${extension}")
                jiraDataAsciidoc.write(".${rq.filename}\n", 'utf-8')
                jiraDataAsciidoc.append("|=== \n")

                // AsciiDoc table headers (custom fields map needs values here)
                jiraDataAsciidoc.append("|Key ", 'utf-8')
                allHeaders.split(",").each {field ->
                    jiraDataAsciidoc.append("|${field.capitalize()} ", 'utf-8')
                }
                jiraDataAsciidoc.append("\n", 'utf-8')

                jira.get(path: 'search',
                    query: ['jql'       : rq.jql.replaceAll('%jiraProject%', jiraProject).replaceAll('%jiraLabel%', jiraLabel),
                            'maxResults': 1000,
                            fields: "${allFieldIds}"
                    ],
                    headers: headers
                ).data.issues.each { issue ->
                    //logger.quiet(">> Whole issue ${issue.key}:\n   ${issue.fields}")
                    jiraDataAsciidoc.append("| ${jiraRoot}/browse/${issue.key}[${issue.key}] ", 'utf-8')
                    jiraDataAsciidoc.append("| ${issue.fields.priority.name} ", 'utf-8')
                    jiraDataAsciidoc.append("| ${Date.parse(jiraDateTimeFormatParse, issue.fields.created).format(jiraDateTimeOutput)} ", 'utf-8')
                    jiraDataAsciidoc.append("| ${issue.fields.resolutiondate ? Date.parse(jiraDateTimeFormatParse, issue.fields.resolutiondate).format(jiraDateTimeOutput) : ''} ", 'utf-8')
                    jiraDataAsciidoc.append("| ${issue.fields.summary} ", 'utf-8')
                    jiraDataAsciidoc.append("| ${issue.fields.assignee ? issue.fields.assignee.displayName : 'not assigned'}", 'utf-8')
                    jiraDataAsciidoc.append("| ${issue.fields.status.name} ", 'utf-8')

                    rq.customfields.each { field ->
                        def foundCustom = issue.fields.find {it.key == field.key}
                        //logger.quiet("Examining issue '${issue.key}' for custom field '${field.key}' has found: '${foundCustom}'")
                        jiraDataAsciidoc.append("| ${foundCustom ? foundCustom.value : '-'}\n", 'utf-8')
                    }
                }
                jiraDataAsciidoc.append("|=== \n")
            } else {
                logger.quiet("Set saveAsciidoc=true in '${mainConfigFile}' to save results in AsciiDoc file")
            }

            // Save Excel file
            if (config.jira.saveExcel) {
                def extension = 'xlsx'
                jiraResultsFilename = "${rq.filename}.${extension}"
                logger.quiet(">> Results will be saved in '${rq.filename}.${extension}' file")

                //def jiraDataAsciidoc = new File(targetFolder, "${rq.filename}.${extension}")

                def jiraDataXls = new File(targetFolder, jiraResultsFilename)
                def jiraFos = new FileOutputStream(jiraDataXls)

                Workbook wb = new XSSFWorkbook();
                CreationHelper hyperlinkHelper = wb.getCreationHelper();
                def sheetName = "${rq.filename}"
                def ws = wb.createSheet(sheetName)

                String rgbS = "A7A7A7"
                byte[] rgbB = Hex.decodeHex(rgbS)
                XSSFColor color = new XSSFColor(rgbB, null) //IndexedColorMap has no usage until now. So it can be set null.
                XSSFCellStyle headerCellStyle = (XSSFCellStyle) wb.createCellStyle()
                headerCellStyle.setFillForegroundColor(color)
                headerCellStyle.setFillPattern(FillPatternType.SOLID_FOREGROUND)

                def titleRow = ws.createRow(0);
                int cellNumber = 0;
                titleRow.createCell(cellNumber).setCellValue("Key")
                allHeaders.split(",").each {field ->
                    titleRow.createCell(++cellNumber).setCellValue("${field.capitalize()}")
                }
                def lastRow = titleRow.getRowNum()
                titleRow.setRowStyle(headerCellStyle)

                jira.get(path: 'search',
                    query: ['jql'       : rq.jql.replaceAll('%jiraProject%', jiraProject).replaceAll('%jiraLabel%', jiraLabel),
                            'maxResults': 1000,
                            fields: "${allFieldIds}"
                    ],
                    headers: headers
                ).data.issues.each { issue ->
                    int cellPosition = 0
                    def row = ws.createRow(++lastRow)
                    Hyperlink link = hyperlinkHelper.createHyperlink(HyperlinkType.URL)
                    link.setAddress("${jiraRoot}/browse/${issue.key}")
                    Cell cellWithUrl = row.createCell(cellPosition)
                    cellWithUrl.setCellValue("${issue.key}")
                    cellWithUrl.setHyperlink(link)

                    row.createCell(++cellPosition).setCellValue("${issue.fields.priority.name}")
                    row.createCell(++cellPosition).setCellValue("${Date.parse(jiraDateTimeFormatParse, issue.fields.created).format(jiraDateTimeOutput)}")
                    row.createCell(++cellPosition).setCellValue("${issue.fields.resolutiondate ? Date.parse(jiraDateTimeFormatParse, issue.fields.resolutiondate).format(jiraDateTimeOutput) : ''}")
                    row.createCell(++cellPosition).setCellValue("${issue.fields.summary}")
                    row.createCell(++cellPosition).setCellValue("${issue.fields.assignee ? issue.fields.assignee.displayName : ''}")
                    row.createCell(++cellPosition).setCellValue("${issue.fields.status.name}")

                    // Custom fields
                    rq.customfields.each { field ->
                        def position = ++cellPosition
                        def foundCustom = issue.fields.find {it.key == field.key}
                        row.createCell(position).setCellValue("${foundCustom ? foundCustom.value : '-'}")
                    }
                }

                // set jira issue key column fits the content width

                for(int colNum = 0; colNum<allHeaders.size()+1;colNum++) {
                    ws.autoSizeColumn(colNum)
                }
                // Set summary column width slightly wider but fixed size, so it doesn't change with every summary update
                ws.setColumnWidth(4, 25*384)


                wb.write(jiraFos)
            } else {
                logger.quiet("Set saveExcel=true in '${mainConfigFile}' to save results in Excel file")
            }
        }
    }
}

// This method can be removed when support for legacy Jira configuration is gone
def writeAsciiDocFileForLegacyConfiguration(def targetFolder, def restClient, def headers, def jiraConfig) {
    def resultsFilename = "${jiraConfig.resultsFilename}_legacy.adoc"

    def openIssues = new File(targetFolder, "${resultsFilename}")
    openIssues.write(".Table {Title}\n", 'utf-8')
    openIssues.append("|=== \n")
    openIssues.append("|Key |Priority |Created | Assignee | Summary\n", 'utf-8')
        def legacyJql = jiraConfig.jql.replaceAll('%jiraProject%', config.jira.project).replaceAll('%jiraLabel%', config.jira.label)
        println ("Results for legacy query '${legacyJql}' will be saved in '${resultsFilename}' file")

        restClient.get(path: 'search',
                query: ['jql'       : legacyJql,
                        'maxResults': 1000,
                        'fields'    : 'created,resolutiondate,priority,summary,timeoriginalestimate, assignee'
                ],
                headers: headers
      ).data.issues.each { issue ->
        openIssues.append("| ${jiraRoot}/browse/${issue.key}[${issue.key}] ", 'utf-8')
        openIssues.append("| ${issue.fields.priority.name} ", 'utf-8')
        openIssues.append("| ${Date.parse(jiraConfig.dateTimeFormatParse, issue.fields.created).format(jiraConfig.dateTimeFormatOutput)} ", 'utf-8')
        openIssues.append("| ${issue.fields.assignee ? issue.fields.assignee.displayName : 'not assigned'}", 'utf-8')
        openIssues.append("| ${issue.fields.summary} ", 'utf-8')
    }
    openIssues.append("|=== \n")
}

3.21. exportJiraSprintChangelogIssues

Jody Winter Ralf D. Müller Vladi

1 minute to read
About This Task

This task exports a simplified (key and summary) list of Jira issues for a specific sprint defined in the task configuration. Only a few additional fields (such as assignee) can be switched using configuration flags.

Once you define the sprint, the relevant AsciiDoc and Excel files will be generated. If a sprint is not defined in the configuration, changelogs for all sprints that match the configuration will be saved in separate AsciiDoc files and in different tabs within an Excel file.

The task configuration can be found within Config.gradle. In addition to the configuration snippet below, it is important to configure the Jira API and credentials in the Jira section of the configuration inside the same file.

Configuration
Config.groovy
// Sprint changelog configuration generate changelog lists based on tickets in sprints of an Jira instance.
// This feature requires at least Jira API & credentials to be properly set in Jira section of this configuration
sprintChangelog = [:]
sprintChangelog.with {
    sprintState = 'closed' // it is possible to define multiple states, i.e. 'closed, active, future'
    ticketStatus = "Done, Closed" // it is possible to define multiple ticket statuses, i.e. "Done, Closed, 'in Progress'"

    showAssignee = false
    showTicketStatus = false
    showTicketType = true
    sprintBoardId = 12345  // Jira instance probably have multiple boards; here it can be defined which board should be used

    // Output folder for this task inside main outputPath
    resultsFolder = 'Sprints'

    // if sprintName is not defined or sprint with that name isn't found, release notes will be created on for all sprints that match sprint state configuration
    sprintName = 'PRJ Sprint 1' // if sprint with a given sprintName is found, release notes will be created just for that sprint
    allSprintsFilename = 'Sprints_Changelogs' // Extension will be automatically added.
}
Source
exportJiraSprintChangelog.gradle
task exportJiraSprintChangelog(
        description: 'exports all jira issues from Sprint for release notes',
        group: 'docToolchain'
) {
    doLast {
        // Pre defined ticket fields for Changelog based on Jira Sprints
        def defaultTicketFields = 'summary,status,assignee,issuetype'

        // retrieving sprints for a given board
        def sprints = { apiSprints, headers, boardId, sprintState ->
                apiSprints.get(path: "agile/latest/board/${boardId}/sprint",
                    query:[state: "${sprintState}"],
                    headers: headers
                ).data
        }

        // retrieving issues for given sprint
        def issues = { apiIssues, headers, boardId, sprintId, status ->
            apiIssues.get(path: "agile/latest/board/${boardId}/sprint/${sprintId}/issue",
                    query: ['jql'       : "status in (${status}) ORDER BY type DESC, status ASC",
                            'maxResults': 1000,
                            fields: defaultTicketFields
                    ],
                    headers: headers
            ).data
        }

        // preparing target folder for generated files
        final String taskSubfolderName = config.sprintChangelog.resultsFolder
        final File targetFolder = new File(targetDir + File.separator + taskSubfolderName)
        if (!targetFolder.exists()) targetFolder.mkdirs()
        logger.debug("Output folder for 'exportJiraSprintChangelog' task is: '${targetFolder}'")

        // Getting configuration
        def jiraRoot = config.jira.api
        def jiraProject = config.jira.project

        def sprintState = config.sprintChangelog.sprintState
        def ticketStatusForReleaseNotes = config.sprintChangelog.ticketStatus
        def sprintBoardId = config.sprintChangelog.sprintBoardId
        def showAssignee = config.sprintChangelog.showAssignee
        def showTicketStatus = config.sprintChangelog.showTicketStatus
        def showTicketType = config.sprintChangelog.showTicketType
        def sprintName = config.sprintChangelog.sprintName
        def allSprintsFilename = config.sprintChangelog.allSprintsFilename

        logger.info("\n==========================\nJira Release notes config\n==========================")
        logger.info("Spring Board ID: ${sprintBoardId}")
        logger.info("Show assignees: ${showAssignee}. Show ticket status: ${showTicketStatus}. Show ticket type: ${showTicketType}")
        logger.info("Filtering for sprints with configured state: '${sprintState}'")
        logger.info("Filtering for issues with configured statuses: ${ticketStatusForReleaseNotes}")
        logger.info("Attempt to generate release notes for sprint with a name: '${sprintName}'")
        logger.info("Filename used for all sprints: '${allSprintsFilename}'")

        def api = new groovyx.net.http.RESTClient(jiraRoot + '/rest/')
        api.encoderRegistry = new groovyx.net.http.EncoderRegistry(charset: 'utf-8')
        def headers = [
                'Authorization': "Basic " + config.jira.credentials,
                'Content-Type' : 'application/json; charset=utf-8'
        ]

        def allChangelogsFilename = "${allSprintsFilename}.xlsx"
        logger.quiet("Changelogs of all sprints will be saved in '${allChangelogsFilename}' file")

        def changelogsXls = new File(targetFolder, allChangelogsFilename)
        def changelogsXlsFos = new FileOutputStream(changelogsXls)
        Workbook wb = new XSSFWorkbook();
        CreationHelper hyperlinkHelper = wb.getCreationHelper();

        String rgbS = "A7A7A7"
        byte[] rgbB = Hex.decodeHex(rgbS) // get byte array from hex string
        XSSFColor color = new XSSFColor(rgbB, null) //IndexedColorMap has no usage until now. So it can be set null.
        XSSFCellStyle headerCellStyle = (XSSFCellStyle) wb.createCellStyle()
        headerCellStyle.setFillForegroundColor(color)
        headerCellStyle.setFillPattern(FillPatternType.SOLID_FOREGROUND)

        // prepare tickets according to configuration
        def columns = ['key'].plus(defaultTicketFields.split(',').collect())
        if (!showAssignee) { columns = columns.minus('assignee')}
        if (!showTicketStatus) { columns = columns.minus('status')}
        if (!showTicketType) { columns = columns.minus('issuetype')}
        logger.info("Release notes will contain following info: ${columns}")

        logger.info("\n=====================\n      Sprints\n=====================")
        //
        def allMatchedSprints = sprints(api, headers, sprintBoardId, sprintState).values
        def foundExactSprint = allMatchedSprints.any {it.name == sprintName}
        logger.info("All sprints that matched configuration: ${allMatchedSprints.size()}")

        def sprintsForChangelog = foundExactSprint ? allMatchedSprints.stream().filter() {it.name == sprintName} : allMatchedSprints
        logger.info("Found exact Sprint with name '${sprintName}': ${foundExactSprint}.")

        sprintsForChangelog.each { sprint ->
            logger.quiet("\nSprint: $sprint.name [id: $sprint.id] state <$sprint.state>")

            /* ================================================
                Create new worksheet inside existing excel file
               ================================================ */
            String safeSprintName = WorkbookUtil.createSafeSheetName("${sprint.name}")
            def ws = wb.createSheet(safeSprintName)

            // Add titles (typically key & summary, but assignee, ticket status, ticket type can be configured in Config.groovy too)
            def titleRow = ws.createRow(0);
            int cellNumber = 0;
            columns.each {columnTitle -> titleRow.createCell(cellNumber++).setCellValue("${columnTitle.capitalize()}")}

            def lastRow = titleRow.getRowNum()
            titleRow.setRowStyle(headerCellStyle)
            // set summary (at position 1) column wider than other columns
            ws.setColumnWidth(1, 35*256)

            /* =========================================
                      AsciiDoc file for each sprint
               ========================================= */
            def asciidocFilename = "${sprint.name.replaceAll(" ", "_")}.adoc"
            logger.info("Results will be saved in '${asciidocFilename}' file")

            def changeLogAdoc = new File(targetFolder, "${asciidocFilename}")
            changeLogAdoc.write(".Table ${sprint.name} Changelog\n", 'utf-8')
            changeLogAdoc.append("|=== \n")

            // AsciiDoc table columns
            columns.each {columnTitle -> changeLogAdoc.append("|${columnTitle} ", 'utf-8')}

            /* =========================================
                      Add tickets for the sprint
               ========================================= */
            issues(api, headers, sprintBoardId, sprint.id, ticketStatusForReleaseNotes).issues.each {issue ->
                def assignee = "${issue.fields.assignee ? issue.fields.assignee.displayName : 'unassigned'} "
                def message = showAssignee ? "by ${assignee} ": ""
                logger.quiet("Issue: [$issue.key] '$issue.fields.summary' ${message}<$issue.fields.status.name>")

                /* ===========================
                      Write ticket to Excel
                   =========================== */
                int cellPosition = 0
                def row = ws.createRow(++lastRow)

                Hyperlink link = hyperlinkHelper.createHyperlink(HyperlinkType.URL)
                link.setAddress("${jiraRoot}/browse/${issue.key}")
                Cell cellWithUrl = row.createCell(cellPosition)
                cellWithUrl.setCellValue("${issue.key}")
                cellWithUrl.setHyperlink(link)

                row.createCell(++cellPosition).setCellValue("${issue.fields.summary}")

                /* =============================
                      Write ticket to Asciidoc
                   ============================= */
                changeLogAdoc.append("\n", 'utf-8')
                changeLogAdoc.append("| ${jiraRoot}/browse/${issue.key}[${issue.key}] ", 'utf-8')
                changeLogAdoc.append("| ${issue.fields.summary} ", 'utf-8')

                /* === Write ticket status, assignee, ticket typee if configured to both Asciidoc & Excel files === */
                if (showTicketStatus) {
                    row.createCell(++cellPosition).setCellValue("${issue.fields.status.name}")
                    changeLogAdoc.append("| ${issue.fields.status.name} ", 'utf-8')
                }

                if (showAssignee) {
                    row.createCell(++cellPosition).setCellValue("${assignee}")
                    changeLogAdoc.append("| ${assignee}", 'utf-8')
                }

                if (showTicketType) {
                    row.createCell(++cellPosition).setCellValue("${issue.fields.issuetype.name}")
                    changeLogAdoc.append("| ${issue.fields.issuetype.name} ", 'utf-8')
                }
            }
            // Close the asciidoc table
            changeLogAdoc.append("\n|=== \n",'utf-8')

            // Set auto-width to KEY column
            ws.autoSizeColumn(0);
        }
        // Write to Excel file
        wb.write(changelogsXlsFos)
    }
}

3.22. exportPPT

Ralf D. Müller Jody Winter Jody Winter Ralf D. Müller ghp-dev

1 minute to read
At a Glance
exportPPT
About This Task

This task lets you export a series of PowerPoint slides to be used within your AsciiDoc documentation. It is currently a Windows-only task.

It exports the slides as .jpg files and the speaker notes as one .adoc file. The tag {slide} within the speaker notes will be replaced with the corresponding image reference. This will help you to get a stable result, even when you insert or delete slides.

Use the tagged regions (//tag::[) feature of asciidoctor] to include only certain slides or parts of your speaker notes.

Further Reading and Resources
Source
exportPPT.gradle
task exportPPT(
        dependsOn: [streamingExecute],
        description: 'exports all slides and some texts from PPT files',
        group: 'docToolchain'
) {
    doLast {
        File sourceDir = file(srcDir)
        logger.info("sourceDir: ${sourceDir}")
        //make sure path for notes exists
        //and remove old notes
        new File(sourceDir, 'ppt').deleteDir()
        //also remove old diagrams
        new File(sourceDir, 'images/ppt').deleteDir()
        //create a readme to clarify things
        def readme = """This folder contains exported slides or notes from .ppt presentations.

Please note that these are generated files but reside in the `src`-folder in order to be versioned.

This is to make sure that they can be used from environments other than windows.

# Warning!

**The contents of this folder	will be overwritten with each re-export!**

use `gradle exportPPT` to re-export files
"""
        new File(sourceDir, 'images/ppt/.').mkdirs()
        new File(sourceDir, 'images/ppt/readme.ad').write(readme)
        new File(sourceDir, 'ppt/.').mkdirs()
        new File(sourceDir, 'ppt/readme.ad').write(readme)
        def searchPath = new File(sourceDir, 'ppt')
        //execute through cscript in order to make sure that we get WScript.echo right
        "%SystemRoot%\\System32\\cscript.exe //nologo ${projectDir}/scripts/exportPPT.vbs -s ${sourceDir.absolutePath}".executeCmd()
    }
}
exportPPT.vbs
    Const   ForAppending = 8
    Const   ppPlaceholderBody = 2

    ' Helper
    ' http://windowsitpro.com/windows/jsi-tip-10441-how-can-vbscript-create-multiple-folders-path-mkdir-command
    Function MakeDir (strPath)
      Dim strParentPath, objFSO
      Set objFSO = CreateObject("Scripting.FileSystemObject")
      On Error Resume Next
      strParentPath = objFSO.GetParentFolderName(strPath)

      If Not objFSO.FolderExists(strParentPath) Then MakeDir strParentPath
      If Not objFSO.FolderExists(strPath) Then objFSO.CreateFolder strPath
      On Error Goto 0
      MakeDir = objFSO.FolderExists(strPath)

    End Function

	Function SearchPresentations(path)

		  For Each folder In path.SubFolders
		    SearchPresentations folder
		  Next

		  For Each file In path.Files
				If (Left(fso.GetExtensionName (file.Path), 3) = "ppt") OR (Left(fso.GetExtensionName (file.Path), 3) = "pps")  Then
					WScript.echo "found "&file.path
					ExportSlides(file.Path)
				End If
		  Next

    End Function

    Sub ExportSlides(sFile)
        Set objRegEx = CreateObject("VBScript.RegExp")
        objRegEx.Global = True
        objRegEx.IgnoreCase = True
        objRegEx.MultiLine = True
        ' "." doesn't work for multiline in vbs, "[\s,\S]" does...
        objRegEx.Pattern = "[\s,\S]*{adoc}"
        ' http://www.pptfaq.com/FAQ00481_Export_the_notes_text_of_a_presentation.htm
        strFileName = fso.GetFIle(sFile).Name
        Err.Clear
        Set oPPT = CreateObject("PowerPoint.Application")
        Set oPres = oPPT.Presentations.Open(sFile, True, False, False) ' Read Only, No Title, No Window
        On Error resume next
        Set oSlides = oPres.Slides
        WScript.echo "number slides: "&oSlides.Count
        strNotesText = ""
        strImagePath = "/images/ppt/" & strFileName & "/"
        MakeDir(searchPath & strImagePath)
        strNotesPath = "/ppt/"
        MakeDir(searchPath & strNotesPath)
        For Each oSl In oSlides
           strSlideName = oSl.Name
           'WScript.echo fso.GetAbsolutePathName(searchPath) & strImagePath & strSlideName & ".jpg"
           oSl.Export fso.GetAbsolutePathName(searchPath) & strImagePath & strSlideName & ".jpg", ".jpg"

            For Each oSh In oSl.NotesPage.Shapes
                If oSh.PlaceholderFormat.Type = ppPlaceholderBody  Then
                    If oSh.HasTextFrame Then
                        If oSh.TextFrame.HasText Then
                            strCurrentNotes = oSh.TextFrame.TextRange.Text
                            strCurrentNotes = Replace(strCurrentNotes,vbVerticalTab, vbCrLf)
                            strCurrentNotes = Replace(strCurrentNotes,"{slide}","image::ppt/"&strFileName&"/"&strSlideName&".jpg[]")
                            ' remove speaker notes before marker "{adoc}"
                            strCurrentNotes = objRegEx.Replace(strCurrentNotes,"")
                            strNotesText = strNotesText  & vbCrLf & strCurrentNotes & vbCrLf & vbCrLf
                        End If
                    End If
                End If
            Next
        Next
        ' WScript.echo fso.GetAbsolutePathName(".") & strNotesPath&""&strFileName&".ad"
        ' http://stackoverflow.com/questions/2524703/save-text-file-utf-8-encoded-with-vba

        Set fsT = CreateObject("ADODB.Stream")
        fsT.Type = 2 'Specify stream type - we want To save text/string data.
        fsT.Charset = "utf-8" 'Specify charset For the source text data.
        fsT.Open 'Open the stream And write binary data To the object
        fsT.WriteText "ifndef::imagesdir[:imagesdir: ../../images]"&vbCrLf&CStr(strNotesText)
        fsT.SaveToFile fso.GetAbsolutePathName(searchPath) & strNotesPath&""&strFileName&".ad", 2 'Save binary data To disk
        oPres.Close()
        oPPT.Quit()

        If Err.Number <> 0 Then
            WScript.Echo "Error: " & Err.Number
            WScript.Echo "Error (Hex): " & Hex(Err.Number)
            WScript.Echo "Source: " &  Err.Source
            WScript.Echo "Description: " &  Err.Description
            Err.Clear             ' Clear the Error
        End If
    End Sub

  set fso = CreateObject("Scripting.fileSystemObject")
  WScript.echo "Slide extractor"
    Set objArguments = WScript.Arguments

    Dim argCount
    argCount = 0
    While objArguments.Count > argCount+1
      Select Case objArguments(argCount)
        Case "-s"
          searchPath = objArguments(argCount+1)
      End Select
      argCount = argCount + 2
    WEnd

  WScript.echo "looking for .ppt files in " & fso.GetAbsolutePathName(searchPath)
  SearchPresentations fso.GetFolder(searchPath)
  WScript.echo "finished exporting slides"

3.23. exportExcel

Jody Winter Schrotti Jody Winter Ralf D. Müller Ralf D. Müller Dr. Stefan Pfeiffer

2 minutes to read
At a Glance
exportExcel
About This Task

Sometimes you need to include tabular data in your documentation. Most likely, this data will be stored as a MS Excel spreadsheet, or you may like to use Excel to create and edit it. Either way, this task lets you export an Excel spreadsheet and include it directly in your docs. It searches for .xlsx files and exports each contained worksheet as .csv and as .adoc. Note that formulas contained in your spreadsheet are evaluated and exported statically.

The generated files are written to src/excel/[filename]/[worksheet].(adoc|cvs). The src folder is used instead of the build folder because a better history of worksheet changes is captured.

The files can be included either as AsciiDoc:

include::excel/Sample.xlsx/Numerical.adoc[]

…​or as a CSV file:

[options="header",format="csv"]
|===
include::excel/Sample.xlsx/Numerical.csv[]
|===

The AsciiDoc version gives you a bit more control because the following are preserved:

  • Horizontal and vertical alignment.

  • col-span and row-span.

  • Line breaks.

  • Column width relative to other columns.

  • Background colors.

Further Reading and Resources

See asciidoctorj-office-extension to learn another way to use Excel spreadsheets in your docs.

Source
build.gradle
task exportExcel(
        description: 'exports all excelsheets to csv and AsciiDoc',
        group: 'docToolchain'
) {
    doFirst {
        File sourceDir = file(srcDir)

        def tree = fileTree(srcDir).include('**/*.xlsx').exclude('**/~*')

        def exportFileDir = new File(sourceDir, 'excel')

        //make sure path for notes exists
        exportFileDir.deleteDir()
        //create a readme to clarify things
        def readme = """This folder contains exported workbooks from Excel.

Please note that these are generated files but reside in the `src`-folder in order to be versioned.

This is to make sure that they can be used from environments other than windows.

# Warning!

**The contents of this folder will be overwritten with each re-export!**

use `gradle exportExcel` to re-export files
"""
        exportFileDir.mkdirs()
        new File(exportFileDir, '/readme.ad').write(readme)
    }
    doLast {
        File sourceDir = file(srcDir)
        def exportFileDir = new File(sourceDir, 'excel')
        def tree = fileTree(srcDir).include('**/*.xlsx').exclude('**/~*')

        def nl = System.getProperty("line.separator")

        def export = { sheet, evaluator, targetFileName ->
            def targetFileCSV = new File(targetFileName + '.csv')
            def targetFileAD = new File(targetFileName + '.adoc')
            def df = new org.apache.poi.ss.usermodel.DataFormatter();
            def regions = []
            sheet.numMergedRegions.times {
                regions << sheet.getMergedRegion(it)
            }
            logger.debug "sheet contains ${regions.size()} regions"
            def color = ''
            def resetColor = false
            def numCols = 0
            def headerCreated = false
            def emptyRows = 0
            for (int rowNum=0; rowNum<=sheet.lastRowNum; rowNum++) {
                def row = sheet.getRow(rowNum)
                if (row && !headerCreated) {
                    headerCreated = true
                    // create AsciiDoc table header
                    def width = []
                    numCols = row.lastCellNum
                    numCols.times { columnIndex ->
                        width << sheet.getColumnWidth((int) columnIndex)
                    }
                    //lets make those numbers nicer:
                    width = width.collect { Math.round(100 * it / width.sum()) }
                    targetFileAD.append('[options="header",cols="' + width.join(',') + '"]' + nl)
                    targetFileAD.append('|===' + nl)
                }
                def data = []
                def style = []
                def colors = []
                // For each row, iterate through each columns
                if (row && (row?.lastCellNum!=-1)) {
                    numCols.times { columnIndex ->
                        def cell = row.getCell(columnIndex)
                        if (cell) {
                            def cellValue = df.formatCellValue(cell, evaluator)
                            if (cellValue.startsWith('*') && cellValue.endsWith('\u20AC')) {
                                // Remove special characters at currency
                                cellValue = cellValue.substring(1).trim();
                            }
                            def cellStyle = ''
                            def region = regions.find { it.isInRange(cell.rowIndex, cell.columnIndex) }
                            def skipCell = false
                            if (region) {
                                //check if we are in the upper left corner of the region
                                if (region.firstRow == cell.rowIndex && region.firstColumn == cell.columnIndex) {
                                    def colspan = 1 + region.lastRow - region.firstRow
                                    def rowspan = 1 + region.lastColumn - region.firstColumn
                                    if (rowspan > 1) {
                                        cellStyle += "${rowspan}"
                                    }
                                    if (colspan > 1) {
                                        cellStyle += ".${colspan}"
                                    }
                                    cellStyle += "+"
                                } else {
                                    skipCell = true
                                }
                            }
                            if (!skipCell) {
                                switch (cell.cellStyle.alignmentEnum.toString()) {
                                    case 'RIGHT':
                                        cellStyle += '>'
                                        break
                                    case 'CENTER':
                                        cellStyle += '^'
                                        break
                                }
                                switch (cell.cellStyle.verticalAlignmentEnum.toString()) {
                                    case 'BOTTOM':
                                        cellStyle += '.>'
                                        break
                                    case 'CENTER':
                                        cellStyle += '.^'
                                        break
                                }
                                color = cell.cellStyle.fillForegroundXSSFColor?.RGB?.encodeHex()
                                color = color != null ? nl + "{set:cellbgcolor:#${color}}" : ''
                                data << cellValue
                                if (color == '' && resetColor) {
                                    colors << nl + "{set:cellbgcolor!}"
                                    resetColor = false
                                } else {
                                    colors << color
                                }
                                if (color != '') {
                                    resetColor = true
                                }
                                style << cellStyle
                            } else {
                                data << ""
                                colors << ""
                                style << "skip"
                            }
                        } else {
                            data << ""
                            colors << ""
                            style << ""
                        }

                    }
                    emptyRows = 0
                } else {
                    if (emptyRows<3) {
                        //insert empty row
                        numCols.times {
                            data << ""
                            colors << ""
                            style << ""
                        }
                        emptyRows++
                    } else {
                        break
                    }
                }

                targetFileCSV.append(data
                        .collect {
                    "\"${it.replaceAll('"', '""')}\""
                }
                .join(',') + nl, 'UTF-8')
                targetFileAD.append(data
                        .withIndex()
                        .collect { value, index ->
                    if (style[index] == "skip") {
                        ""
                    } else {
                        style[index] + "| ${value.replaceAll('[|]', '{vbar}').replaceAll("\n", ' +$0') + colors[index]}"
                    }
                }
                .join(nl) + nl * 2, 'UTF-8')
            }
            targetFileAD.append('|===' + nl)
        }


        tree.each { File excel ->
            println "file: " + excel
            def excelDir = new File(exportFileDir, excel.getName())
            excelDir.mkdirs()
            InputStream inp
            inp = new FileInputStream(excel)
            def wb = org.apache.poi.ss.usermodel.WorkbookFactory.create(inp);
            def evaluator = wb.getCreationHelper().createFormulaEvaluator();
            for (int wbi = 0; wbi < wb.getNumberOfSheets(); wbi++) {
                def sheetName = wb.getSheetAt(wbi).getSheetName()
                println " -- sheet: " + sheetName
                def targetFile = new File(excelDir, sheetName)
                export(wb.getSheetAt(wbi), evaluator, targetFile.getAbsolutePath())
            }
            inp.close();
        }
    }
}

Ralf D. Müller

1 minute to read

4. About This Task

The exportMarkdown task can be used to include markdown files into the documentation. It scans the /src/docs directory for markdown (*.md) files and converts them into Asciidoc files. The converted files can then be referenced from within the /build-folder.

4.1. Source

exportMarkdown.gradle
task exportMarkdown(
        description: 'exports all markdown files to AsciiDoc',
        group: 'docToolchain',
        type: Copy
) {
    from srcDir

    include("**/*.md") //include only markdown files
    includeEmptyDirs = false
    rename(/(.+).md/, '$1.adoc') //rename all files from *.md to *.adoc
    filter(Markdown2AdocFilter) // convert the content of the files

    into targetDir
}

class Markdown2AdocFilter extends FilterReader {
    Markdown2AdocFilter(Reader input) {
        super(new StringReader(nl.jworks.markdown_to_asciidoc.Converter.convertMarkdownToAsciiDoc(input.text)))
    }
}

4.2. exportOpenAPI

Jody Winter Ralf D. Müller Vladi

1 minute to read
About This Task

This task exports an OpenAPI Specification definition yaml file to a AsciiDoc document. Currently this task depends on OpenAPI Generator (v4.3.1) and its gradle plugin.

openAPI
Configuration
Config.groovy
// Configuration for OpenAPI related task
openApi = [:]

// 'specFile' is the name of OpenAPI specification yaml file. Tool expects this file inside working dir (as a filename or relative path with filename)
// 'infoUrl' and 'infoEmail' are specification metadata about further info related to the API. By default this values would be filled by openapi-generator plugin placeholders
//

openApi.with {
    specFile = 'src/docs/petstore-v2.0.yaml' // i.e. 'petstore.yaml', 'src/doc/petstore.yaml'
    infoUrl = 'https://my-api.company.com'
    infoEmail = 'info@company.com'
}
Source
exportOpenApi.gradle
task exportOpenApi (
        type: org.openapitools.generator.gradle.plugin.tasks.GenerateTask,
        group: 'docToolchain',
        description: 'exports OpenAPI specification to the asciidoc file') {

    if (!specFile) {
        logger.info("\n---> OpenAPI specification file not found in Config.groovy (https://doctoolchain.github.io/docToolchain/#_exportopenapi)")
        return
    } else {
        logger.info("Found OpenAPI specification in Config.groovy")
    }

    outputs.upToDateWhen { false }
    outputs.cacheIf { false }
    generatorName = 'asciidoc'
    outputDir = "${targetDir}/OpenAPI".toString()
    inputSpec = "${docDir}/${specFile}" // plugin is not able to find file if inputPath is defined as '.'

    logger.debug("\n=====================\nProject Config:\n=====================")
    logger.debug("Docdir: ${docDir}")
    logger.debug("Target: ${targetDir}")
    logger.info("\n=====================\nOpenAPI Config:\n=====================")
    logger.info("Specification file: ${specFile}")
    logger.info("inputSpec: ${inputSpec}")
    logger.info("outputDir: ${outputDir}\n")

    additionalProperties = [
        infoEmail:"${config.openApi.infoEmail}",
        infoUrl:"${config.openApi.infoUrl}"
    ]
}

4.3. htmlSanityCheck

Jody Winter Ralf D. Müller Alexander Schwartz Stefan Pfeiffer

1 minute to read
At a Glance
htmlSanityCheck
About This Task

This task invokes the htmlSanityCheck gradle plugin. It is a standalone (batch- and command-line) HTML sanity checker whose role is to detect missing images, dead links and duplicated bookmarks. In docToolchain, the htmlSanityCheck task ensures that generated HTML contains no missing links or other problems. It is the last default task, and creates a report in build/report/htmlchecks/index.html (see example below).

htmlSanityCheck
Figure 4. sample report
Further Reading and Resources
Source
htmlSanityCheck.gradle
htmlSanityCheck {
    sourceDir = new File(config.htmlSanityCheck.sourceDir?targetDir+"/"+config.htmlSanityCheck.sourceDir:"$targetDir/html5")
    // files to check - in Set-notation
    //sourceDocuments = [ "one-file.html", "another-file.html", "index.html"]

    // where to put results of sanityChecks...
    checkingResultsDir = new File(config.htmlSanityCheck.checkingResultsDir?:checkingResultsPath)
    // directory where the results written to in JUnit XML format
    junitResultsDir = new File(config.htmlSanityCheck.junitResultsDir?:"$targetDir/test-results/htmlchecks")

    // which statuscodes shall be interpreted as warning, error or success defaults to standard
    httpSuccessCodes = config.htmlSanityCheck.httpSuccessCodes?:[]
    httpWarningCodes = config.htmlSanityCheck.httpWarningCodes?:[]
    httpErrorCodes = config.htmlSanityCheck.httpErrorCodes?:[]

    // fail build on errors?
    failOnErrors = config.htmlSanityCheck.failOnErrors?:false

    logger.info "docToolchain> HSC sourceDir: ${sourceDir}"
    logger.info "docToolchain> HSC checkingResultsDir: ${checkingResultsDir}"
}

4.4. dependencyUpdates

Jody Winter Jody Winter Ralf D. Müller Alexander Schwartz Dr. Stefan Pfeiffer jakubjab

1 minute to read
About This Task

This task uses the Gradle versions plugin created by Ben Manes to check for outdated build dependencies. Use this task to keep all dependencies up to date.

If you discover newer version, it doesn’t mean that versions and dependencies will play nicely together. To ensure that everything works, we recommend the versions selected by docToolchain contributors.
Further Reading and Resources

Unresolved directive in <stdin> - include::../010_manual/08_development.adoc[leveloffset=+1]


Unresolved directive in <stdin> - include::../010_manual/050_Frequently_asked_Questions.adoc[leveloffset=+1]


Unresolved directive in <stdin> - include::../010_manual/04_further_reading.adoc[leveloffset=+1]

Unresolved directive in <stdin> - include::../010_manual/100_config.adoc[leveloffset=+1]