Unit Test Plug-in

Unit Testing

• The unit-test transform allows a user to runs a sequence of dita commands and checks that the documents created match the expected output. This is useful for regression testing, and confirming that any custom plug-ins do not conflict when upgrading the base DITA-OT engine.
• The resource/antlib.xml library offers a series of convenience methods for creating DITA-OT unit tests.

Code Coverage

• The token-report transform checks to see if a series of tokens representing all potential output values are covered by unit tests
• The xsl-instrument transform annotates an DITA-OT plug-in to enable code coverage reporting
• The xsl-deinstrument transform removes the instrumentation annotation from a specified plugin
• The xsl-report transform displays which templates have been invoked whilst running unit tests

ANT Profiling

• The antro transform runs an ANT script profiler against a specified transform and outputs a profiler JSON file
• The antro.ui transform starts up the UI for the ANT script profiler, allowing a user to load a JSON file and interpret the results.

Obtaining a unit test report

To run, use the unit-test transform.

dita --format unit-test \
     --output out \
     --input PATH_TO_UNIT_TESTS

Once the command has run, a test report file is created. Additionally, if any error occurs, the command will fail.

Sample Test Report This is the test report from the example tests found within the plug-in sample directory. Nine tests are run over two test suites (PDF and HTML processing) - a third test suite has been disabled.

• If the --input input directory is a test suite, all tests within the suite will be run.
• If the --input input directory is not a test suite, all test suites directly beneath that directory will be run.

XSLT Coverage Report

To run a XSLT Coverage Report, each DITA-OT Plug-In must be instrumented - this annotates the XSLT templates within the plugin to be able to generate coverage information. A copy of each *.xsl file is also saved with the *.orig suffix.

dita --format xsl-instrument \
     --input PATH_TO_PLUG_IN

To revert back to the original files just run the de-instrument transform as shown:

dita --format xsl-deinstrument \
     --input PATH_TO_PLUG_IN

Once a Plug-In has been instrumented the test suite should contain a template-coverage.xml file which holds a list of all template decision points. To obtain coverage information, use the unit-test transform as stated previously:

dita --format unit-test \
     --input PATH_TO_UNIT_TESTS

• If the --input input directory is a test suite, XSL coverage for that test suite will be reported.
• If the --input input directory is not a test suite, XSL coverage for all test suites directly beneath that directory will be reported.

Once the command has run, both a test report file and an XSL coverage report file are created. Additionally, if any error occurs, the command will fail.

It is also possible to run XSL coverage over a single test. An individual test can be run directly from the command line by running the default target within that test. This can be followed by direct invocation of the coverage report.

ant -f PATH_TO_PLUGIN/test/TEST_NAME/build.xml
dita --format xsl-report \
     --input PATH_TO_PLUGIN

Sample XSL Report

The XSL coverage report will show a schematic of all template decision points, with covered lines displayed in green, and uncovered lines displayed in red.

Token Coverage Report

This is a quicker alternative report to XSLT instrumentation and code coverage, but it requires the developer to create the token coverage.xml file manually. Each test suite should contain a coverage.xml file which holds a series of tokens representing all potential output values. To obtain coverage information, use the token-report transform.

dita --format token-report \
     --input PATH_TO_UNIT_TESTS

Once the command has run, a coverage report is created.

Sample Token Report

This is the token report from the example tests found within the plug-in sample directory.

• Coverage for the Spell-checker is looking to ensure that all rules have been run. Uncovered rules are highlighted in RED.
• Coverage for HTML processing is looking for the presence of <codeph>, <codeblock> and <p> tags being rendered in the tests.

• If the --input input directory is a test suite, coverage for that test suite will be reported.
• If the --input input directory is not a test suite, coverage for all test suites directly beneath that directory will be reported.

Obtaining ANT script profile information

Antro is a hierarchical and line-level profiler for Ant build scripts. It can be run to check which ANT scripts have been invoked and how long they took.

To obtain profile information, use the antro transform and supply an additional test transtype.

dita --format antro--test.transform=TRANSFORM_TO_PROFILE \
     --input document.ditamap

A profiler JSON file will be generated.

Viewing profiler information

To run the UI for the Antro profiler, use the antro-ui transform. The --input parameter is mandatory for all DITA-OT plug-ins, and should point to a real file, but is not used for this transform.

dita --format antro-ui \
     --input document.ditamap

The Antro UI is the displayed, load the profiler json file from to display a bar graph showing how long each ANT target took:

Sample Antro Display The Profiler was run against the pdf transform, FOP processing took 16s

You can drill down to an individual line to see if it has been invoked and how long it took:

Parameter Reference

• test.colorize - When set, successes and failures are output highlighted using ANSI color codes
• test.copy - Specifies whether regenerated expecations should be copied. Default is false
• test.transtype - The real transtype to run the antro profiler against
• test.propertyfile - A properties file to use when running the unit tests or antro profiler

Compare-Output

Fail the test if the test output file does not match the expectation file.

<compare-output/>

compares the file out/fragment.fo with expected.fo and fails if they do not match.

<compare-output suffix=".html"/>

compares the file out/fragment.html with expected.html and fails if they do not match.

<compare-output suffix=".svrl" expectation="expected.svrl.win" os="windows"/>

if running on a Windows system, compares the file out/fragment.svrl with expected.svrl.win and fails if they do not match.

if running on a UNIX system, no comparison is made.

Contains-Text

Fail the test if the log from the test does not contain the given string.

<contains-text expected="Lorem Ipsum"/>

compares the output of DITA-OT and fails if the text "Lorem Ipsum" cannot be found.

<contains-text expected="This is running on Windows" os="windows"/>

if running on a Windows system, compares the output of DITA-OT and fails if the text "This is running on Windows" cannot be found.

if running on a UNIX system, no comparison is made.

Exec-HTML5

Execute the HTML5 DITA-OT transform in verbose mode The test will fail if the result was not as expected or took too long.

<exec-html5 transtype="custom-html"/>

runs DITA-OT using the custom-html HTML transtype. the output will be placed in the /out/html directory.

Exec-PDF

Execute the PDF DITA-OT transform in verbose mode.

<exec-pdf transtype="custom-pdf"/>

runs DITA-OT using the custom-pdf PDF transtype. topic.fo and document.pdf will be placed in the /out directory.

Exec-SVRL

Execute the HERE Validator SVRL DITA-OT transform in verbose mode. The test will fail if the result was not as expected or took too long.

<exec-svrl transtype="text-rules"/>

runs DITA-OT using the text-rules SVRL transtype.

<exec-svrl transtype="svrl-echo" expectedresult="1"/>

runs DITA-OT using the svrl-echo SVRL transtype - the invocation is expected to fail with validation errors.

Exec-Transtype

Execute an arbitrary DITA-OT transform in verbose mode. The test will fail if the result was not as expected or took too long.

<exec-svrl transtype="custom"/>

runs DITA-OT using the custom transtype.

<exec-svrl transtype="custom" expectedresult="1"/>

runs DITA-OT using the custom SVRL transtype - the invocation is expected to fail.

Get-HTML-Article

Loads a given HTML file and extracts the first <article> element (which corresponds to a DITA topic) for further examination.

<get-html-article from="topics/body-text.html"/>

creates a file called fragment.html holding the <article> element from the topics/body-text.html file.

Get-PDF-Article

Loads a given topic.fo file and extracts the last fo.flow element (which corresponds to a DITA topic) for further examination. Also remove colors, fonts and excess attributes.

<get-pdf-article/>

creates a file called fragment.fo holding the final <fo.flow> element from the topics.fo file.