1// Copyright 2023 The Khronos Group Inc.
2// SPDX-License-Identifier: CC-BY-4.0
3
4= Spec Build Tests
5
6This directory contains test source and expectation files for the `testBuild`
7script. The test specification files are:
8
9 * `testspec.adoc`
10 * `chapters/*.adoc`
11 * `chapters/commonvalidity/*.adoc`
12 * `appendices/*.adoc`
13 * `images/*.svg`
14
15The expectation files are:
16
17 * `expectations/*.html`
18 * `expectations/validusage.json`
19
20The `testBuild` script builds the test specficiation (`testspec.adoc`) in an
21array of configurations, such as Core (latest), Core 1.0, with all extensions,
22with a specific extension etc. Additionally, it extracts the VUs into
23validusage.json.
24
25Each build by `testBuild` is done in a separate directory under `gen-<build>`.
26In all cases except `gen-validusage`, the output is
27`gen-<build>/out/html/vkspec.html`. In the case of `gen-validusage`, the output
28is `gen-validusage/out/validition/validusage.json`.
29
30The primary reason for these tests is ensuring correctness of asciidoc
31extensions implemented in this repository. The `testBuild` script first builds
32all configurations, then verifies the results against the expectations.
33
34If a build itself regresses (and fails), the script will stop so the issue can
35be addressed after looking at the logs of the failing build. If the builds
36succeed but the output does not match the expectations, the diff is output for
37investigation.
38
39In some cases, the diff is small enough to be verified. Otherwise, each output
40in `gen-<build>/` needs to be manually reviewed to ensure the results are still
41correct; for example because the html is styled differently and the differences
42are as expected.
43
44If the mismatch between the output and expectations is expected, and the output
45is verified to be correct, update the expectations with:
46
47----
48$ ./update-expectations
49----
50