66f3efb63b
* migrate runtime safety tests to the new test harness
- this required adding compare output / execution support for stage1
to the test harness.
* rename `zig build test-stage2` to `zig build test-cases` since it now
does quite a bit of stage1 testing actually. I named it this way
since the main directory in the source tree associated with these
tests is "test/cases/".
* add some documentation for the test manifest format.
62 lines
1.4 KiB
Markdown
62 lines
1.4 KiB
Markdown
# Test Case Quick Reference
|
|
|
|
Use comments at the **end of the file** to indicate metadata about the test
|
|
case. Here are examples of different kinds of tests:
|
|
|
|
## Compile Error Test
|
|
|
|
If you want it to be run with `zig test` and match expected error messages:
|
|
|
|
```zig
|
|
// error
|
|
// is_test=1
|
|
//
|
|
// :4:13: error: 'try' outside function scope
|
|
```
|
|
|
|
## Execution
|
|
|
|
This will do `zig run` on the code and expect exit code 0.
|
|
|
|
```zig
|
|
// run
|
|
```
|
|
|
|
## Incremental Compilation
|
|
|
|
Make multiple files that have ".", and then an integer, before the ".zig"
|
|
extension, like this:
|
|
|
|
```
|
|
hello.0.zig
|
|
hello.1.zig
|
|
hello.2.zig
|
|
```
|
|
|
|
Each file can be a different kind of test, such as expecting compile errors,
|
|
or expecting to be run and exit(0). The test harness will use these to simulate
|
|
incremental compilation.
|
|
|
|
At the time of writing there is no way to specify multiple files being changed
|
|
as part of an update.
|
|
|
|
## Subdirectories
|
|
|
|
Subdirectories do not have any semantic meaning but they can be used for
|
|
organization since the test harness will recurse into them. The full directory
|
|
path will be prepended as a prefix on the test case name.
|
|
|
|
## Limiting which Backends and Targets are Tested
|
|
|
|
```zig
|
|
// run
|
|
// backend=stage2,llvm
|
|
// target=x86_64-linux,x86_64-macos
|
|
```
|
|
|
|
Possible backends are:
|
|
|
|
* `stage1`: equivalent to `-fstage1`.
|
|
* `stage2`: equivalent to passing `-fno-stage1 -fno-LLVM`.
|
|
* `llvm`: equivalent to `-fLLVM -fno-stage1`.
|