ComputerSurge/bind9
3
0
Fork 0
Code Pull Requests 1 Activity

Store system test output in bin/tests/system/

Browse Source 
Some clean.sh scripts contain overly broad file deletion wildcards which
cause the test.output file (used by the system test framework for
collecting output) in a given system test's directory to be erroneously
removed immediately after the test is started (due to setup.sh scripts
calling clean.sh at the beginning).  This prevents the test's output
from being placed in bin/tests/system/systests.output at the end of a
test suite run and thus can lead to test failures being ignored.  Fix by
storing each test's output in a test.output.<test-name> file in
bin/tests/system/, which prevents clean.sh scripts from removing it (as
they should only ever affect files contained in a given system test's
directory).
This commit is contained in:
Michał Kępień
2019-12-06 14:11:01 +01:00
parent 34fb70b17c
commit b0916bba41
6 changed files with 20 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files

20
bin/tests/system/README
View File

@@ -717,17 +717,17 @@ separated by at least 100 ports.
Cleaning Up From Tests
---
When a test is run, files are created in the test directory. These files fall
into three categories:
When a test is run, up to three different types of files are created:
1. Files generated by the test itself, e.g. output from "dig" and "rndc".
1. Files generated by the test itself, e.g. output from "dig" and "rndc", are
stored in the test directory.
2. Files produced by named which may not be cleaned up if named exits
abnormally, e.g. core files, PID files etc.
abnormally, e.g. core files, PID files etc., are stored in the test directory.
3. The file "test.output" containing the text written to stdout by the test.
This is only produced when the test is run as part of the entire test suite
(e.g. via "runall.sh").
3. A file "test.output.<test-name>" containing the text written to stdout by the
test is written to bin/tests/system/. This file is only produced when the test
is run as part of the entire test suite (e.g. via "runall.sh").
If the test fails, all these files are retained. But if the test succeeds,
they are cleaned up at different times:
@@ -739,9 +739,9 @@ called from "run.sh".
2. Files that may not be cleaned up if named exits abnormally are removed
by the system's "clean.sh".
3. "test.output" files are deleted when the test suite ends. At this point,
the file "testsummary.sh" is called which concatenates all the "test.output"
files into a single "systest.output" file before deleting them.
3. "test.output.*" files are deleted when the test suite ends. At this point,
the file "testsummary.sh" is called which concatenates all the "test.output.*"
files into a single "systests.output" file before deleting them.
A complication arises with the "test.output" file however: