Commit Graph
7301 Commits
Author SHA1 Message Date
Stephen MorrisandEvan Hunt 89772c3dda [rt46602] Miscellenous small changes
(cherry picked from commit b88b508b22)
(cherry picked from commit 3329876ffd)
(cherry picked from commit d2e214bff3)
2018-02-25 09:27:12 -08:00
Stephen MorrisandEvan Hunt c447d0e26d [rt46602] Handle echoing of multi-line output from tests
(cherry picked from commit 1f07c69f25)
(cherry picked from commit 6602a31229)
(cherry picked from commit c136a2fdb2)
2018-02-25 09:27:12 -08:00
Stephen MorrisandEvan Hunt 4ee549091e [rt46602] Rename "test" variable to "systest"
... to avoid confusion between the variable and the command of the
same name.

(cherry picked from commit c12e06755f)
(cherry picked from commit 7a5476173a)
(cherry picked from commit b2319386f4)
2018-02-25 09:27:12 -08:00
Stephen MorrisandEvan Hunt 275ece8341 [rt46602] Add ability to prevent deletion of output files when running all tests
Some tests may need to examine all output files from all system tests.
Allow the deletion of these files to be inhibited if required.

(cherry picked from commit 00bc29640b)
(cherry picked from commit 6917ea971a)
(cherry picked from commit 9040d3d367)
2018-02-25 09:27:12 -08:00
Stephen MorrisandEvan Hunt 5a5f853966 [rt46602] Error messages before start of test are output to stderr
Error messages concerning the invocation of run.sh are output to
stderr.  Messages after the test has started are output to stdout.

(cherry picked from commit 244d1c30e2)
(cherry picked from commit ff5349b3b3)
(cherry picked from commit 5a1b50c023)
2018-02-25 09:27:12 -08:00
Stephen MorrisandEvan Hunt 9e9356e65e [rt46602] Ensure that output sent to stderr is routed to the output file
(cherry picked from commit a5cf6bd12a)
(cherry picked from commit 6bd944c9d0)
(cherry picked from commit 67ff8ef970)
2018-02-25 09:27:12 -08:00
Stephen MorrisandEvan Hunt b38ccb911f [rt46602] Minor tidying up of documentation
(cherry picked from commit ce66752adc)
(cherry picked from commit 6b5e5a2da2)
(cherry picked from commit 2a5707943a)
2018-02-25 09:27:12 -08:00
Stephen MorrisandEvan Hunt 9b9a387f15 [rt46602] Check prerequsities before network interfaces when running test.
(cherry picked from commit 549a6c6bc8)
(cherry picked from commit 0afcdf658d)
(cherry picked from commit 6dd8c2ab08)
2018-02-25 09:27:12 -08:00
Stephen MorrisandEvan Hunt 73d62eca87 [rt46602] Minor formatting changes to Perl files
(cherry picked from commit f0f5b5f045)
(cherry picked from commit 5fc5b514e4)
(cherry picked from commit 09e1b5e143)
2018-02-25 09:27:09 -08:00
Stephen MorrisandEvan Hunt 3234ad4c63 [rt46603] Provide echostart() and echoend() to output test start/end messages.
(cherry picked from commit c344e1d768)
(cherry picked from commit 88081b765e)
(cherry picked from commit dc0078e03b)
2018-02-25 09:26:35 -08:00
Stephen MorrisandEvan Hunt fad413366a [rt46602] Ensure test output in systests.output is not mixed up
When running all the system tests, output from a test is sent to a
test.output file in the test directory.  These are combined in to
systests.output when the run finishes.

(cherry picked from commit 055e5be9fd)
(cherry picked from commit 50769a4a9e)
(cherry picked from commit 818ac05cfc)
2018-02-25 09:26:32 -08:00
Stephen MorrisandEvan Hunt 2e36ec8950 [rt46602] Miscellaneous minor fixes
... for misspellings, typos, trailing spaces, duplicated lines etc.

(cherry picked from commit 3c25e5fa02)
(cherry picked from commit 0e2e251a0d)
(cherry picked from commit f55e2791fc)
2018-02-25 09:26:23 -08:00
Stephen MorrisandEvan Hunt e553b00a8b [rt46602] Allocate 100 ports to each test
(cherry picked from commit edc7a18d84)
(cherry picked from commit 61514873e9)
(cherry picked from commit 21e33308d1)
2018-02-25 09:26:23 -08:00
Stephen MorrisandEvan Hunt ee59ba17b2 [rt46602] Pass port numbers to tests via environment variables
(cherry picked from commit f5d8f07900)
(cherry picked from commit f10b6cfeb6)
(cherry picked from commit 73d3527931)
2018-02-25 09:26:20 -08:00
Stephen MorrisandEvan Hunt d702992d06 [rt46602] Ensure parallel tests are in alphabetical order
(cherry picked from commit b2d90da0c2)
(cherry picked from commit bf2992dbe6)
(cherry picked from commit 3ddf401b1b)
2018-02-25 09:25:42 -08:00
Stephen MorrisandEvan Hunt ce03881a4a [rt46602] Expanded system tests README
Add more information on running the tests, together with a section
on how the tests are organised, aimed at new developers.

(cherry picked from commit af005cdbcf)
(cherry picked from commit 3952d4631e)
(cherry picked from commit 67f135c1d0)
2018-02-25 09:25:40 -08:00
Stephen MorrisandEvan Hunt becd4cd050 [rt46602] More updates to the test running framework
Tidy up the stop/start files and make switch usage consistent. Also
tidy up the various "clean" targets in the Makefile.

(cherry picked from commit b24c2e11d8)
(cherry picked from commit 78494c3a4d)
(cherry picked from commit 28c2b0be93)
2018-02-25 09:25:09 -08:00
Stephen MorrisandEvan Hunt 00ef25535b [rt46602] Modify some test files to cope with the "-p" flag
(cherry picked from commit 517d990bd4)
(cherry picked from commit b01b553d72)
(cherry picked from commit ca76477b36)
2018-02-25 09:22:30 -08:00
Stephen MorrisandEvan Hunt 320d22fa45 [rt46602] Tidy up run management
Miscellaneous tidying up of run management.  The most significant
change is that "runall.sh" now runs _all_ the tests, even the
ones that can run in parallel.  runsequential.sh is the script
to run tests that have not been converted to parallel running.

(cherry picked from commit 32fe6f7682)
(cherry picked from commit a49afdaeb8)
(cherry picked from commit dec6d02f28)
2018-02-25 09:22:30 -08:00
Stephen MorrisandEvan Hunt 679bc83b07 [rt46602] Assign block of ports for each test
Some tests use more ports than just the query and control ports.
Each test that can run in parallel with other tests is now assigned
a unique block of 10 ports.

(cherry picked from commit e0ff77f9d3)
(cherry picked from commit df1348ad2b)
(cherry picked from commit c64af8abe5)
2018-02-25 09:22:30 -08:00
Stephen MorrisandEvan Hunt cbe47e1752 [rt46602] Miscellaneous changes to existing parallelised tests
Currently these tests are allow_query, rpzrecurse and serve-stale

1. Function to copy files and set port numbers renamed from copy_config
to copy_setports, as this is used to change the ports in Perl and Python
test scripts as well.
2. Changes to rpzrecurse/tests.sh to handle two calls to getopts (one to
parse port numbers, the other to parse rpzrecurse-specific options). Also
fixed various commands to use correct ports.
3. Updates to "clean.sh" scripts to ensure that all files created in the
test are removed.

(cherry picked from commit 78f2b9ca01)
(cherry picked from commit 01eb4d7c97)
(cherry picked from commit f5b71b3658)
2018-02-25 09:22:13 -08:00
Stephen MorrisandEvan Hunt 0342f7e559 [rt46602] Ensure that tests running in parallel use unique ports
Via an intermediate make file, tests that have been modified to be able
to run in parallel are assigned unique query and control port numbers
(other than 5300 and 9953 respectively).  Tests that have not yet been
modified all use ports 5300 and 9953, so must be run sequentially.

(cherry picked from commit e7429b124b)
(cherry picked from commit 4f68e67e5c)
(cherry picked from commit 704e2324e3)
2018-02-25 09:21:52 -08:00
Stephen MorrisandEvan Hunt 8fa27c7358 [rt46602] Miscellaneous tidying up of code cherry-picked from experimental branch
(cherry picked from commit e1a69e5d1f)
(cherry picked from commit ba694ca38c)
(cherry picked from commit c7bf09069d)
2018-02-25 09:21:01 -08:00
Ondřej SurýandEvan Hunt 6f801ceceb [rt46602] Allow parallel run of new dynamic port tests and the old ones
(cherry picked from commit 3b14d54fd7)
(cherry picked from commit a14bd48268)
(cherry picked from commit fbd24ff359)
2018-02-25 09:21:00 -08:00
Ondřej SurýandEvan Hunt d080192323 [rt46602] Put back original .gitignore files in the system tests
(cherry picked from commit caaec1ef6d)
(cherry picked from commit cf7a86661a)
(cherry picked from commit ff2356395b)
2018-02-25 09:21:00 -08:00
Ondřej SurýandEvan Hunt 043bd61301 [rt46602] Adjust rpzrecurse for dynamic port usage
Resolve conflict in run.sh during cherry-pick from an experimental branch.

(cherry picked from commit 2a435708e4)
(cherry picked from commit c5aa6ed87b)
(cherry picked from commit 9bdb4d67f1)
2018-02-25 09:20:58 -08:00
Ondřej SurýandEvan Hunt 0173209dda [rt46602] Add $SEDPORTS variable to help with @PORT@ and @CONTROLPORT@ replacement
(cherry picked from commit e6802badc0)
(cherry picked from commit d37a52cc56)
(cherry picked from commit 76848ca9d4)
2018-02-25 09:17:30 -08:00
Ondřej SurýandEvan Hunt ffa6566643 [rt46602] Move most of ignored files to top of system tests directory
(cherry picked from commit d2c8286163)
(cherry picked from commit a5c3077a9c)
(cherry picked from commit f11975087c)
2018-02-25 09:17:27 -08:00
Ondřej SurýandEvan Hunt 0f99295e12 [rt46602] Fix output from rndc reload
(cherry picked from commit 53bade5ece)
(cherry picked from commit 112355adb8)
(cherry picked from commit 77e5c32baf)
2018-02-25 09:14:51 -08:00
Ondřej SurýandEvan Hunt dce8925c61 [rt46602] Add .gitignore to allow_query
(cherry picked from commit bc276f6c05)
(cherry picked from commit 76dfb5c339)
(cherry picked from commit 51ccb16653)
2018-02-25 09:14:50 -08:00
Ondřej SurýandEvan Hunt 2d8cf8c8cc [rt46602] Utilize echo_i to allow mixed test output
(cherry picked from commit 7e3c96c867)
(cherry picked from commit 93034acb4d)
(cherry picked from commit 9bcf40cbed)
2018-02-25 09:14:50 -08:00
Ondřej SurýandEvan Hunt 585d362afe [rt46602] Prefix R:<result> as R:<test>:<result>
Resolve conflict in run.sh during cherry-pick from experimental branch.

(cherry picked from commit 278c72fef6)
(cherry picked from commit b8e866d60f)
(cherry picked from commit ff56ecafc4)
2018-02-25 09:14:50 -08:00
Ondřej SurýandEvan Hunt 9cff66b3b9 [rt46602] Add echo_i function to print I:<testname>:string
(cherry picked from commit f7ac41757a)
(cherry picked from commit bd5b36f7a4)
(cherry picked from commit cdc72937d6)
2018-02-25 09:14:47 -08:00
Ondřej SurýandEvan Hunt 5b45f67f61 [rt46602] Add serve-stale working files to .gitignore
(cherry picked from commit cba162e70e)
(cherry picked from commit 57adb949a4)
(cherry picked from commit 2327f41b35)
2018-02-25 09:14:30 -08:00
Ondřej SurýandEvan Hunt 10da6c8a87 [rt46602] Convert allow_query to dynamic port
(cherry picked from commit 992a2bafec)
(cherry picked from commit 1d10178e83)
(cherry picked from commit 1a568d5a76)
2018-02-25 09:14:30 -08:00
Ondřej SurýandEvan Hunt a74f729d21 [rt46602] Change the system tests script to support running individual tests on different ports
Resolved conflict in run.sh.

(cherry picked from commit fb00f18005)
(cherry picked from commit 0ee883d443)
(cherry picked from commit 75428f4951)
2018-02-25 09:14:30 -08:00
Mark Andrews 29652923f8 remove 'check flushtree clears adb correctly' test 2018-02-22 14:35:40 +11:00
Michał KępieńandMark Andrews 598d5538fe Improve the way cache contents are searched for "ns.flushtest.example"
During the "check flushtree clears adb correctly" check, expecting
"ns.flushtest.example" to always be the first name in the ADB dump is
fragile, because in a certain corner case "a.root-servers.nil" will be
the first name instead.

As the purpose of the relevant check is to ensure "ns.flushtest.example"
is removed from ADB by "rndc flushtree flushtest.example", search the
entire list of names present in ADB instead of just the first entry when
looking for "ns.flushtest.example".

(cherry picked from commit ca1049b2f3)
(cherry picked from commit 3513a585e3)
2018-02-21 11:28:21 +11:00
Michał KępieńandMark Andrews 580fb63a68 Wait until a cache dump completes instead of waiting for a fixed amount of time
Dumping the cache is an asynchronous operation, so sleeping for a fixed
amount of time after running "rndc dumpdb" is imperfect as dumping cache
contents may take longer than expected on slower machines.  Instead of
always sleeping for 1 second, wait until the "; Dump complete" line
appears in the dump or 10 seconds pass, whichever comes first.

(cherry picked from commit 2bbff06d36)
(cherry picked from commit e617b91527)
2018-02-21 11:28:16 +11:00
Michał KępieńandMark Andrews 44e946e9e9 Do not overwrite cache dumps
Unless configured otherwise in named.conf, "rndc dumpdb" causes a cache
dump to be written to a file called "named_dump.db" in the working
directory of the given named instance.  Repeatedly using this command
throughout different checks in the cacheclean system test causes cache
dumps for older checks to be overwritten, which hinders failure
diagnosis.  Prevent this by moving each cache dump to a check-specific
location after running "rndc dumpdb".

Furthermore, during the "check flushtree clears adb correctly" check,
dump_cache() is called twice without renaming the resulting files.
Prevent the first cache dump from being overwritten by moving it to a
different file before calling "rndc dumpdb" for the second time.

(cherry picked from commit aeea1faf01)
(cherry picked from commit dcd91ab322)
2018-02-21 11:26:58 +11:00
Michał KępieńandMark Andrews 5c0afa76f7 Assign an index to each check in the cacheclean system test
This will enable storing per-check forensic data, which facilitates
troubleshooting test failures.

(cherry picked from commit 01de79b332)
(cherry picked from commit 7679819e15)
2018-02-21 10:52:43 +11:00
Mark Andrews 4404fcb384 return failure count 2018-02-16 18:42:11 +11:00
Mark Andrews 7245f12d3e conditionally declare strbuf 2018-02-16 18:41:31 +11:00
Mark Andrews 872543eca7 add brackets 2018-02-16 18:37:05 +11:00
Mark Andrews 5de799a1c8 reorder test to silence cppcheck 2018-02-16 18:37:05 +11:00
Mark Andrews 32166fae3b remove event which is only used for sizeof 2018-02-16 18:37:05 +11:00
Mark Andrews a4d8e7526a reorder test to silence cppcheck 2018-02-16 18:36:17 +11:00
Mark Andrews f7c7a87cb4 use %u instead of %d 2018-02-16 18:30:18 +11:00
Mark Andrews b7a4a93b0d use %u instead of %d 2018-02-16 18:17:59 +11:00
Mark Andrews cc21083141 use %u instead of %d 2018-02-16 18:16:48 +11:00