Commit Graph

24286 Commits

Author SHA1 Message Date
Stephen Morris
2a24aac12e [rt46602] Mention conf.sh.win32 in instructions for adding a test.
(cherry picked from commit 23dfa27a84)
(cherry picked from commit 06a9a37d3b)
(cherry picked from commit d9740c20da)
2018-02-25 09:28:45 -08:00
Michał Kępień
d634594e56 [rt46602] Fix various typos, formatting issues and stylistic nits
(cherry picked from commit f0721de624)
(cherry picked from commit b40e75a75c)
(cherry picked from commit 85f03bb30f)
2018-02-25 09:28:45 -08:00
Michał Kępień
2a31548089 [rt46602] Remove redundant variable assignment
(cherry picked from commit 42cf18d239)
(cherry picked from commit 4436930712)
(cherry picked from commit 90654248f6)
2018-02-25 09:28:45 -08:00
Michał Kępień
df5de60665 [rt46602] named.port is no longer created by setup.sh for allow_query
(cherry picked from commit 1ac4c3e4e5)
(cherry picked from commit 77e6ff0fae)
(cherry picked from commit 2e3d58ad30)
2018-02-25 09:28:45 -08:00
Michał Kępień
653dcebec8 [rt46602] Consistently print usage for runall.sh to stderr
(cherry picked from commit fe1ccafc34)
(cherry picked from commit 47e278b7c9)
(cherry picked from commit 66985fad62)
2018-02-25 09:28:45 -08:00
Michał Kępień
4ef8f1c774 [rt46602] Clean up getopts usage
Ensure case clauses are in sync with the string passed to getopts.
Remove catch-all clauses as they will never be evaluated.

(cherry picked from commit 0dcee1cad3)
(cherry picked from commit 0c8e56568f)
(cherry picked from commit 6d9fb2b429)
2018-02-25 09:28:45 -08:00
Michał Kępień
17fce84a63 [rt46602] Assign default values to variables set by command line options and consistently treat them as integers
(cherry picked from commit fe51c34df3)
(cherry picked from commit 3752edb28e)
(cherry picked from commit 1adbfeb3ed)
2018-02-25 09:28:45 -08:00
Michał Kępień
6d8fde8a92 [rt46602] named.port cannot be ignored by git because it is required to be present by notify/ns4
(cherry picked from commit 1960de5530)
(cherry picked from commit f47ed5ceb0)
(cherry picked from commit 01e21e3ee2)
2018-02-25 09:28:45 -08:00
Stephen Morris
98ddc2c422 [rt46602] Remove option to format the printed start time of a test
There was too much incompatibility between operating systems as to
the allowed options for "date".

(cherry picked from commit 451b7309ca)
(cherry picked from commit 8376949ea8)
(cherry picked from commit 52f784315e)
2018-02-25 09:28:45 -08:00
Stephen Morris
90dccece92 [rt46602] Escape line ends in shell script variable assignment
Escape the line ends in a multi-line variable assignment.  Under some
circumstances, substituting the variable caused syntax errors when
used as the list of values in a shell script "for" statement.

(cherry picked from commit 462766cd76)
(cherry picked from commit 53ebdf225f)
(cherry picked from commit 82ffebc73a)
2018-02-25 09:28:42 -08:00
Stephen Morris
6f8c363547 [rt46602] Fix EXTRAPORT substitutions in copy_setport
Fix problems where all EXTRAPORT macros were being replaced by
the EXTRAPORT1 value. (Copy and paste issue.)

(cherry picked from commit fb78f82ba1)
(cherry picked from commit 8513c4256b)
(cherry picked from commit c8880ce14a)
2018-02-25 09:27:13 -08:00
Stephen Morris
0c59973a13 [rt46602] Minor documentation changes
(cherry picked from commit cdf18a5009)
(cherry picked from commit 4731a82eef)
(cherry picked from commit 04120c4223)
2018-02-25 09:27:13 -08:00
Stephen Morris
2dae3e430a [rt46602] Correct way allow_query test produces RNDC output
(cherry picked from commit 7f2d41d4c4)
(cherry picked from commit 36136dec55)
(cherry picked from commit 7125bddda8)
2018-02-25 09:27:13 -08:00
Stephen Morris
9490bd8355 [rt46602] Simplify the cleanup code
(cherry picked from commit 0855a8a6cc)
(cherry picked from commit fb5974f902)
(cherry picked from commit 5d69915cd8)
2018-02-25 09:27:12 -08:00
Stephen Morris
62f2f90bd3 [rt46602] Extensions to README
Added descriptions of how nameservers are started and stopped
during the tests, and how the framework cleans up the files created.

(cherry picked from commit f606b17dfb)
(cherry picked from commit 56ba2e822b)
(cherry picked from commit 5be3ea811e)
2018-02-25 09:27:12 -08:00
Stephen Morris
89772c3dda [rt46602] Miscellenous small changes
(cherry picked from commit b88b508b22)
(cherry picked from commit 3329876ffd)
(cherry picked from commit d2e214bff3)
2018-02-25 09:27:12 -08:00
Stephen Morris
c447d0e26d [rt46602] Handle echoing of multi-line output from tests
(cherry picked from commit 1f07c69f25)
(cherry picked from commit 6602a31229)
(cherry picked from commit c136a2fdb2)
2018-02-25 09:27:12 -08:00
Stephen Morris
4ee549091e [rt46602] Rename "test" variable to "systest"
... to avoid confusion between the variable and the command of the
same name.

(cherry picked from commit c12e06755f)
(cherry picked from commit 7a5476173a)
(cherry picked from commit b2319386f4)
2018-02-25 09:27:12 -08:00
Stephen Morris
275ece8341 [rt46602] Add ability to prevent deletion of output files when running all tests
Some tests may need to examine all output files from all system tests.
Allow the deletion of these files to be inhibited if required.

(cherry picked from commit 00bc29640b)
(cherry picked from commit 6917ea971a)
(cherry picked from commit 9040d3d367)
2018-02-25 09:27:12 -08:00
Stephen Morris
5a5f853966 [rt46602] Error messages before start of test are output to stderr
Error messages concerning the invocation of run.sh are output to
stderr.  Messages after the test has started are output to stdout.

(cherry picked from commit 244d1c30e2)
(cherry picked from commit ff5349b3b3)
(cherry picked from commit 5a1b50c023)
2018-02-25 09:27:12 -08:00
Stephen Morris
9e9356e65e [rt46602] Ensure that output sent to stderr is routed to the output file
(cherry picked from commit a5cf6bd12a)
(cherry picked from commit 6bd944c9d0)
(cherry picked from commit 67ff8ef970)
2018-02-25 09:27:12 -08:00
Stephen Morris
b38ccb911f [rt46602] Minor tidying up of documentation
(cherry picked from commit ce66752adc)
(cherry picked from commit 6b5e5a2da2)
(cherry picked from commit 2a5707943a)
2018-02-25 09:27:12 -08:00
Stephen Morris
9b9a387f15 [rt46602] Check prerequsities before network interfaces when running test.
(cherry picked from commit 549a6c6bc8)
(cherry picked from commit 0afcdf658d)
(cherry picked from commit 6dd8c2ab08)
2018-02-25 09:27:12 -08:00
Stephen Morris
73d62eca87 [rt46602] Minor formatting changes to Perl files
(cherry picked from commit f0f5b5f045)
(cherry picked from commit 5fc5b514e4)
(cherry picked from commit 09e1b5e143)
2018-02-25 09:27:09 -08:00
Stephen Morris
3234ad4c63 [rt46603] Provide echostart() and echoend() to output test start/end messages.
(cherry picked from commit c344e1d768)
(cherry picked from commit 88081b765e)
(cherry picked from commit dc0078e03b)
2018-02-25 09:26:35 -08:00
Stephen Morris
fad413366a [rt46602] Ensure test output in systests.output is not mixed up
When running all the system tests, output from a test is sent to a
test.output file in the test directory.  These are combined in to
systests.output when the run finishes.

(cherry picked from commit 055e5be9fd)
(cherry picked from commit 50769a4a9e)
(cherry picked from commit 818ac05cfc)
2018-02-25 09:26:32 -08:00
Stephen Morris
2e36ec8950 [rt46602] Miscellaneous minor fixes
... for misspellings, typos, trailing spaces, duplicated lines etc.

(cherry picked from commit 3c25e5fa02)
(cherry picked from commit 0e2e251a0d)
(cherry picked from commit f55e2791fc)
2018-02-25 09:26:23 -08:00
Stephen Morris
e553b00a8b [rt46602] Allocate 100 ports to each test
(cherry picked from commit edc7a18d84)
(cherry picked from commit 61514873e9)
(cherry picked from commit 21e33308d1)
2018-02-25 09:26:23 -08:00
Stephen Morris
ee59ba17b2 [rt46602] Pass port numbers to tests via environment variables
(cherry picked from commit f5d8f07900)
(cherry picked from commit f10b6cfeb6)
(cherry picked from commit 73d3527931)
2018-02-25 09:26:20 -08:00
Stephen Morris
d702992d06 [rt46602] Ensure parallel tests are in alphabetical order
(cherry picked from commit b2d90da0c2)
(cherry picked from commit bf2992dbe6)
(cherry picked from commit 3ddf401b1b)
2018-02-25 09:25:42 -08:00
Stephen Morris
ce03881a4a [rt46602] Expanded system tests README
Add more information on running the tests, together with a section
on how the tests are organised, aimed at new developers.

(cherry picked from commit af005cdbcf)
(cherry picked from commit 3952d4631e)
(cherry picked from commit 67f135c1d0)
2018-02-25 09:25:40 -08:00
Stephen Morris
becd4cd050 [rt46602] More updates to the test running framework
Tidy up the stop/start files and make switch usage consistent. Also
tidy up the various "clean" targets in the Makefile.

(cherry picked from commit b24c2e11d8)
(cherry picked from commit 78494c3a4d)
(cherry picked from commit 28c2b0be93)
2018-02-25 09:25:09 -08:00
Stephen Morris
00ef25535b [rt46602] Modify some test files to cope with the "-p" flag
(cherry picked from commit 517d990bd4)
(cherry picked from commit b01b553d72)
(cherry picked from commit ca76477b36)
2018-02-25 09:22:30 -08:00
Stephen Morris
320d22fa45 [rt46602] Tidy up run management
Miscellaneous tidying up of run management.  The most significant
change is that "runall.sh" now runs _all_ the tests, even the
ones that can run in parallel.  runsequential.sh is the script
to run tests that have not been converted to parallel running.

(cherry picked from commit 32fe6f7682)
(cherry picked from commit a49afdaeb8)
(cherry picked from commit dec6d02f28)
2018-02-25 09:22:30 -08:00
Stephen Morris
679bc83b07 [rt46602] Assign block of ports for each test
Some tests use more ports than just the query and control ports.
Each test that can run in parallel with other tests is now assigned
a unique block of 10 ports.

(cherry picked from commit e0ff77f9d3)
(cherry picked from commit df1348ad2b)
(cherry picked from commit c64af8abe5)
2018-02-25 09:22:30 -08:00
Stephen Morris
cbe47e1752 [rt46602] Miscellaneous changes to existing parallelised tests
Currently these tests are allow_query, rpzrecurse and serve-stale

1. Function to copy files and set port numbers renamed from copy_config
to copy_setports, as this is used to change the ports in Perl and Python
test scripts as well.
2. Changes to rpzrecurse/tests.sh to handle two calls to getopts (one to
parse port numbers, the other to parse rpzrecurse-specific options). Also
fixed various commands to use correct ports.
3. Updates to "clean.sh" scripts to ensure that all files created in the
test are removed.

(cherry picked from commit 78f2b9ca01)
(cherry picked from commit 01eb4d7c97)
(cherry picked from commit f5b71b3658)
2018-02-25 09:22:13 -08:00
Stephen Morris
0342f7e559 [rt46602] Ensure that tests running in parallel use unique ports
Via an intermediate make file, tests that have been modified to be able
to run in parallel are assigned unique query and control port numbers
(other than 5300 and 9953 respectively).  Tests that have not yet been
modified all use ports 5300 and 9953, so must be run sequentially.

(cherry picked from commit e7429b124b)
(cherry picked from commit 4f68e67e5c)
(cherry picked from commit 704e2324e3)
2018-02-25 09:21:52 -08:00
Stephen Morris
8fa27c7358 [rt46602] Miscellaneous tidying up of code cherry-picked from experimental branch
(cherry picked from commit e1a69e5d1f)
(cherry picked from commit ba694ca38c)
(cherry picked from commit c7bf09069d)
2018-02-25 09:21:01 -08:00
Ondřej Surý
6f801ceceb [rt46602] Allow parallel run of new dynamic port tests and the old ones
(cherry picked from commit 3b14d54fd7)
(cherry picked from commit a14bd48268)
(cherry picked from commit fbd24ff359)
2018-02-25 09:21:00 -08:00
Ondřej Surý
d080192323 [rt46602] Put back original .gitignore files in the system tests
(cherry picked from commit caaec1ef6d)
(cherry picked from commit cf7a86661a)
(cherry picked from commit ff2356395b)
2018-02-25 09:21:00 -08:00
Ondřej Surý
043bd61301 [rt46602] Adjust rpzrecurse for dynamic port usage
Resolve conflict in run.sh during cherry-pick from an experimental branch.

(cherry picked from commit 2a435708e4)
(cherry picked from commit c5aa6ed87b)
(cherry picked from commit 9bdb4d67f1)
2018-02-25 09:20:58 -08:00
Ondřej Surý
0173209dda [rt46602] Add $SEDPORTS variable to help with @PORT@ and @CONTROLPORT@ replacement
(cherry picked from commit e6802badc0)
(cherry picked from commit d37a52cc56)
(cherry picked from commit 76848ca9d4)
2018-02-25 09:17:30 -08:00
Ondřej Surý
ffa6566643 [rt46602] Move most of ignored files to top of system tests directory
(cherry picked from commit d2c8286163)
(cherry picked from commit a5c3077a9c)
(cherry picked from commit f11975087c)
2018-02-25 09:17:27 -08:00
Ondřej Surý
0f99295e12 [rt46602] Fix output from rndc reload
(cherry picked from commit 53bade5ece)
(cherry picked from commit 112355adb8)
(cherry picked from commit 77e5c32baf)
2018-02-25 09:14:51 -08:00
Ondřej Surý
dce8925c61 [rt46602] Add .gitignore to allow_query
(cherry picked from commit bc276f6c05)
(cherry picked from commit 76dfb5c339)
(cherry picked from commit 51ccb16653)
2018-02-25 09:14:50 -08:00
Ondřej Surý
2d8cf8c8cc [rt46602] Utilize echo_i to allow mixed test output
(cherry picked from commit 7e3c96c867)
(cherry picked from commit 93034acb4d)
(cherry picked from commit 9bcf40cbed)
2018-02-25 09:14:50 -08:00
Ondřej Surý
585d362afe [rt46602] Prefix R:<result> as R:<test>:<result>
Resolve conflict in run.sh during cherry-pick from experimental branch.

(cherry picked from commit 278c72fef6)
(cherry picked from commit b8e866d60f)
(cherry picked from commit ff56ecafc4)
2018-02-25 09:14:50 -08:00
Ondřej Surý
9cff66b3b9 [rt46602] Add echo_i function to print I:<testname>:string
(cherry picked from commit f7ac41757a)
(cherry picked from commit bd5b36f7a4)
(cherry picked from commit cdc72937d6)
2018-02-25 09:14:47 -08:00
Ondřej Surý
5b45f67f61 [rt46602] Add serve-stale working files to .gitignore
(cherry picked from commit cba162e70e)
(cherry picked from commit 57adb949a4)
(cherry picked from commit 2327f41b35)
2018-02-25 09:14:30 -08:00
Ondřej Surý
10da6c8a87 [rt46602] Convert allow_query to dynamic port
(cherry picked from commit 992a2bafec)
(cherry picked from commit 1d10178e83)
(cherry picked from commit 1a568d5a76)
2018-02-25 09:14:30 -08:00