Index: docs/manual/megatest_manual.html ================================================================== --- docs/manual/megatest_manual.html +++ docs/manual/megatest_manual.html @@ -742,27 +742,10 @@ <matt@kiatoa.com>
version 1.0, April 2012
-
-

Dedication

-
-

Dedicated to my wife Joanna who has kindly supported my working on various projects over the years.

-
-

Thanks

-
-

Thank you the many people I’ve worked over the years who have -shared their knowledge and insights with me.

-

Thanks also to the creators of the various open source projects that -Megatest is built on. These include Linux, xemacs, chicken scheme, -fossil and asciidoc. Without these projects something like Megatest -would be difficult or impossible to do.

-
-
-
-

Preface

This book is organised as three sub-books; getting started, writing tests and reference.

@@ -897,30 +880,131 @@

The testconfig File

+
+

Setup section

+
+

Header

+
+
+
[setup]
+
+

The runscript method is a brute force way to run scripts where the +user is responsible for setting STATE and STATUS

+
+
+
runscript main.csh
+
+
+
+
+

Requirements section

+
+

Header

+
+
+
[requirements]
+
+
+
+

Wait on Other Tests

+
+
+
# A normal waiton waits for the prior tests to be COMPLETED
+# and PASS, CHECK or WAIVED
+waiton test1 test2
+
+
+
+

Mode

+

The default (i.e. if mode is not specified) is normal. All pre-dependent tests +must be COMPLETED and PASS, CHECK or WAIVED before the test will start

+
+
+
mode   normal
+
+

The toplevel mode requires only that the prior tests are COMPLETED.

+
+
+
mode toplevel
+
+

A item based waiton will start items in a test when the +same-named item is COMPLETED and PASS, CHECK or WAIVED +in the prior test

+
+
+
mode itemmatch
+
-
# testconfig
-[requirements]
-
-# A normal waiton waits for the prior tests to be COMPLETED
-# and PASS, CHECK or WAIVED
-waiton test1 test2
-
-# A item based waiton will start items in a test when the
-# same-named item is COMPLETED and PASS, CHECK or WAIVED
-# in the prior test
-#
-mode   normal | toplevel
-
-# With a toplevel test you may wish to generate your list
+
# With a toplevel test you may wish to generate your list
 # of tests to run dynamically
 #
 # waiton #{shell get-valid-tests-to-run.sh}
+
+
+

Run time limit

+
+
+
runtimelim 1h 2m 3s  # this will automatically kill the test if it runs for more than 1h 2m and 3s
+
+
+
+

Skip

+
+
+

Header

+
+
+
[skip]
+
+
+
+

Skip on Still-running Tests

+
+
+
# NB// If the prevrunning line exists with *any* value the test will
+# automatically SKIP if the same-named test is currently RUNNING
+
+prevrunning x
+
+
+
+

Skip if a File Exists

+
+
+
fileexists /path/to/a/file # skip if /path/to/a/file exists
+
+
+
+

Controlled waiver propagation

+

If test is FAIL and previous test in run with same MT_TARGET is WAIVED then apply the following rules from the testconfig: +If a waiver check is specified in the testconfig apply the check and if it passes then set this FAIL to WAIVED

+

Waiver check has two parts, 1) a list of waiver, rulename, filepatterns and 2) the rulename script spec (note that "diff" and "logpro" are predefined)

+
+
+
###### EXAMPLE FROM testconfig #########
+# matching file(s) will be diff'd with previous run and logpro applied
+# if PASS or WARN result from logpro then WAIVER state is set
+#
+[waivers]
+# logpro_file    rulename      input_glob
+waiver_1         logpro        lookittmp.log
+
+[waiver_rules]
+
+# This builtin rule is the default if there is no <waivername>.logpro file
+# diff   diff %file1% %file2%
+
+# This builtin rule is applied if a <waivername>.logpro file exists
+# logpro diff %file1% %file2% | logpro %waivername%.logpro %waivername%.html
+
+
+

Appendix A: Example Appendix

@@ -1000,10 +1084,10 @@
Index: docs/manual/megatest_manual.txt ================================================================== --- docs/manual/megatest_manual.txt +++ docs/manual/megatest_manual.txt @@ -44,38 +44,10 @@ megatest uses the network filesystem to call home to your master sqlite3 database. include::getting_started.txt[] include::writing_tests.txt[] include::reference.txt[] - - -Controlled waiver propagation -============================= -If test is FAIL and previous test in run with same MT_TARGET is WAIVED then apply the following rules from the testconfig: -If a waiver check is specified in the testconfig apply the check and if it passes then set this FAIL to WAIVED - -Waiver check has two parts, 1) a list of waiver, rulename, filepatterns and 2) the rulename script spec (note that "diff" and "logpro" are predefined) - -========================================================= -###### EXAMPLE FROM testconfig ######### -# matching file(s) will be diff'd with previous run and logpro applied -# if PASS or WARN result from logpro then WAIVER state is set -# -[waivers] -# logpro_file rulename input_glob -waiver_1 logpro lookittmp.log - -[waiver_rules] - -# This builtin rule is the default if there is no .logpro file -# diff diff %file1% %file2% - -# This builtin rule is applied if a .logpro file exists -# logpro diff %file1% %file2% | logpro %waivername%.logpro %waivername%.html -========================================================= - - [appendix] Example Appendix ================ One or more optional appendixes go here at section level zero. Index: docs/manual/reference.txt ================================================================== --- docs/manual/reference.txt +++ docs/manual/reference.txt @@ -8,27 +8,138 @@ sub-sections. The testconfig File ------------------- +Setup section +~~~~~~~~~~~~~ + +Header +^^^^^^ + +------------------- +[setup] +------------------- + +The runscript method is a brute force way to run scripts where the +user is responsible for setting STATE and STATUS + +------------------- +runscript main.csh ------------------- -# testconfig + +Requirements section +~~~~~~~~~~~~~~~~~~~~ + +Header +^^^^^^ + +------------------- [requirements] +------------------- + +Wait on Other Tests +^^^^^^^^^^^^^^^^^^^ +------------------- # A normal waiton waits for the prior tests to be COMPLETED # and PASS, CHECK or WAIVED waiton test1 test2 +------------------- + +Mode +^^^^ + +The default (i.e. if mode is not specified) is normal. All pre-dependent tests +must be COMPLETED and PASS, CHECK or WAIVED before the test will start + +------------------- +mode normal +------------------- + +The toplevel mode requires only that the prior tests are COMPLETED. + +------------------- +mode toplevel +------------------- + +A item based waiton will start items in a test when the +same-named item is COMPLETED and PASS, CHECK or WAIVED +in the prior test + +------------------- +mode itemmatch +------------------- -# A item based waiton will start items in a test when the -# same-named item is COMPLETED and PASS, CHECK or WAIVED -# in the prior test -# -mode normal | toplevel +------------------- # With a toplevel test you may wish to generate your list # of tests to run dynamically # # waiton #{shell get-valid-tests-to-run.sh} ------------------ + + + +Run time limit +^^^^^^^^^^^^^^ + +----------------- +runtimelim 1h 2m 3s # this will automatically kill the test if it runs for more than 1h 2m and 3s +----------------- + +Skip +^^^^ + +Header +^^^^^^ + +----------------- +[skip] +----------------- + +Skip on Still-running Tests +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +----------------- +# NB// If the prevrunning line exists with *any* value the test will +# automatically SKIP if the same-named test is currently RUNNING + +prevrunning x +----------------- + +Skip if a File Exists +^^^^^^^^^^^^^^^^^^^^^ + +----------------- +fileexists /path/to/a/file # skip if /path/to/a/file exists +----------------- + +Controlled waiver propagation +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +If test is FAIL and previous test in run with same MT_TARGET is WAIVED then apply the following rules from the testconfig: +If a waiver check is specified in the testconfig apply the check and if it passes then set this FAIL to WAIVED + +Waiver check has two parts, 1) a list of waiver, rulename, filepatterns and 2) the rulename script spec (note that "diff" and "logpro" are predefined) + +----------------- +###### EXAMPLE FROM testconfig ######### +# matching file(s) will be diff'd with previous run and logpro applied +# if PASS or WARN result from logpro then WAIVER state is set +# +[waivers] +# logpro_file rulename input_glob +waiver_1 logpro lookittmp.log + +[waiver_rules] + +# This builtin rule is the default if there is no .logpro file +# diff diff %file1% %file2% + +# This builtin rule is applied if a .logpro file exists +# logpro diff %file1% %file2% | logpro %waivername%.logpro %waivername%.html +----------------- + :numbered!: