Use only if necessary, megatest will extract the location from where it used to launch and add append that to the PATH for test runs.
+
+
+
+
+
+runsdir
+
+
+full path to where the link tree to all runs will be created
+
+
+no
+
+
+Because your runs may be spread out over several disk partitions a central link tree is created to make finding all the runs easy.
+
+
+
[fields]
string of letters, numbers and underscore
string of letters, numbers and underscore
+
+
+at least one
@@ -456,10 +497,13 @@
launcher
command line used to launch jobs - the job command (megatest -execute) will be appended to this
+
+
+no
@@ -473,10 +517,13 @@
list of hostnames to run jobs on NOT SUPPORTED RIGHT NOW
+n/a
+
+
@@ -486,10 +533,13 @@
string of letters, numbers and underscore
any string
+
+
+no
These are set on the test launching machine, not the test running machine. Typical usage is to control the host or run queue for launching tests. These values will not be seen by the test when it runs.
@@ -501,18 +551,222 @@
string of letters, numbers and underscore
a valid path writable by the test launching process and by the test process
+
+
+yes
The disk usage balancing algorithm is to choose the disk with the least space for each test run.
+variables set in this section will be available for all runs, defining the same variable in another section will override the value from the default section
+
+
+
+
+
+[field1value/field2value...]
+
+
+string of letters, numbers and underscore
+
+
+any
+
+
+no
+
+
+the values in this section will be set for any run where field1 is field1value, field2 is field2value and fieldN is fieldNvalue.
+
+
+
+
+
+Example: a test suite that checks that a piece of software works correctly for different customer configurations and locations each of which is done as a separate release regression run. The fields, CUSTOMER and LOCATION were chosen. The following runconfigs.config file would set some variables specific to runs for megacorp in India and femtocorp in the Cook Islands and New Zealand:
+
+The script must be executable and either provide the full path or put a copy at the top of your test directory
+
+
+
+
+
+[requirements]
+
+
+waiton
+
+
+list of valid test names
+
+
+no
+
+
+This test will not run until the named tests are state completed and status PASS
+
+
+
+
+
+[items]
+
+
+any valid
+
+
+list of values
+
+
+no
+
+
+The test will be repeated once for each item with the variable name set to the value. If there is more than one variable then the test will be run against all unique combinations of the values
+
Copyright (C) 2011 Matt Welland
Index: docs/megatest.lyx
==================================================================
--- docs/megatest.lyx
+++ docs/megatest.lyx
@@ -1,10 +1,10 @@
#LyX 1.6.5 created this file. For more info see http://www.lyx.org/
\lyxformat 345
\begin_document
\begin_header
-\textclass docbook
+\textclass docbook-book
\use_default_options false
\language english
\inputencoding auto
\font_roman default
\font_sans default
@@ -15,10 +15,11 @@
\font_sf_scale 100
\font_tt_scale 100
\graphics default
\paperfontsize default
+\spacing single
\use_hyperref false
\papersize default
\use_geometry false
\use_amsmath 0
\use_esint 0
@@ -593,21 +594,22 @@
\end_layout
\begin_layout Subsection
-megatest.config settings
+Configuration files
\end_layout
\begin_layout Subsubsection
-[setup]
+megatest.config
\end_layout
\begin_layout Standard
\begin_inset Tabular
-
+
+
@@ -634,10 +636,19 @@
\begin_layout Plain Layout
value
\end_layout
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+required
+\end_layout
+
\end_inset
\begin_inset Text
@@ -672,10 +683,19 @@
\begin_layout Plain Layout
if variable is not defined no limit on jobs
\end_layout
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+no
+\end_layout
+
\end_inset
\begin_inset Text
@@ -710,10 +730,19 @@
\begin_layout Plain Layout
full path to megatest binary
\end_layout
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+no
+\end_layout
+
\end_inset
\begin_inset Text
@@ -720,10 +749,58 @@
\begin_layout Plain Layout
Use only if necessary, megatest will extract the location from where it
used to launch and add append that to the PATH for test runs.
\end_layout
+\end_inset
+
+
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+
+\end_layout
+
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+runsdir
+\end_layout
+
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+full path to where the link tree to all runs will be created
+\end_layout
+
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+no
+\end_layout
+
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+Because your runs may be spread out over several disk partitions a central
+ link tree is created to make finding all the runs easy.
+\end_layout
+
\end_inset
@@ -749,10 +826,19 @@
\begin_layout Plain Layout
string of letters, numbers and underscore
\end_layout
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+at least one
+\end_layout
+
\end_inset
\begin_inset Text
@@ -788,10 +874,19 @@
\begin_layout Plain Layout
command line used to launch jobs - the job command (megatest -execute) will
be appended to this
\end_layout
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+no
+\end_layout
+
\end_inset
\begin_inset Text
@@ -826,10 +921,19 @@
\begin_layout Plain Layout
list of hostnames to run jobs on NOT SUPPORTED RIGHT NOW
\end_layout
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+n/a
+\end_layout
+
\end_inset
\begin_inset Text
@@ -864,10 +968,19 @@
\begin_layout Plain Layout
any string
\end_layout
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+no
+\end_layout
+
\end_inset
\begin_inset Text
@@ -904,10 +1017,19 @@
\begin_layout Plain Layout
a valid path writable by the test launching process and by the test process
\end_layout
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+yes
+\end_layout
+
\end_inset
\begin_inset Text
@@ -915,10 +1037,470 @@
The disk usage balancing algorithm is to choose the disk with the least
space for each test run.
\end_layout
+\end_inset
+
+
+
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Subsubsection
+runconfigs.config file
+\end_layout
+
+\begin_layout Standard
+\begin_inset Tabular
+
+
+
+
+
+
+
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+section
+\end_layout
+
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+variable
+\end_layout
+
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+value
+\end_layout
+
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+required?
+\end_layout
+
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+comment
+\end_layout
+
+\end_inset
+
+
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+[default]
+\end_layout
+
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+string of letters, numbers and underscore
+\end_layout
+
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+any
+\end_layout
+
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+no
+\end_layout
+
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+variables set in this section will be available for all runs, defining the
+ same variable in another section will override the value from the default
+ section
+\end_layout
+
+\end_inset
+
+
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+[field1value/field2value...]
+\end_layout
+
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+string of letters, numbers and underscore
+\end_layout
+
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+any
+\end_layout
+
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+no
+\end_layout
+
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+the values in this section will be set for any run where field1 is field1value,
+ field2 is field2value and fieldN is fieldNvalue.
+
+\end_layout
+
+\end_inset
+
+
+
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Standard
+Example: a test suite that checks that a piece of software works correctly
+ for different customer configurations and locations each of which is done
+ as a separate release regression run.
+ The fields, CUSTOMER and LOCATION were chosen.
+ The following runconfigs.config file would set some variables specific to
+ runs for megacorp in India and femtocorp in the Cook Islands and New Zealand:
+\end_layout
+
+\begin_layout Code
+# runconfigs.config
+\end_layout
+
+\begin_layout Code
+[default]
+\end_layout
+
+\begin_layout Code
+ENCRYTION true
+\end_layout
+
+\begin_layout Code
+[megacorp/india]
+\end_layout
+
+\begin_layout Code
+TESTPATH /nfs/testing/megacorp_runs
+\end_layout
+
+\begin_layout Code
+[femtocorp/cook_islands]
+\end_layout
+
+\begin_layout Code
+ENCRYTION false
+\end_layout
+
+\begin_layout Code
+TESTPATH /afs/kiatoa/testing/cook_islands
+\end_layout
+
+\begin_layout Code
+[femtocorp/new_zealand]
+\end_layout
+
+\begin_layout Code
+TESTPATH /afs/kiatao/testing/new_zealand
+\end_layout
+
+\begin_layout Code
+[megacorp/new_zealand]
+\end_layout
+
+\begin_layout Code
+TESTPATH /nfs/testing/megacorp_runs
+\end_layout
+
+\begin_layout Paragraph*
+Running megatest like this:
+\end_layout
+
+\begin_layout Code
+megatest :CUSTOMER megacorp :LOCATION new_zealand :runname week12_2011_run1
+ -runall
+\end_layout
+
+\begin_layout Paragraph*
+Would set:
+\end_layout
+
+\begin_layout Code
+ENCRYPTION true
+\end_layout
+
+\begin_layout Code
+TESTPATH /nfs/testing/megacorp_runs
+\end_layout
+
+\begin_layout Subsection
+Writing tests
+\end_layout
+
+\begin_layout Subsubsection
+testconfig file
+\end_layout
+
+\begin_layout Standard
+\begin_inset Tabular
+
+
+
+
+
+
+
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+section
+\end_layout
+
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+variable
+\end_layout
+
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+value
+\end_layout
+
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+required?
+\end_layout
+
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+comments
+\end_layout
+
+\end_inset
+
+
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+[setup]
+\end_layout
+
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+runscript
+\end_layout
+
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+name of script to execute for this test
+\end_layout
+
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+yes
+\end_layout
+
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+The script must be executable and either provide the full path or put a
+ copy at the top of your test directory
+\end_layout
+
+\end_inset
+
+
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+[requirements]
+\end_layout
+
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+waiton
+\end_layout
+
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+list of valid test names
+\end_layout
+
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+no
+\end_layout
+
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+This test will not run until the named tests are state completed and status
+ PASS
+\end_layout
+
+\end_inset
+
+
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+[items]
+\end_layout
+
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+any valid
+\end_layout
+
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+list of values
+\end_layout
+
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+no
+\end_layout
+
+\end_inset
+
+
+\begin_inset Text
+
+\begin_layout Plain Layout
+The test will be repeated once for each item with the variable name set
+ to the value.
+ If there is more than one variable then the test will be run against all
+ unique combinations of the values
+\end_layout
+
\end_inset
Index: launch.scm
==================================================================
--- launch.scm
+++ launch.scm
@@ -79,11 +79,13 @@
"runname"))
(key-vals (get-key-vals db run-id))
(key-str (string-intersperse key-vals "/"))
(dfullp (conc disk-path "/" key-str "/" runname "/" testname
item-path))
- (lnkpath (conc *toppath* "/runs/" key-str "/" runname item-path)))
+ (runsdir (config-lookup *configdat* "setup" "runsdir"))
+ (lnkpath (conc (if runsdir runsdir (conc *toppath* "/runs"))
+ "/" key-str "/" runname item-path)))
(print "Setting up test run area")
(print " - creating run area in " dfullp)
(system (conc "mkdir -p " dfullp))
(print " - creating link from " dfullp "/" testname " to " lnkpath)
(system (conc "mkdir -p " lnkpath))
Index: tests/megatest.config
==================================================================
--- tests/megatest.config
+++ tests/megatest.config
@@ -2,11 +2,13 @@
sysname TEXT
fsname TEXT
datapath TEXT
[setup]
+# exectutable /path/to/megatest
max_concurrent_jobs 5
+runsdir /tmp/runs
[jobtools]
# ## launcher launches jobs, the job is managed on the target host
## by megatest, comment out launcher to run local
# workhosts localhost hermes
@@ -17,13 +19,13 @@
status pass fail n/a
# These are set before all tests, override them
# in the testconfig [pre-launch-env-overrides] section
[env-override]
-SPECIAL_ENV_VARS overide them here - all tests see these
+SPECIAL_ENV_VARS overide them here - seen only by the test launcher
## disks are:
## name host:/path/to/area
## -or-
## name /path/to/area
[disks]
1 /tmp
Index: tests/tests/sqlitespeed/testconfig
==================================================================
--- tests/tests/sqlitespeed/testconfig
+++ tests/tests/sqlitespeed/testconfig
@@ -1,14 +1,8 @@
[setup]
runscript runscript.rb
[requirements]
-diskspace 1M
-memory 1G
waiton runfirst
-[env-override]
-# Test specific environment overrides go here
-SPECIAL_ENV_VAR override with everything after the space.
-
[items]
MANYITEMS a b c d e f g h i j k l m