Megatest

reference.txt at [efc02c437a]
Login

File docs/manual/reference.txt artifact bbd73cd577 part of check-in efc02c437a



Reference
=========

Config File Settings
--------------------

Trim trailing spaces
~~~~~~~~~~~~~~~~~~~~

------------------
[configf:settings trim-trailing-spaces yes]
------------------

The testconfig File
-------------------

Setup section
~~~~~~~~~~~~~

Header
^^^^^^

-------------------
[setup]
-------------------

The runscript method is a brute force way to run scripts where the
user is responsible for setting STATE and STATUS

-------------------
runscript main.csh
-------------------

Requirements section
~~~~~~~~~~~~~~~~~~~~

Header
^^^^^^

-------------------
[requirements]
-------------------

Wait on Other Tests
^^^^^^^^^^^^^^^^^^^

-------------------
# A normal waiton waits for the prior tests to be COMPLETED 
# and PASS, CHECK or WAIVED
waiton test1 test2
-------------------

Mode
^^^^

The default (i.e. if mode is not specified) is normal. All pre-dependent tests
must be COMPLETED and PASS, CHECK or WAIVED before the test will start

-------------------
[requirements]
mode   normal
-------------------

The toplevel mode requires only that the prior tests are COMPLETED.

-------------------
[requirements]
mode toplevel
-------------------

A item based waiton will start items in a test when the same-named
item is COMPLETED and PASS, CHECK or WAIVED in the prior test. This
was historically called "itemwait" mode. The terms "itemwait" and
"itemmatch" are synonyms.

-------------------
[requirements]
mode itemmatch
-------------------

Itemmap
^^^^^^^

For cases were the dependent test has a similar but not identical
itempath to the downstream test an itemmap can allow for itemmatch
mode

-------------------
[requirements]
mode itemmatch
itemmap .*x/ y/

# ## pattern replacement notes
#
# ## Example
# ## Remove everything up to the last /
itemmap .*/
#
# ## Example
# ## Replace foo/ with bar/
itemmap foo/ bar/
-------------------

.Autogeneration waiton list for dynamic flow dependency trees
-------------------
[requirements]
# With a toplevel test you may wish to generate your list 
# of tests to run dynamically
#
# waiton #{shell get-valid-tests-to-run.sh}
-------------------

Run time limit
^^^^^^^^^^^^^^

-----------------
runtimelim 1h 2m 3s  # this will automatically kill the test if it runs for more than 1h 2m and 3s
-----------------

Skip
^^^^

A test with a skip section will conditional skip running.

.Skip section example
-----------------
[skip]
prevrunning x
# rundelay 30m 15s
-----------------

Skip on Still-running Tests
^^^^^^^^^^^^^^^^^^^^^^^^^^^

-----------------
# NB// If the prevrunning line exists with *any* value the test will
# automatically SKIP if the same-named test is currently RUNNING. The
# "x" can be any string. Comment out the prevrunning line to turn off
# skip.

[skip]
prevrunning x
-----------------

Skip if a File Exists
^^^^^^^^^^^^^^^^^^^^^

-----------------
[skip]
fileexists /path/to/a/file # skip if /path/to/a/file exists
-----------------

Skip if test ran more recently than specified time
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.Skip if this test has been run in the past fifteen minutes and 15 seconds.
-----------------
[skip]
rundelay 15m 15s
-----------------

Controlled waiver propagation
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

If test is FAIL and previous test in run with same MT_TARGET is WAIVED then apply the following rules from the testconfig:
If a waiver check is specified in the testconfig apply the check and if it passes then set this FAIL to WAIVED

Waiver check has two parts, 1) a list of waiver, rulename, filepatterns and 2) the rulename script spec (note that "diff" and "logpro" are predefined)

-----------------
###### EXAMPLE FROM testconfig #########
# matching file(s) will be diff'd with previous run and logpro applied
# if PASS or WARN result from logpro then WAIVER state is set
#
[waivers]
# logpro_file    rulename      input_glob
waiver_1         logpro        lookittmp.log

[waiver_rules]

# This builtin rule is the default if there is no <waivername>.logpro file
# diff   diff %file1% %file2%

# This builtin rule is applied if a <waivername>.logpro file exists
# logpro diff %file1% %file2% | logpro %waivername%.logpro %waivername%.html
-----------------

Ezsteps
~~~~~~~

To transfer the environment to the next step you can do the following:

----------------------------
$MT_MEGATEST -env2file .ezsteps/${stepname}	  
----------------------------

Triggers
~~~~~~~~

In your testconfig triggers can be specified 

-----------------
[triggers]

# Call script running.sh when test goes to state=RUNNING, status=PASS
RUNNING/PASS running.sh

# Call script running.sh any time state goes to RUNNING
RUNNING/ running.sh

# Call script onpass.sh any time status goes to PASS
PASS/ onpass.sh
-----------------

Scripts called will have; test-id test-rundir trigger, added to the commandline.

HINT

To start an xterm (useful for debugging), use a command line like the following:

-----------------
[triggers]
COMPLETED/ xterm -e bash -s -- 
-----------------

NOTE: There is a trailing space after the --


Override the Toplevel HTML File
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Megatest generates a simple html file summary for top level tests of
iterated tests. The generation can be overridden. NOTE: the output of
the script is captured from stdout to create the html.


.For test "runfirst" override the toplevel generation with a script "mysummary.sh"
-----------------
# Override the rollup for specific tests
[testrollup]
runfirst mysummary.sh
-----------------

Programming API
---------------

These routines can be called from the megatest repl. 

.API Server Management Calls
[width="70%",cols="^,2m,2m,2m",frame="topbot",options="header,footer"]
|======================
|API Call                  | Purpose comments   | Returns                 | Comments
|(rmt:start-server run-id) |                    | #( success/fail   n/a ) | 
|(rmt:kill-server  run-id) |                    | #( success/fail   n/a ) | Works only if the server is still reachable
|(rmt:login        run-id) | Verify the the version, testsuite area etc. are correct. | #( #t "successful login" ) |
|======================

.API Keys Related Calls
[width="70%",cols="^,2m,2m,2m",frame="topbot",options="header,footer"]
|======================
|API Call                        | Purpose comments   | Returns                 | Comments
|(rmt:get-keys run-id)           |                    | ( key1 key2 ... )       | 
| (rmt:get-key-val-pairs run-id) |                    | #t=success/#f=fail      | Works only if the server is still reachable
|======================


:numbered!: