Megatest

-------------------------
[a/2/b]
VAR1 VAL2
-------------------------

Can use either wildcard of "%" or a regular expression:

-------------------------
[/abc.*def/]
-------------------------

Disk Space Checks
^^^^^^^^^^^^^^^^^

Some parameters you can put in the [setup] section of megatest.config:

-------------------

Attempt to rerun tests in "STUCK/DEAD", "n/a", "ZERO_ITEMS" states.

.In megatest.config
------------------
[setup]
reruns 5
------------------

Replace the default blacklisted environment variables with user supplied
list.

Default list: USER HOME DISPLAY LS_COLORS XKEYSYMDB EDITOR MAKEFLAGS MAKEF MAKEOVERRIDES

.Add a "bad" variable "PROMPT" to the variables that will be commented out
in the megatest.sh and megatest.csh files:
-----------------
[setup]
blacklistvars USER HOME DISPLAY LS_COLORS XKEYSYMDB EDITOR MAKEFLAGS PROMPT
-----------------

Run time limit
++++++++++++++

-----------------
[setup]
# this will automatically kill the test if it runs for more than 1h 2m and 3s

Itemmap Handling
~~~~~~~~~~~~~~~~

For cases were the dependent test has a similar but not identical
itempath to the downstream test an itemmap can allow for itemmatch
mode

.example for removing part of itemmap for waiton test (eg: item +foo-x/bar+ depends on waiton's item +y/bar+)
-------------------
[requirements]
mode itemwait
# itemmap <item pattern for this test>  <item replacement pattern for waiton test>
itemmap .*x/ y/

-------------------

.example for removing part of itemmap for waiton test (eg: item +foo/bar/baz+ in this test depends on waiton's item +baz+)
-------------------

# ## pattern replacement notes
#
# ## Example
# ## Remove everything up to the last /
[requirements]
mode itemwait
# itemmap <item pattern for this test> <nothing here indicates removal>
itemmap .*/
-------------------

.example replacing part of itemmap for (eg: item +foo/1234+ will imply waiton's item +bar/1234+)
-------------------

#
# ## Example
# ## Replace foo/ with bar/
[requirements]
mode itemwait
# itemmap <item pattern for this test>  <item replacement pattern for waiton test>
itemmap foo/ bar/

-------------------

.example for backreference (eg: item +foo23/thud+ will imply waiton's item +num-23/bar/thud+
-------------------
#
# ## Example
# ## can use \{number} in replacement pattern to backreference a (capture) from matching pattern similar to sed or perl
[requirements]
mode itemwait
# itemmap <item pattern for this test>  <item replacement pattern for waiton test>
itemmap foo(\d+)/ num-\1/bar/

-------------------

.example multiple itemmaps
-------------------

# multi-line; matches are applied in the listed order
# The following would map:
#   a123b321 to b321fooa123 then to 321fooa123p
#
[requirements]
itemmap (a\d+)(b\d+) \2foo\1
  b(.*) \1p
-------------------


Complex mapping
^^^^^^^^^^^^^^^
Complex mappings can be handled with a separate [itemmap] section (instead if an itemmap line in the [requirements] section)

Each line in an itemmap section starts with a waiton test name followed by an itemmap expression

.eg: The following causes waiton test A item +bar/1234+ to run when our test's +foo/1234+ item is requested as well as causing waiton test B's +blah+ item to run when our test's +stuff/blah+ item is requested
--------------
[itemmap]
A foo/ bar/
B stuff/
--------------


Complex mapping example
^^^^^^^^^^^^^^^^^^^^^^^



// image::itemmap.png[]
image::complex-itemmap.png[]








We accomplish this by configuring the testconfigs of our tests C D and E as follows:

.Testconfig for Test E has
----------------------
[requirements]
waiton C

itemmap (\d+)/res \1/bb


----------------------

.Testconfig for Test D has
----------------------
[requirements]
waiton C
itemmap (\d+)/res \1/aa
----------------------

.Testconfig for Test C has
----------------------
[requirements]
waiton A B

[itemmap]
A (\d+)/aa aa/\1
B (\d+)/bb bb/\1
----------------------

.Testconfigs for Test B and Test A have no waiton or itemmap configured
-------------------
-------------------

.Walk through one item -- we want the following to happen for testpatt +D/1/res+ (see blue boxes in complex itemmaping figure above):

. eg from command line +megatest -run -testpatt D/1/res -target mytarget -runname myrunname+
. Full list to be run is now: +D/1/res+
. Test D has a waiton - test C.  Test D's itemmap rule +itemmap (\d&plus;)/res \1/aa+ ->  causes +C/1/aa+ to run before +D/1/res+
. Full list to be run is now: +D/1/res+, +C/1/aa+
. Test C was a waiton - test A. Test C's rule +A (\d&plus;)/aa aa/\1+ -> causes +A/aa/1+ to run before +C/1/aa+
. Full list to be run is now: +D/1/res+, +C/1/aa+, +A/aa/1+
. Test A has no waitons.  All waitons of all tests in full list have been processed.  Full list is finalized.


Dynamic Flow Dependency Tree
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.Autogeneration waiton list for dynamic flow dependency trees
-------------------
[requirements]

----------------------------
$MT_MEGATEST -env2file .ezsteps/${stepname}	  
----------------------------

Triggers
~~~~~~~~

In your testconfig or megatest.config triggers can be specified 

-----------------
[triggers]

# Call script running.sh when test goes to state=RUNNING, status=PASS
RUNNING/PASS running.sh

# Call script running.sh any time state goes to RUNNING
RUNNING/ running.sh

# Call script onpass.sh any time status goes to PASS
PASS/ onpass.sh
-----------------

Scripts called will have; test-id test-rundir trigger test-name item-path state status event-time, added to the commandline.

HINT

To start an xterm (useful for debugging), use a command line like the following:

-----------------
[triggers]
COMPLETED/ xterm -e bash -s -- 
-----------------

NOTE: There is a trailing space after the --

There are a number of environment variables available to the trigger script
but since triggers can be called in various contexts not all variables are
available at all times. The trigger script should check for the variable and
fail gracefully if it doesn't exist.

.Environment variables visible to the trigger script
[width="90%",cols="^,2m",frame="topbot",options="header"]
|======================
|Variable             | Purpose            
| MT_TEST_RUN_DIR     | The directory where Megatest ran this test                   
| MT_CMDINFO          | Encoded command data for the test                   
| MT_DEBUG_MODE       | Used to pass the debug mode to nested calls to Megatest                   
| MT_RUN_AREA_HOME    | Megatest home area 
| MT_TESTSUITENAME    | The name of this testsuite or area                   
| MT_TEST_NAME        | The name of this test
| MT_ITEM_INFO        | The variable and values for the test item
| MT_MEGATEST         | Which Megatest binary is being used by this area
| MT_TARGET           | The target variable values, separated by '/'
| MT_LINKTREE         | The base of the link tree where all run tests can be found
| MT_ITEMPATH         | The values of the item path variables, separated by '/'
| MT_RUNNAME          | The name of the run
|======================


Override the Toplevel HTML File
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Megatest generates a simple html file summary for top level tests of
iterated tests. The generation can be overridden. NOTE: the output of

------------------------------

Then in runconfigs.config

.Example of using modified.config in a testconfig
------------------------------
cat testconfig

[pre-launch-env-vars]
[include modified.config]
------------------------------

Managing Old Runs
-----------------

It is often desired to keep some older runs around but this must be balanced with the costs of disk space.

. Use -remove-keep
. Use -archive (can also be done from the -remove-keep interface)
. use -remove-runs with -keep-records

.For each target, remove all runs but the most recent 3 if they are over 1 week old
---------------------
# use -precmd 'sleep 5;nbfake' to limit overloading the host computer but to allow the removes to run in parallel.
megatest -actions print,remove-runs -remove-keep 3 -target %/%/%/% -runname % -age 1w -precmd 'sleep 5;nbfake'"
---------------------

Nested Runs
-----------

A Megatest test can run a full Megatest run in either the same
Megatest area or in another area. This is a powerful way of chaining
complex suites of tests and or actions.

If you are not using the current area you can use ezsteps to retrieve
and setup the sub-Megatest run area.

In the testconfig:
---------------
[subrun]

# Required: wait for the run or just launch it
#           if no then the run will be an automatic PASS irrespective of the actual result
runwait yes|no

# Optional: where to execute the run. Default is the current runarea
runarea /some/path/to/megatest/area

# Optional: method to use to determine pass/fail status of the run
#   auto (default) - roll up the net state/status of the sub-run
#   logpro         - use the provided logpro rules, happens automatically if there is a logpro section
# passfail auto|logpro
# Example of logpro:
passfail logpro

# Optional: 
logpro ;; if this section exists then logpro is used to determine pass/fail
  (expect:required in "LogFileBody" >= 1 "At least one pass" #/PASS/)
  (expect:fail     in "LogFileBody"  = 0 "No FAILs allowed"  #/FAIL/)

# Optional: target translator, default is to use the parent target
target #{shell somescript.sh}

# Optional: runname translator/generator, default is to use the parent runname
runname #{somescript.sh}

# Optional: testpatt spec, default is to first look for TESTPATT spec from runconfigs unless there is a contour spec
testpatt %/item1,test2

# Optional: contour spec, use the named contour from the megatest.config contour spec
contour contourname ### NOTE: Not implemented yet! Let us know if you need this feature.

# Optional: mode-patt, use this spec for testpatt from runconfigs
mode-patt TESTPATT

# Optional: tag-expr, use this tag-expr to select tests
tag-expr quick

# Optional: (not yet implemented), propagate these actions from the parent
#           test
#   Note// default is % for all
propagate remove-runs archive ...

---------------

Programming API
---------------

These routines can be called from the megatest repl. 

.API Keys Related Calls