Preface
-This book is organised as three sub-books; getting started, writing tests and reference.
Why Megatest?
-The Megatest project was started for two reasons, the first was an -immediate and pressing need for a generalized tool to manage a suite -of regression tests and the second was the fact that the author had -written or maintained several such tools at different companies over -the years and it seemed a good thing to have a single open source -tool, flexible enough to meet the needs of any team doing continuous -integrating and or running a complex suite of tests for release -qualification.
Megatest Design Philosophy
-Megatest is intended to provide the minimum needed resources to make -writing a suite of tests and tasks for implementing continuous build -for software, design engineering or process control (via owlfs for -example) without being specialized for any specific problem -space. Megatest in of itself does not know what constitutes a PASS or -FAIL of a test. In most cases megatest is best used in conjunction -with logpro or a similar tool to parse, analyze and decide on the test -outcome.
Megatest Architecture
-All data to specify the tests and configure the system is stored in -plain text files. All system state is stored in an sqlite3 -database. Tests are launched using the launching system available for -the distributed compute platform in use. A template script is provided -which can launch jobs on local and remote Linux hosts. Currently -megatest uses the network filesystem to call home to your master -sqlite3 database.
Road Map
-Note 1: This road-map is tentative and subject to change without notice.
Note 2: Starting over. Old plan is commented out.
Current Items
-ww05 - migrate to inmem-db
-Keep as much the same as possible. Add internal reference to almost -eliminate contention on db(s).
-
-
-
-
-Add internal reference db -
-
- -
-
-Verify that actions are accessing correct db -
---
-
-
-
--runtests - inmem -
-
- -
-
--list-runs - local (but not megatest.db) -
-
- -
-
-dashboard - local (but not megatest.db) -
-
-
- -
-
-
-
-Mirror db to /var/tmp… -
-
- -
-
-Dashboard read db from per-run db. -
-
- -
-
-Dashboard read db from /var/tmp -
-
- -
-
-Runs register in tasks table in monitor.db -
-
- -
-
-Server polls tasks table for next action (in addition?) -
-
- -
-
-Change run loop to execute in server, triggered by call to polling of tasks table -
-
-
Getting Started
-How to install Megatest and set it up for running your regressions and continuous integration process.
Installation
-Dependencies
-Chicken scheme and a number of "eggs" are required for building -Megatest. See the script installall.sch in the utils directory of the -distribution for a mostly automated way to install everything needed -for building Megatest on Linux.
[An example footnote.]
And now for something completely different: monkeys, lions and -tigers (Bengal and Siberian) using the alternative syntax index -entries. - - - -Note that multi-entry terms generate separate index entries.
Here are a couple of image examples: an
-
-
-example inline image followed by an example block image:
-Followed by an example table:
| Option | -Description | -
|---|---|
-a USER GROUP |
-Add USER to GROUP. |
-
-R GROUP |
-Disables access to GROUP. |
-
Lorum ipum…
Sub-section with Anchor
-Sub-section at level 2.
Chapter Sub-section
-Sub-section at level 3.
Chapter Sub-section
-Sub-section at level 4.
This is the maximum sub-section depth supported by the distributed
-AsciiDoc configuration.
-
[A second example footnote.]
The Second Chapter
-An example link to anchor at start of the first sub-section.
An example link to a bibliography entry [taoup].
Writing Tests
-The First Chapter of the Second Part
-Chapters grouped into book parts are at level 1 and can contain -sub-sections.
How To Do Things
-Tricks
-This section is a compendium of a various useful tricks for debugging, -configuring and generally getting the most out of Megatest.
Limiting your running jobs
-The following example will limit a test in the jobgroup "group1" to no more than 10 tests simultaneously.
In your testconfig:
[test_meta] -jobgroup group1-
In your megatest.config:
[jobgroups] -group1 10 -custdes 4-
Debugging Tricks
-Examining The Environment
-During Config File Processing
-Organising Your Tests and Tasks
-[tests-paths]
-1 #{get misc parent}/simplerun/tests
-[setup]-
The runscript method is a brute force way to run scripts where the -user is responsible for setting STATE and STATUS
runscript main.csh-
Debugging Server Problems
-sudo lsof -i -sudo netstat -lptu -sudo netstat -tulpn-
Reference
-The First Chapter of the Second Part
-Chapters grouped into book parts are at level 1 and can contain -sub-sections.
The testconfig File
-Setup section
-Header
-[setup]-
The runscript method is a brute force way to run scripts where the -user is responsible for setting STATE and STATUS
runscript main.csh-
Requirements section
-Header
-[requirements]-
Wait on Other Tests
-# A normal waiton waits for the prior tests to be COMPLETED -# and PASS, CHECK or WAIVED -waiton test1 test2-
Mode
-The default (i.e. if mode is not specified) is normal. All pre-dependent tests -must be COMPLETED and PASS, CHECK or WAIVED before the test will start
mode normal-
The toplevel mode requires only that the prior tests are COMPLETED.
mode toplevel-
A item based waiton will start items in a test when the -same-named item is COMPLETED and PASS, CHECK or WAIVED -in the prior test
mode itemmatch-
# With a toplevel test you may wish to generate your list
-# of tests to run dynamically
-#
-# waiton #{shell get-valid-tests-to-run.sh}
-Run time limit
-runtimelim 1h 2m 3s # this will automatically kill the test if it runs for more than 1h 2m and 3s-
Skip
-Header
-[skip]-
Skip on Still-running Tests
-# NB// If the prevrunning line exists with *any* value the test will -# automatically SKIP if the same-named test is currently RUNNING - -prevrunning x-
Skip if a File Exists
-fileexists /path/to/a/file # skip if /path/to/a/file exists-
Controlled waiver propagation
-If test is FAIL and previous test in run with same MT_TARGET is WAIVED then apply the following rules from the testconfig: -If a waiver check is specified in the testconfig apply the check and if it passes then set this FAIL to WAIVED
Waiver check has two parts, 1) a list of waiver, rulename, filepatterns and 2) the rulename script spec (note that "diff" and "logpro" are predefined)
###### EXAMPLE FROM testconfig ######### -# matching file(s) will be diff'd with previous run and logpro applied -# if PASS or WARN result from logpro then WAIVER state is set -# -[waivers] -# logpro_file rulename input_glob -waiver_1 logpro lookittmp.log - -[waiver_rules] - -# This builtin rule is the default if there is no <waivername>.logpro file -# diff diff %file1% %file2% - -# This builtin rule is applied if a <waivername>.logpro file exists -# logpro diff %file1% %file2% | logpro %waivername%.logpro %waivername%.html-
Ezsteps
-To transfer the environment to the next step you can do the following:
$MT_MEGATEST -env2file .ezsteps/${stepname}
-Triggers
-In your testconfig triggers can be specified
[triggers] - -# Call script running.sh when test goes to state=RUNNING, status=PASS -RUNNING/PASS running.sh - -# Call script running.sh any time state goes to RUNNING -RUNNING/ running.sh - -# Call script onpass.sh any time status goes to PASS -PASS/ onpass.sh-
Megatest Internals
-
-Appendix A: Example Appendix
-One or more optional appendixes go here at section level zero.
Appendix Sub-section
-|
- Note
- |
-Preface and appendix subsections start out of sequence at level -2 (level 1 is skipped). This only applies to multi-part book -documents. | -
Example Bibliography
-The bibliography list is a style of AsciiDoc bulleted list.
Example Glossary
-Glossaries are optional. Glossaries entries are an example of a style -of AsciiDoc labeled lists.
-
-
- -A glossary term - -
-
-
- The corresponding (indented) definition. -
-
- - -A second glossary term - -
-
-
- The corresponding (indented) definition. -
-
-
Example Colophon
-Text at the end of a book describing facts about its production.