+
+
+
+
+
+
+
+
-Megatest +
--5/5/2011 -
-Matt Welland +Matthew Welland +
++Sept. 20,
-
-
-
+
+
+
+
+
+
+
+Megatest is free software released under the General Public License v2.0. Please see the file COPYING in the source distribution for details.
+
+
+
+
+
+
+Email: matt@kiatoa.com.
+
+Web: www.kiatoa.com/fossils/megatest
+
+
+
+
+
+
+This document is believed to be acurate at the time of writing but as with any opensource project the source code itself is the final arbiter of the softwares behaviour. It is the responsibility of the end user to validate that the code will perform as they expect. The author assumes no responsibility for any inaccuracies that this document may contain. In no event will Matthew Welland be liable for direct, indirect, special, exemplary, incidental, or consequential damages resulting from any defect or omission in this document, even if advised of the possibility of such damages.
+
+This document is a snapshot in time and the Megatest software has likely been changed since publication. This document and the product that it describes may be improved at any time, without notice or obligation.
+
+
+
+
+Document Revision History +
+
+Notable revisions of the software are occasionally documented here.
+
+
+
+
+
+
+
+
| +Version + | ++Author + | ++Description + | ++Date + | + +
| +v1.25 + | ++matt + | ++converted to new document format + | +
+
+
+
+
+ |
+
+
Table of Contents
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
--1 About Megatest -
--1.1 Megatest design philosophy -
-
-Megatest is intended to provide the minimum needed resources to make writing a suite of tests for software, design engineering or process control (via owlfs for example) without being specialized for any specific problem space. Megatest in of itself does not know what constitutes a PASS or FAIL of a test. In most cases megatest is best used in conjunction with logpro or a similar tool to parse, analyze and decide on the test outcome. A call to megatest can then be made to record the result.
-
--1.2 Megatest architecture -
-
-All data to specify the tests and configure the system is stored in plain text files. All system state is stored in an sqlite3 database. Tests are launched using the launching system available for the distributed compute platform in use. A template script is provided which can launch jobs on local and remote Linux hosts. Currently megatest uses the network filesystem to “call home” to your master sqlite3 database.
-
--2 Installation -
--2.1 Compilation -
--2.1.1 Dependencies -
-
-Chicken scheme and a number of eggs are required for building megatest. See the file utils/installall.sh for an automated way to install the dependencies on Linux.
-
--2.1.2 Build and install -
-
-Run “make test” to create the megatest executable. You may wish to copy the executable to a centrally accessible location.
-
--2.2 Setup -
--2.2.1 Create megatest.config -
-
-Create the file megatest.config using the megatest.config template from the tests directory. At a minimum you need the following:
-
-
+
+
+
+
+
+1 Introduction +
++1.1 Megatest design philosophy +
+
+Megatest is intended to provide the minimum needed resources to make writing a suite of tests for software, design engineering or process control (via owlfs for example) without being specialized for any specific problem space. Megatest in of itself does not know what constitutes a PASS or FAIL of a test. In most cases megatest is best used in conjunction with logpro or a similar tool to parse, analyze and decide on the test outcome. A call to megatest can then be made to record the result.
+
++1.2 Megatest architecture +
+
+All data to specify the tests and configure the system is stored in plain text files. All system state is stored in an sqlite3 database. Tests are launched using the launching system available for the distributed compute platform in use. A template script is provided which can launch jobs on local and remote Linux hosts. Currently megatest uses the network filesystem to “call home” to your master sqlite3 database.
+
++2 Installation +
++2.1 Dependencies +
+
+Chicken scheme and a number of “eggs” are required for building megatest. See the file utils/installall.sh for an automated way to install the dependencies on Linux.
+
++2.2 Build and Install +
+
+Run “make test” to create the megatest executable. You may wish to copy the executable to a centrally accessible location.
+
++3 Setup +
++3.1 Create megatest.config +
+
+Create the file megatest.config using the megatest.config template from the tests directory. At a minimum you need the following:
+
+
+
-# Fields are the keys under which your test runs are organized [fields] field1 TEXT field2 TEXT @@ -245,29 +329,28 @@ disk1 /tmp
-2.2.2 Create runconfigs.config -
-
-This file is used to set environment variables that are run specific. You can simply create an empty file to start.
+
+3.2 Create runconfigs.config +
+
+This file is used to set environment variables that are run specific. You can simply create an empty file to start.
+
+
+
-# runconfigs.config -
-2.2.3 Create the tests directory and your first test -
-
-The structure should look like this:
-
-
-
+
+3.3 Create the tests directory and your first test +
+
+
-../tests
├── megatest.config
├── runconfigs.config
└── tests
└── mytest
@@ -275,40 +358,39 @@
└── testconfig
-2.2.4 Create the testconfig file for your test -
-
-
+
+3.4 Create the testconfig file for your test +
+
+
-[setup] runscript main.sh
-2.2.5 Create your test running script, main.sh -
-
-
+
+3.5 Create your test running script, main.sh +
+
+
-#!/bin/bash $MT_MEGATEST -runstep mystep1 "sleep 20;echo Done" -m "mystep1 is done" - $MT_MEGATEST -test-status :state COMPLETED :status PASS -m "This is a comment"
-2.2.6 Run megatest and watch your run progress -
-
-
+
-3 How to Write Tests
+
+3.6 Run megatest and watch your run progress +
+
+
-megatest :field1 abc :field2 def :runname 2011week08.4a -runall watch megatest -list-runs % # OR use the dashboard @@ -316,51 +398,45 @@ dashboard &
-
-
-
-3 How to Write Tests
+
+4 How to Write Tests
-
-3.1 A Simple Test with one Step
+
+4.1 A Simple Test with one Step
-
-3.1.1 Create your test directory. The directory name will be the name of the test
-
-
-
+
+
mkdir simpletest
cd simpletest
-
-3.1.2 Create your testconfig file. This file contains various specifications for your test. For our example the test author has chosen to write the test using csh scripting.
-
-
-
+
+4.2 Create your testconfig file
+
+
+
# testconfig
[setup]
runscript main.csh
-
-3.1.3 Create your main.csh script
-
-
-Note: Using csh is NOT recommended. Use bash, perl, ruby, zsh or anything other than csh. We use csh here because it is popular in some industries.
+
+4.3 Create the main.csh script
+
+
+Note: Using csh is NOT recommended. Use bash, perl, ruby, zsh or anything other than csh. We use csh here because it is popular in the EDA industry for which Megatest was originally created.
-
-
+
+
#!/bin/tcsh -x
# run the cpu1 simulation.
# The step name is "run_simulation"
# The commandline being run for this step is "runsim cpu1"
@@ -370,21 +446,21 @@
$MT_MEGATEST -test-status :state COMPLETED :status $?
-
+
You can now run megatest and the created test directory will contain the new files “run_simulation.html” and “run_simulation.log”. If you are using the dashboard you can click on the run and then push the “View log” button to view the log file in firefox.
-
-3.2 Simple Test with Multiple Steps
+
+4.4 Simple Test with Multiple Steps
-
+
To run multiple steps simply add them to the main.csh file. Here we add a step to test “cpu2”. The second step that tests cpu2 will only run after the step that tested “cpu1” completes.
-
-
+
+
#!/bin/tcsh -x
# run the cpu1 simulation.
# The step name is "run_simulation"
# The commandline being run for this step is "runsim cpu1"
@@ -395,21 +471,21 @@
$MT_MEGATEST -test-status :state COMPLETED :status $?
-
-3.3 Simple Test with Multiple Steps, Some in Parallel
+
+5 Simple Test with Multiple Steps, Some in Parallel
+
+
+5.1 The Makefile
-
-3.3.1 The Makefile
-
-
+
A good way to run steps in parallel within a single test, especially when there are following steps, is to use the Unix Make utility. Writing Makefiles is beyond the scope of this document but here is a minimal example that will run “runsim cpu1” and “runsim cpu2” in parallel. For more information on make try “info make” at the Linux command prompt.
-
-
+
+
# Example Makefile to run two steps in parallel
RTLDIR=/path/to/rtl
CPUS = cpu1 cpu2
@@ -417,125 +493,200 @@
$(MT_MEGATEST) -runstep run_simulation_$(CPUS) -logpro runsim.logpro "runsim $(CPUS)
-
-3.3.2 The main.csh file
-
-
+
+5.2 The main.csh file
+
+
#!/bin/tcsh -x
# run the cpu1 and cpu2 simulations in parallel.
# The -j parameter tells make how many jobs it may run in parallel
+
make -j 2
+
+
$MT_MEGATEST -test-status :state COMPLETED :status $?
-
-3.4 Simple Test with Iteration
-
-
+
+6 Simple Test with Iteration
+
+
Since no jobs run after the cpu1 and cpu2 simulations in this test it is possible to use iterated mode.
-
-3.4.1 Update you testconfig file for iteration
-
-
-
+
+6.1 Update your testconfig file for iteration
+
+
+
[setup]
runscript main.csh
-
+
+
+
+
[items]
CPU cpu1 cpu2
-
-3.4.2 Rewrite your main.csh for iteration
-
-
-
-#!/bin/tcsh -x
-
-# run the cpu simulation but now use the environment variable $CPU
-# to select what cpu to run the simulation against
-
-$MT_MEGATEST -runstep run_simulation -logpro runsim.logpro "runsim $CPU"
-# As of version 1.07 Megatest automatically converts a status of "0"
-# to "PASS", any other number to "FAIL" and directly uses the value of
-# a string passed in.
-$MT_MEGATEST -test-status :state COMPLETED :status $?
-
-
-
-
-
-3.5 Tests with Inter-test dependencies
-
-
-Sometimes a test depends on the output from a previous test or it may not make sense to run a test is another test does not complete with status “PASS”. In either of these scenarios you can use the “waiton” keyword in your testconfig file to indicate that this test must wait on one or more tests to complete before being launched. In this example there is no point in running the “system” test if the “cpu” and “mem” tests either do not complete or complete but with status “FAIL”.
-
-
-
-4 Reference
-
-
-4.1 Configuration file Syntax
-
-
-4.1.1 Sections
-
-
-[section name]
-
-
-This creates a section named “section name”
-
-
-4.1.2 Variables
-
-
-VARX has this value
-
-
-The variable “VARX” will have the value “has this value”
-
-
-4.1.3 Includes
-
-
-[include filename]
-
-
-The file named “filename” will be included as if part of the calling file. NOTE: This means no section can be named “include “ (with the whitespace).
-
-
-4.1.4 Setting a variable by running a command
-
-
-VARNAME [system ls /tmp]
-
-
-The variable “VARNAME” will get a value created by the Unix command “ls /tmp”. All lines of output from the command will be joined with a space.
-
-
-4.1.5 Notes
+
+6.2 Rewrite your main.csh for iteration
+
+
+
+
+#!/bin/tcsh -x
+
+
+
+
+
+
+
+# run the cpu simulation but now use the environment variable $CPU
+
+
+
+# to select what cpu to run the simulation against
+
+
+
+
+
+
+
+$MT_MEGATEST -runstep run_simulation -logpro runsim.logpro "runsim $CPU"
+
+
+
+# As of version 1.07 Megatest automatically converts a status of "0"
+
+
+
+# to "PASS", any other number to "FAIL" and directly uses the value of
+
+
+
+# a string passed in.
+
+
+
+$MT_MEGATEST -test-status :state COMPLETED :status $?
+
+
+
+
+
+
+
+6.3 Tests with Inter-test dependencies
+
+
+Sometimes a test depends on the output from a previous test or it may not make sense to run a test is another test does not complete with status “PASS”. In either of these scenarios you can use the “waiton” keyword in your testconfig file to indicate that this test must wait on one or more tests to complete before being launched. In this example there is no point in running the “system” test if the “cpu” and “mem” tests either do not complete or complete but with status “FAIL”.
+
+
+
+
+# testconfig for the "system" test
+
+
+
+[setup]
+
+
+
+runscript main.csh
+
+
+
+waiton cpu mem
+
+
+
+
+
+
+
+7 Dashboard
+
+
+
+
+
+
+
+
+
+
+8 Reference
+
+
+8.1 Configuration file Syntax
+
+
+8.1.1 Sections
+
+
+
+[section name]
+
+
+
+
+
+This creates a section named “section name”
+
+
+8.1.2 Variables
+
+
+
+VARX has this value
+
+
+
+
+
+The variable “VARX” will have the value “has this value”
+
+
+8.1.3 Includes
+
+
+
+[include filename]
+
+
+
+
+
+The file named “filename” will be included as if part of the calling file. NOTE: This means no section can be named “include “ (with the whitespace).
+
+
+8.1.4 Setting a variable by running a command
+
+
+
+VARNAME [system ls /tmp]
+
+
+
+
+
+The variable “VARNAME” will get a value created by the Unix command “ls /tmp”. All lines of output from the command will be joined with a space.
+
+
+8.1.5 Notes
-
Some variables are infered as lists. Each token on the line separated by whitespace will be member of the list.
@@ -542,108 +693,84 @@
-
Comments (lines starting with #) and blank lines are ignored.
-
-4.2 Environment variables
+
+8.2 Environment variables
-
+
Variable
Purpose
-
-Default value/Comment
-
MT_CMDINFO
Conveys test variables to the megatest test runner.
-
-Not for use by the end user
-
MT_TEST_RUN_DIR
Directory assigned by megatest for the test to run.
-
-Tests can consider this their “home” directory
-
MT_TEST_NAME
Name of the test, corrosponds to the directory name under tests.
-
-
-
MT_ITEM_INFO
Iterated tests will set this to a sequence of key/values ((KEY val) ...)
-
-
-
MT_RUN_AREA_HOME
Directory where megatest was launched from and where the tests code can be found
-
-
-
MT_RUNNAME
Name of this run as set by the :runname parameter
-
-
-
MT_MEGATEST
Path/Filename to megatest executable. Found either from called path or but using the “exectuable” keyword in the [setup] section.
-
-
-
@@ -650,26 +777,20 @@
<field1> ....
The field values as set on the megatest -runall command line (e.g. :field1 abc)
-
-Planning ahead your field names is important so as to avoid collisions with other environment variables
-
-
-4.3 Configuration files
+
+8.3 megatest.config
-
-4.3.1 megatest.config
-
-
+
section
@@ -833,14 +954,14 @@
-
-4.3.2 runconfigs.config file
-
-
+
+8.4 runconfigs.config file
+
+
section
@@ -896,14 +1017,14 @@
-
+
Example: a test suite that checks that a piece of software works correctly for different customer configurations and locations each of which is done as a separate release regression run. The fields, CUSTOMER and LOCATION were chosen. The following runconfigs.config file would set some variables specific to runs for megacorp in India and femtocorp in the Cook Islands and New Zealand:
-
+
# runconfigs.config
[default]
ENCRYTION true
@@ -936,17 +1057,17 @@
ENCRYPTION true
TESTPATH /nfs/testing/megacorp_runs
-
-4.4 Writing tests
+
+8.5 Writing tests
-
-4.4.1 testconfig file
+
+8.5.1 testconfig file
-
+
section
@@ -1020,14 +1141,14 @@
-
-4.4.2 Command line
+
+8.5.2 Command line
-
Index: docs/megatest.lyx
==================================================================
--- docs/megatest.lyx
+++ docs/megatest.lyx
@@ -389,29 +389,29 @@
|
\begin_inset Text
\begin_layout Plain Layout
-0.1
+v1.25
\end_layout
\end_inset
|
\begin_inset Text
\begin_layout Plain Layout
-
+matt
\end_layout
\end_inset
|
\begin_inset Text
\begin_layout Plain Layout
-
+converted to new document format
\end_layout
\end_inset
|
@@ -1324,10 +1324,34 @@
\noindent
\align left
waiton cpu mem
\end_layout
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Section
+Dashboard
+\end_layout
+
+\begin_layout Standard
+\begin_inset Graphics
+ filename dashboard.png
+ scale 80
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Standard
+\begin_inset Graphics
+ filename dashboard-test.png
+ scale 80
\end_inset
\end_layout
@@ -2378,15 +2402,15 @@
\begin_layout Code
TESTPATH /nfs/testing/megacorp_runs
\end_layout
-\begin_layout Section
+\begin_layout Subsection
Writing tests
\end_layout
-\begin_layout Subsection
+\begin_layout Subsubsection
testconfig file
\end_layout
\begin_layout Standard
\begin_inset Tabular
@@ -2595,11 +2619,11 @@
\end_inset
\end_layout
-\begin_layout Subsection
+\begin_layout Subsubsection
Command line
\end_layout
\begin_layout Standard
\begin_inset Tabular
|
+4.1 A Simple Test with one Step
--3.1.1 Create your test directory. The directory name will be the name of the test -
-
-
+
+
-mkdir simpletest cd simpletest
-3.1.2 Create your testconfig file. This file contains various specifications for your test. For our example the test author has chosen to write the test using csh scripting. -
-
-
+
+4.2 Create your testconfig file +
+
+
-# testconfig [setup] runscript main.csh
-3.1.3 Create your main.csh script -
-
-Note: Using csh is NOT recommended. Use bash, perl, ruby, zsh or anything other than csh. We use csh here because it is popular in some industries.
+
+4.3 Create the main.csh script +
+
+Note: Using csh is NOT recommended. Use bash, perl, ruby, zsh or anything other than csh. We use csh here because it is popular in the EDA industry for which Megatest was originally created.
-
-
+
+
-#!/bin/tcsh -x # run the cpu1 simulation. # The step name is "run_simulation" # The commandline being run for this step is "runsim cpu1" @@ -370,21 +446,21 @@ $MT_MEGATEST -test-status :state COMPLETED :status $?
+
-3.2 Simple Test with Multiple Steps
+
You can now run megatest and the created test directory will contain the new files “run_simulation.html” and “run_simulation.log”. If you are using the dashboard you can click on the run and then push the “View log” button to view the log file in firefox.
-
-3.2 Simple Test with Multiple Steps
+
+4.4 Simple Test with Multiple Steps
-
+
To run multiple steps simply add them to the main.csh file. Here we add a step to test “cpu2”. The second step that tests cpu2 will only run after the step that tested “cpu1” completes.
-
-
+
+
#!/bin/tcsh -x
# run the cpu1 simulation.
# The step name is "run_simulation"
# The commandline being run for this step is "runsim cpu1"
@@ -395,21 +471,21 @@
$MT_MEGATEST -test-status :state COMPLETED :status $?
-
-3.3 Simple Test with Multiple Steps, Some in Parallel
+
+5 Simple Test with Multiple Steps, Some in Parallel
+
+
+5.1 The Makefile
-
-3.3.1 The Makefile
-
-
+
A good way to run steps in parallel within a single test, especially when there are following steps, is to use the Unix Make utility. Writing Makefiles is beyond the scope of this document but here is a minimal example that will run “runsim cpu1” and “runsim cpu2” in parallel. For more information on make try “info make” at the Linux command prompt.
-
-
+
+
# Example Makefile to run two steps in parallel
RTLDIR=/path/to/rtl
CPUS = cpu1 cpu2
@@ -417,125 +493,200 @@
$(MT_MEGATEST) -runstep run_simulation_$(CPUS) -logpro runsim.logpro "runsim $(CPUS)
-
-3.3.2 The main.csh file
-
-
+
+5.2 The main.csh file
+
+
#!/bin/tcsh -x
# run the cpu1 and cpu2 simulations in parallel.
# The -j parameter tells make how many jobs it may run in parallel
+
make -j 2
+
+
$MT_MEGATEST -test-status :state COMPLETED :status $?
-
-3.4 Simple Test with Iteration
-
-
+
+6 Simple Test with Iteration
+
+
Since no jobs run after the cpu1 and cpu2 simulations in this test it is possible to use iterated mode.
-
-3.4.1 Update you testconfig file for iteration
-
-
-
+
+6.1 Update your testconfig file for iteration
+
+
+
[setup]
runscript main.csh
-
+
+
+
+
[items]
CPU cpu1 cpu2
-
-3.4.2 Rewrite your main.csh for iteration
-
-
-
-#!/bin/tcsh -x
-
-# run the cpu simulation but now use the environment variable $CPU
-# to select what cpu to run the simulation against
-
-$MT_MEGATEST -runstep run_simulation -logpro runsim.logpro "runsim $CPU"
-# As of version 1.07 Megatest automatically converts a status of "0"
-# to "PASS", any other number to "FAIL" and directly uses the value of
-# a string passed in.
-$MT_MEGATEST -test-status :state COMPLETED :status $?
-
-
-
-
-
-3.5 Tests with Inter-test dependencies
-
-
-Sometimes a test depends on the output from a previous test or it may not make sense to run a test is another test does not complete with status “PASS”. In either of these scenarios you can use the “waiton” keyword in your testconfig file to indicate that this test must wait on one or more tests to complete before being launched. In this example there is no point in running the “system” test if the “cpu” and “mem” tests either do not complete or complete but with status “FAIL”.
-
-
-
-4 Reference
-
-
-4.1 Configuration file Syntax
-
-
-4.1.1 Sections
-
-
-[section name]
-
-
-This creates a section named “section name”
-
-
-4.1.2 Variables
-
-
-VARX has this value
-
-
-The variable “VARX” will have the value “has this value”
-
-
-4.1.3 Includes
-
-
-[include filename]
-
-
-The file named “filename” will be included as if part of the calling file. NOTE: This means no section can be named “include “ (with the whitespace).
-
-
-4.1.4 Setting a variable by running a command
-
-
-VARNAME [system ls /tmp]
-
-
-The variable “VARNAME” will get a value created by the Unix command “ls /tmp”. All lines of output from the command will be joined with a space.
-
-
-4.1.5 Notes
+
+6.2 Rewrite your main.csh for iteration
+
+
+
+
+#!/bin/tcsh -x
+
+
+
+
+
+
+
+# run the cpu simulation but now use the environment variable $CPU
+
+
+
+# to select what cpu to run the simulation against
+
+
+
+
+
+
+
+$MT_MEGATEST -runstep run_simulation -logpro runsim.logpro "runsim $CPU"
+
+
+
+# As of version 1.07 Megatest automatically converts a status of "0"
+
+
+
+# to "PASS", any other number to "FAIL" and directly uses the value of
+
+
+
+# a string passed in.
+
+
+
+$MT_MEGATEST -test-status :state COMPLETED :status $?
+
+
+
+
+
+
+
+6.3 Tests with Inter-test dependencies
+
+
+Sometimes a test depends on the output from a previous test or it may not make sense to run a test is another test does not complete with status “PASS”. In either of these scenarios you can use the “waiton” keyword in your testconfig file to indicate that this test must wait on one or more tests to complete before being launched. In this example there is no point in running the “system” test if the “cpu” and “mem” tests either do not complete or complete but with status “FAIL”.
+
+
+
+
+# testconfig for the "system" test
+
+
+
+[setup]
+
+
+
+runscript main.csh
+
+
+
+waiton cpu mem
+
+
+
+
+
+
+
+7 Dashboard
+
+
+
+
+
+
+
+
+
+
+8 Reference
+
+
+8.1 Configuration file Syntax
+
+
+8.1.1 Sections
+
+
+
+[section name]
+
+
+
+
+
+This creates a section named “section name”
+
+
+8.1.2 Variables
+
+
+
+VARX has this value
+
+
+
+
+
+The variable “VARX” will have the value “has this value”
+
+
+8.1.3 Includes
+
+
+
+[include filename]
+
+
+
+
+
+The file named “filename” will be included as if part of the calling file. NOTE: This means no section can be named “include “ (with the whitespace).
+
+
+8.1.4 Setting a variable by running a command
+
+
+
+VARNAME [system ls /tmp]
+
+
+
+
+
+The variable “VARNAME” will get a value created by the Unix command “ls /tmp”. All lines of output from the command will be joined with a space.
+
+
+8.1.5 Notes
-
Some variables are infered as lists. Each token on the line separated by whitespace will be member of the list.
@@ -542,108 +693,84 @@
-
Comments (lines starting with #) and blank lines are ignored.
-
-4.2 Environment variables
+
+8.2 Environment variables
-
+
Variable
Purpose
-
-Default value/Comment
-
MT_CMDINFO
Conveys test variables to the megatest test runner.
-
-Not for use by the end user
-
MT_TEST_RUN_DIR
Directory assigned by megatest for the test to run.
-
-Tests can consider this their “home” directory
-
MT_TEST_NAME
Name of the test, corrosponds to the directory name under tests.
-
-
-
MT_ITEM_INFO
Iterated tests will set this to a sequence of key/values ((KEY val) ...)
-
-
-
MT_RUN_AREA_HOME
Directory where megatest was launched from and where the tests code can be found
-
-
-
MT_RUNNAME
Name of this run as set by the :runname parameter
-
-
-
MT_MEGATEST
Path/Filename to megatest executable. Found either from called path or but using the “exectuable” keyword in the [setup] section.
-
-
-
@@ -650,26 +777,20 @@
<field1> ....
The field values as set on the megatest -runall command line (e.g. :field1 abc)
-
-Planning ahead your field names is important so as to avoid collisions with other environment variables
-
-
-4.3 Configuration files
+
+8.3 megatest.config
-
-4.3.1 megatest.config
-
-
+
section
@@ -833,14 +954,14 @@
-
-4.3.2 runconfigs.config file
-
-
+
+8.4 runconfigs.config file
+
+
section
@@ -896,14 +1017,14 @@
-
+
Example: a test suite that checks that a piece of software works correctly for different customer configurations and locations each of which is done as a separate release regression run. The fields, CUSTOMER and LOCATION were chosen. The following runconfigs.config file would set some variables specific to runs for megacorp in India and femtocorp in the Cook Islands and New Zealand:
-
+
# runconfigs.config
[default]
ENCRYTION true
@@ -936,17 +1057,17 @@
ENCRYPTION true
TESTPATH /nfs/testing/megacorp_runs
-
-4.4 Writing tests
+
+8.5 Writing tests
-
-4.4.1 testconfig file
+
+8.5.1 testconfig file
-
+
section
@@ -1020,14 +1141,14 @@
-
-4.4.2 Command line
+
+8.5.2 Command line
-
Index: docs/megatest.lyx
==================================================================
--- docs/megatest.lyx
+++ docs/megatest.lyx
@@ -389,29 +389,29 @@
|
\begin_inset Text
\begin_layout Plain Layout
-0.1
+v1.25
\end_layout
\end_inset
|
\begin_inset Text
\begin_layout Plain Layout
-
+matt
\end_layout
\end_inset
|
\begin_inset Text
\begin_layout Plain Layout
-
+converted to new document format
\end_layout
\end_inset
|
@@ -1324,10 +1324,34 @@
\noindent
\align left
waiton cpu mem
\end_layout
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Section
+Dashboard
+\end_layout
+
+\begin_layout Standard
+\begin_inset Graphics
+ filename dashboard.png
+ scale 80
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Standard
+\begin_inset Graphics
+ filename dashboard-test.png
+ scale 80
\end_inset
\end_layout
@@ -2378,15 +2402,15 @@
\begin_layout Code
TESTPATH /nfs/testing/megacorp_runs
\end_layout
-\begin_layout Section
+\begin_layout Subsection
Writing tests
\end_layout
-\begin_layout Subsection
+\begin_layout Subsubsection
testconfig file
\end_layout
\begin_layout Standard
\begin_inset Tabular
@@ -2595,11 +2619,11 @@
\end_inset
\end_layout
-\begin_layout Subsection
+\begin_layout Subsubsection
Command line
\end_layout
\begin_layout Standard
\begin_inset Tabular
|
To run multiple steps simply add them to the main.csh file. Here we add a step to test “cpu2”. The second step that tests cpu2 will only run after the step that tested “cpu1” completes.
-
-
+
-3.3 Simple Test with Multiple Steps, Some in Parallel
+
+
-#!/bin/tcsh -x # run the cpu1 simulation. # The step name is "run_simulation" # The commandline being run for this step is "runsim cpu1" @@ -395,21 +471,21 @@ $MT_MEGATEST -test-status :state COMPLETED :status $?
-3.3 Simple Test with Multiple Steps, Some in Parallel
+
+5 Simple Test with Multiple Steps, Some in Parallel
+
+
+5.1 The Makefile
-
-3.3.1 The Makefile
-
-
+
A good way to run steps in parallel within a single test, especially when there are following steps, is to use the Unix Make utility. Writing Makefiles is beyond the scope of this document but here is a minimal example that will run “runsim cpu1” and “runsim cpu2” in parallel. For more information on make try “info make” at the Linux command prompt.
-
-
+
+
# Example Makefile to run two steps in parallel
RTLDIR=/path/to/rtl
CPUS = cpu1 cpu2
@@ -417,125 +493,200 @@
$(MT_MEGATEST) -runstep run_simulation_$(CPUS) -logpro runsim.logpro "runsim $(CPUS)
-
-3.3.2 The main.csh file
-
-
+
+5.2 The main.csh file
+
+
#!/bin/tcsh -x
# run the cpu1 and cpu2 simulations in parallel.
# The -j parameter tells make how many jobs it may run in parallel
+
make -j 2
+
+
$MT_MEGATEST -test-status :state COMPLETED :status $?
-
-3.4 Simple Test with Iteration
-
-
+
+6 Simple Test with Iteration
+
+
Since no jobs run after the cpu1 and cpu2 simulations in this test it is possible to use iterated mode.
-
-3.4.1 Update you testconfig file for iteration
-
-
-
+
+6.1 Update your testconfig file for iteration
+
+
+
[setup]
runscript main.csh
-
+
+
+
+
[items]
CPU cpu1 cpu2
-
-3.4.2 Rewrite your main.csh for iteration
-
-
-
-#!/bin/tcsh -x
-
-# run the cpu simulation but now use the environment variable $CPU
-# to select what cpu to run the simulation against
-
-$MT_MEGATEST -runstep run_simulation -logpro runsim.logpro "runsim $CPU"
-# As of version 1.07 Megatest automatically converts a status of "0"
-# to "PASS", any other number to "FAIL" and directly uses the value of
-# a string passed in.
-$MT_MEGATEST -test-status :state COMPLETED :status $?
-
-
-
-
-
-3.5 Tests with Inter-test dependencies
-
-
-Sometimes a test depends on the output from a previous test or it may not make sense to run a test is another test does not complete with status “PASS”. In either of these scenarios you can use the “waiton” keyword in your testconfig file to indicate that this test must wait on one or more tests to complete before being launched. In this example there is no point in running the “system” test if the “cpu” and “mem” tests either do not complete or complete but with status “FAIL”.
-
-
-
-4 Reference
-
-
-4.1 Configuration file Syntax
-
-
-4.1.1 Sections
-
-
-[section name]
-
-
-This creates a section named “section name”
-
-
-4.1.2 Variables
-
-
-VARX has this value
-
-
-The variable “VARX” will have the value “has this value”
-
-
-4.1.3 Includes
-
-
-[include filename]
-
-
-The file named “filename” will be included as if part of the calling file. NOTE: This means no section can be named “include “ (with the whitespace).
-
-
-4.1.4 Setting a variable by running a command
-
-
-VARNAME [system ls /tmp]
-
-
-The variable “VARNAME” will get a value created by the Unix command “ls /tmp”. All lines of output from the command will be joined with a space.
-
-
-4.1.5 Notes
+
+6.2 Rewrite your main.csh for iteration
+
+
+
+
+#!/bin/tcsh -x
+
+
+
+
+
+
+
+# run the cpu simulation but now use the environment variable $CPU
+
+
+
+# to select what cpu to run the simulation against
+
+
+
+
+
+
+
+$MT_MEGATEST -runstep run_simulation -logpro runsim.logpro "runsim $CPU"
+
+
+
+# As of version 1.07 Megatest automatically converts a status of "0"
+
+
+
+# to "PASS", any other number to "FAIL" and directly uses the value of
+
+
+
+# a string passed in.
+
+
+
+$MT_MEGATEST -test-status :state COMPLETED :status $?
+
+
+
+
+
+
+
+6.3 Tests with Inter-test dependencies
+
+
+Sometimes a test depends on the output from a previous test or it may not make sense to run a test is another test does not complete with status “PASS”. In either of these scenarios you can use the “waiton” keyword in your testconfig file to indicate that this test must wait on one or more tests to complete before being launched. In this example there is no point in running the “system” test if the “cpu” and “mem” tests either do not complete or complete but with status “FAIL”.
+
+
+
+
+# testconfig for the "system" test
+
+
+
+[setup]
+
+
+
+runscript main.csh
+
+
+
+waiton cpu mem
+
+
+
+
+
+
+
+7 Dashboard
+
+
+
+
+
+
+
+
+
+
+8 Reference
+
+
+8.1 Configuration file Syntax
+
+
+8.1.1 Sections
+
+
+
+[section name]
+
+
+
+
+
+This creates a section named “section name”
+
+
+8.1.2 Variables
+
+
+
+VARX has this value
+
+
+
+
+
+The variable “VARX” will have the value “has this value”
+
+
+8.1.3 Includes
+
+
+
+[include filename]
+
+
+
+
+
+The file named “filename” will be included as if part of the calling file. NOTE: This means no section can be named “include “ (with the whitespace).
+
+
+8.1.4 Setting a variable by running a command
+
+
+
+VARNAME [system ls /tmp]
+
+
+
+
+
+The variable “VARNAME” will get a value created by the Unix command “ls /tmp”. All lines of output from the command will be joined with a space.
+
+
+8.1.5 Notes
-
Some variables are infered as lists. Each token on the line separated by whitespace will be member of the list.
@@ -542,108 +693,84 @@
-
Comments (lines starting with #) and blank lines are ignored.
-
-4.2 Environment variables
+
+8.2 Environment variables
-
+
Variable
Purpose
-
-Default value/Comment
-
MT_CMDINFO
Conveys test variables to the megatest test runner.
-
-Not for use by the end user
-
MT_TEST_RUN_DIR
Directory assigned by megatest for the test to run.
-
-Tests can consider this their “home” directory
-
MT_TEST_NAME
Name of the test, corrosponds to the directory name under tests.
-
-
-
MT_ITEM_INFO
Iterated tests will set this to a sequence of key/values ((KEY val) ...)
-
-
-
MT_RUN_AREA_HOME
Directory where megatest was launched from and where the tests code can be found
-
-
-
MT_RUNNAME
Name of this run as set by the :runname parameter
-
-
-
MT_MEGATEST
Path/Filename to megatest executable. Found either from called path or but using the “exectuable” keyword in the [setup] section.
-
-
-
@@ -650,26 +777,20 @@
<field1> ....
The field values as set on the megatest -runall command line (e.g. :field1 abc)
-
-Planning ahead your field names is important so as to avoid collisions with other environment variables
-
-
-4.3 Configuration files
+
+8.3 megatest.config
-
-4.3.1 megatest.config
-
-
+
section
@@ -833,14 +954,14 @@
-
-4.3.2 runconfigs.config file
-
-
+
+8.4 runconfigs.config file
+
+
section
@@ -896,14 +1017,14 @@
-
+
Example: a test suite that checks that a piece of software works correctly for different customer configurations and locations each of which is done as a separate release regression run. The fields, CUSTOMER and LOCATION were chosen. The following runconfigs.config file would set some variables specific to runs for megacorp in India and femtocorp in the Cook Islands and New Zealand:
-
+
# runconfigs.config
[default]
ENCRYTION true
@@ -936,17 +1057,17 @@
ENCRYPTION true
TESTPATH /nfs/testing/megacorp_runs
-
-4.4 Writing tests
+
+8.5 Writing tests
-
-4.4.1 testconfig file
+
+8.5.1 testconfig file
-
+
section
@@ -1020,14 +1141,14 @@
-
-4.4.2 Command line
+
+8.5.2 Command line
-
Index: docs/megatest.lyx
==================================================================
--- docs/megatest.lyx
+++ docs/megatest.lyx
@@ -389,29 +389,29 @@
|
\begin_inset Text
\begin_layout Plain Layout
-0.1
+v1.25
\end_layout
\end_inset
|
\begin_inset Text
\begin_layout Plain Layout
-
+matt
\end_layout
\end_inset
|
\begin_inset Text
\begin_layout Plain Layout
-
+converted to new document format
\end_layout
\end_inset
|
@@ -1324,10 +1324,34 @@
\noindent
\align left
waiton cpu mem
\end_layout
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Section
+Dashboard
+\end_layout
+
+\begin_layout Standard
+\begin_inset Graphics
+ filename dashboard.png
+ scale 80
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Standard
+\begin_inset Graphics
+ filename dashboard-test.png
+ scale 80
\end_inset
\end_layout
@@ -2378,15 +2402,15 @@
\begin_layout Code
TESTPATH /nfs/testing/megacorp_runs
\end_layout
-\begin_layout Section
+\begin_layout Subsection
Writing tests
\end_layout
-\begin_layout Subsection
+\begin_layout Subsubsection
testconfig file
\end_layout
\begin_layout Standard
\begin_inset Tabular
@@ -2595,11 +2619,11 @@
\end_inset
\end_layout
-\begin_layout Subsection
+\begin_layout Subsubsection
Command line
\end_layout
\begin_layout Standard
\begin_inset Tabular
|
A good way to run steps in parallel within a single test, especially when there are following steps, is to use the Unix Make utility. Writing Makefiles is beyond the scope of this document but here is a minimal example that will run “runsim cpu1” and “runsim cpu2” in parallel. For more information on make try “info make” at the Linux command prompt.
-
-
+
+
-# Example Makefile to run two steps in parallel RTLDIR=/path/to/rtl CPUS = cpu1 cpu2 @@ -417,125 +493,200 @@ $(MT_MEGATEST) -runstep run_simulation_$(CPUS) -logpro runsim.logpro "runsim $(CPUS)
-3.3.2 The main.csh file -
-
+
-
+5.2 The main.csh file +
+#!/bin/tcsh -x # run the cpu1 and cpu2 simulations in parallel. # The -j parameter tells make how many jobs it may run in parallel +make -j 2 ++ $MT_MEGATEST -test-status :state COMPLETED :status $?
-3.4 Simple Test with Iteration -
-
+
+6 Simple Test with Iteration +
+
Since no jobs run after the cpu1 and cpu2 simulations in this test it is possible to use iterated mode.
--3.4.1 Update you testconfig file for iteration -
-
-
+
-4.1.5 Notes
+
+6.1 Update your testconfig file for iteration +
+
+
-[setup] runscript main.csh - ++ ++ [items] CPU cpu1 cpu2
-3.4.2 Rewrite your main.csh for iteration -
-
-
-
-
-
-#!/bin/tcsh -x - -# run the cpu simulation but now use the environment variable $CPU -# to select what cpu to run the simulation against - -$MT_MEGATEST -runstep run_simulation -logpro runsim.logpro "runsim $CPU" -# As of version 1.07 Megatest automatically converts a status of "0" -# to "PASS", any other number to "FAIL" and directly uses the value of -# a string passed in. -$MT_MEGATEST -test-status :state COMPLETED :status $? --
-3.5 Tests with Inter-test dependencies -
-
-Sometimes a test depends on the output from a previous test or it may not make sense to run a test is another test does not complete with status “PASS”. In either of these scenarios you can use the “waiton” keyword in your testconfig file to indicate that this test must wait on one or more tests to complete before being launched. In this example there is no point in running the “system” test if the “cpu” and “mem” tests either do not complete or complete but with status “FAIL”.
-
-
--4 Reference -
--4.1 Configuration file Syntax -
--4.1.1 Sections -
-
-[section name]
-
--This creates a section named “section name” --
-4.1.2 Variables -
-
-VARX has this value
-
--The variable “VARX” will have the value “has this value” --
-4.1.3 Includes -
-
-[include filename]
-
--The file named “filename” will be included as if part of the calling file. NOTE: This means no section can be named “include “ (with the whitespace). --
-4.1.4 Setting a variable by running a command -
-
-VARNAME [system ls /tmp]
-
--The variable “VARNAME” will get a value created by the Unix command “ls /tmp”. All lines of output from the command will be joined with a space. --
-4.1.5 Notes
+
+6.2 Rewrite your main.csh for iteration
+
+
+
+
+#!/bin/tcsh -x
+
+
+
+
+
+
+
+# run the cpu simulation but now use the environment variable $CPU
+
+
+
+# to select what cpu to run the simulation against
+
+
+
+
+
+
+
+$MT_MEGATEST -runstep run_simulation -logpro runsim.logpro "runsim $CPU"
+
+
+
+# As of version 1.07 Megatest automatically converts a status of "0"
+
+
+
+# to "PASS", any other number to "FAIL" and directly uses the value of
+
+
+
+# a string passed in.
+
+
+
+$MT_MEGATEST -test-status :state COMPLETED :status $?
+
+
+
+
+
+
+
+6.3 Tests with Inter-test dependencies
+
+
+Sometimes a test depends on the output from a previous test or it may not make sense to run a test is another test does not complete with status “PASS”. In either of these scenarios you can use the “waiton” keyword in your testconfig file to indicate that this test must wait on one or more tests to complete before being launched. In this example there is no point in running the “system” test if the “cpu” and “mem” tests either do not complete or complete but with status “FAIL”.
+
+
+
+
+# testconfig for the "system" test
+
+
+
+[setup]
+
+
+
+runscript main.csh
+
+
+
+waiton cpu mem
+
+
+
+
+
+
+
+7 Dashboard
+
+
+
+
+
+
+
+
+
+
+8 Reference
+
+
+8.1 Configuration file Syntax
+
+
+8.1.1 Sections
+
+
+
+[section name]
+
+
+
+
+
+This creates a section named “section name”
+
+
+8.1.2 Variables
+
+
+
+VARX has this value
+
+
+
+
+
+The variable “VARX” will have the value “has this value”
+
+
+8.1.3 Includes
+
+
+
+[include filename]
+
+
+
+
+
+The file named “filename” will be included as if part of the calling file. NOTE: This means no section can be named “include “ (with the whitespace).
+
+
+8.1.4 Setting a variable by running a command
+
+
+
+VARNAME [system ls /tmp]
+
+
+
+
+
+The variable “VARNAME” will get a value created by the Unix command “ls /tmp”. All lines of output from the command will be joined with a space.
+
+
+8.1.5 Notes
-
Some variables are infered as lists. Each token on the line separated by whitespace will be member of the list.
@@ -542,108 +693,84 @@
-
Comments (lines starting with #) and blank lines are ignored.
-
-4.2 Environment variables
+
+8.2 Environment variables
-
+
Variable
Purpose
-
-Default value/Comment
-
MT_CMDINFO
Conveys test variables to the megatest test runner.
-
-Not for use by the end user
-
MT_TEST_RUN_DIR
Directory assigned by megatest for the test to run.
-
-Tests can consider this their “home” directory
-
MT_TEST_NAME
Name of the test, corrosponds to the directory name under tests.
-
-
-
MT_ITEM_INFO
Iterated tests will set this to a sequence of key/values ((KEY val) ...)
-
-
-
MT_RUN_AREA_HOME
Directory where megatest was launched from and where the tests code can be found
-
-
-
MT_RUNNAME
Name of this run as set by the :runname parameter
-
-
-
MT_MEGATEST
Path/Filename to megatest executable. Found either from called path or but using the “exectuable” keyword in the [setup] section.
-
-
-
@@ -650,26 +777,20 @@
<field1> ....
The field values as set on the megatest -runall command line (e.g. :field1 abc)
-
-Planning ahead your field names is important so as to avoid collisions with other environment variables
-
-
-4.3 Configuration files
+
+8.3 megatest.config
-
-4.3.1 megatest.config
-
-
+
section
@@ -833,14 +954,14 @@
-
-4.3.2 runconfigs.config file
-
-
+
+8.4 runconfigs.config file
+
+
section
@@ -896,14 +1017,14 @@
-
+
Example: a test suite that checks that a piece of software works correctly for different customer configurations and locations each of which is done as a separate release regression run. The fields, CUSTOMER and LOCATION were chosen. The following runconfigs.config file would set some variables specific to runs for megacorp in India and femtocorp in the Cook Islands and New Zealand:
-
+
# runconfigs.config
[default]
ENCRYTION true
@@ -936,17 +1057,17 @@
ENCRYPTION true
TESTPATH /nfs/testing/megacorp_runs
-
-4.4 Writing tests
+
+8.5 Writing tests
-
-4.4.1 testconfig file
+
+8.5.1 testconfig file
-
+
section
@@ -1020,14 +1141,14 @@
-
-4.4.2 Command line
+
+8.5.2 Command line
-
Index: docs/megatest.lyx
==================================================================
--- docs/megatest.lyx
+++ docs/megatest.lyx
@@ -389,29 +389,29 @@
|
\begin_inset Text
\begin_layout Plain Layout
-0.1
+v1.25
\end_layout
\end_inset
|
\begin_inset Text
\begin_layout Plain Layout
-
+matt
\end_layout
\end_inset
|
\begin_inset Text
\begin_layout Plain Layout
-
+converted to new document format
\end_layout
\end_inset
|
@@ -1324,10 +1324,34 @@
\noindent
\align left
waiton cpu mem
\end_layout
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Section
+Dashboard
+\end_layout
+
+\begin_layout Standard
+\begin_inset Graphics
+ filename dashboard.png
+ scale 80
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Standard
+\begin_inset Graphics
+ filename dashboard-test.png
+ scale 80
\end_inset
\end_layout
@@ -2378,15 +2402,15 @@
\begin_layout Code
TESTPATH /nfs/testing/megacorp_runs
\end_layout
-\begin_layout Section
+\begin_layout Subsection
Writing tests
\end_layout
-\begin_layout Subsection
+\begin_layout Subsubsection
testconfig file
\end_layout
\begin_layout Standard
\begin_inset Tabular
@@ -2595,11 +2619,11 @@
\end_inset
\end_layout
-\begin_layout Subsection
+\begin_layout Subsubsection
Command line
\end_layout
\begin_layout Standard
\begin_inset Tabular
|
+
+
+++#!/bin/tcsh -x ++ ++ ++ ++# run the cpu simulation but now use the environment variable $CPU ++ ++# to select what cpu to run the simulation against ++ ++ ++ ++$MT_MEGATEST -runstep run_simulation -logpro runsim.logpro "runsim $CPU" ++ ++# As of version 1.07 Megatest automatically converts a status of "0" ++ ++# to "PASS", any other number to "FAIL" and directly uses the value of ++ ++# a string passed in. ++ ++$MT_MEGATEST -test-status :state COMPLETED :status $? ++ +
+
+
+++# testconfig for the "system" test ++ ++[setup] ++ ++runscript main.csh ++ ++waiton cpu mem ++ +
+
+
+
+
+
+
+[section name] ++
+
+
+VARX has this value ++
+
+
+[include filename] ++
+
+
+VARNAME [system ls /tmp] ++
+8.2 Environment variables
-
+
-
-4.3 Configuration files
+
| Variable | Purpose | --Default value/Comment - |
| MT_CMDINFO | Conveys test variables to the megatest test runner. | --Not for use by the end user - |
| MT_TEST_RUN_DIR | Directory assigned by megatest for the test to run. | --Tests can consider this their “home” directory - |
| MT_TEST_NAME | Name of the test, corrosponds to the directory name under tests. - | -- |
| MT_ITEM_INFO | Iterated tests will set this to a sequence of key/values ((KEY val) ...) - | -- |
| MT_RUN_AREA_HOME | Directory where megatest was launched from and where the tests code can be found - | -- |
| MT_RUNNAME | Name of this run as set by the :runname parameter - | -- |
| MT_MEGATEST | Path/Filename to megatest executable. Found either from called path or but using the “exectuable” keyword in the [setup] section. - | -- |
| @@ -650,26 +777,20 @@ <field1> .... | The field values as set on the megatest -runall command line (e.g. :field1 abc) | --Planning ahead your field names is important so as to avoid collisions with other environment variables - |
-4.3 Configuration files
+
+8.3 megatest.config
-
-4.3.1 megatest.config
-
-
+
section
@@ -833,14 +954,14 @@
-
-4.3.2 runconfigs.config file
-
-
+
+8.4 runconfigs.config file
+
+
section
@@ -896,14 +1017,14 @@
-
+
Example: a test suite that checks that a piece of software works correctly for different customer configurations and locations each of which is done as a separate release regression run. The fields, CUSTOMER and LOCATION were chosen. The following runconfigs.config file would set some variables specific to runs for megacorp in India and femtocorp in the Cook Islands and New Zealand:
-
+
# runconfigs.config
[default]
ENCRYTION true
@@ -936,17 +1057,17 @@
ENCRYPTION true
TESTPATH /nfs/testing/megacorp_runs
-
-4.4 Writing tests
+
+8.5 Writing tests
-
-4.4.1 testconfig file
+
+8.5.1 testconfig file
-
+
section
@@ -1020,14 +1141,14 @@
-
-4.4.2 Command line
+
+8.5.2 Command line
-
Index: docs/megatest.lyx
==================================================================
--- docs/megatest.lyx
+++ docs/megatest.lyx
@@ -389,29 +389,29 @@
|
\begin_inset Text
\begin_layout Plain Layout
-0.1
+v1.25
\end_layout
\end_inset
|
\begin_inset Text
\begin_layout Plain Layout
-
+matt
\end_layout
\end_inset
|
\begin_inset Text
\begin_layout Plain Layout
-
+converted to new document format
\end_layout
\end_inset
|
@@ -1324,10 +1324,34 @@
\noindent
\align left
waiton cpu mem
\end_layout
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Section
+Dashboard
+\end_layout
+
+\begin_layout Standard
+\begin_inset Graphics
+ filename dashboard.png
+ scale 80
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Standard
+\begin_inset Graphics
+ filename dashboard-test.png
+ scale 80
\end_inset
\end_layout
@@ -2378,15 +2402,15 @@
\begin_layout Code
TESTPATH /nfs/testing/megacorp_runs
\end_layout
-\begin_layout Section
+\begin_layout Subsection
Writing tests
\end_layout
-\begin_layout Subsection
+\begin_layout Subsubsection
testconfig file
\end_layout
\begin_layout Standard
\begin_inset Tabular
@@ -2595,11 +2619,11 @@
\end_inset
\end_layout
-\begin_layout Subsection
+\begin_layout Subsubsection
Command line
\end_layout
\begin_layout Standard
\begin_inset Tabular
|
| section | @@ -833,14 +954,14 @@
-4.3.2 runconfigs.config file -
-
+
-
+8.4 runconfigs.config file +
+| section | @@ -896,14 +1017,14 @@
+
Example: a test suite that checks that a piece of software works correctly for different customer configurations and locations each of which is done as a separate release regression run. The fields, CUSTOMER and LOCATION were chosen. The following runconfigs.config file would set some variables specific to runs for megacorp in India and femtocorp in the Cook Islands and New Zealand:
-
+
-4.4 Writing tests
+
# runconfigs.config [default] ENCRYTION true @@ -936,17 +1057,17 @@ ENCRYPTION true
TESTPATH /nfs/testing/megacorp_runs
-
-4.4 Writing tests
+
+8.5 Writing tests
-
-4.4.1 testconfig file
+
+8.5.1 testconfig file
-
+
section
@@ -1020,14 +1141,14 @@
-
-4.4.2 Command line
+
+8.5.2 Command line
-
Index: docs/megatest.lyx
==================================================================
--- docs/megatest.lyx
+++ docs/megatest.lyx
@@ -389,29 +389,29 @@
|
\begin_inset Text
\begin_layout Plain Layout
-0.1
+v1.25
\end_layout
\end_inset
|
\begin_inset Text
\begin_layout Plain Layout
-
+matt
\end_layout
\end_inset
|
\begin_inset Text
\begin_layout Plain Layout
-
+converted to new document format
\end_layout
\end_inset
|
@@ -1324,10 +1324,34 @@
\noindent
\align left
waiton cpu mem
\end_layout
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Section
+Dashboard
+\end_layout
+
+\begin_layout Standard
+\begin_inset Graphics
+ filename dashboard.png
+ scale 80
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Standard
+\begin_inset Graphics
+ filename dashboard-test.png
+ scale 80
\end_inset
\end_layout
@@ -2378,15 +2402,15 @@
\begin_layout Code
TESTPATH /nfs/testing/megacorp_runs
\end_layout
-\begin_layout Section
+\begin_layout Subsection
Writing tests
\end_layout
-\begin_layout Subsection
+\begin_layout Subsubsection
testconfig file
\end_layout
\begin_layout Standard
\begin_inset Tabular
@@ -2595,11 +2619,11 @@
\end_inset
\end_layout
-\begin_layout Subsection
+\begin_layout Subsubsection
Command line
\end_layout
\begin_layout Standard
\begin_inset Tabular
|
+8.5.1 testconfig file
-
+
-
-4.4.2 Command line
+
| section | @@ -1020,14 +1141,14 @@