Megatest

</div>
</div>
<div class="sect1">
<h2 id="_why_megatest">Why Megatest?</h2>
<div class="sectionbody">
<div class="paragraph"><p>The Megatest project was started for two reasons, the first was an
immediate and pressing need for a generalized tool to manage a suite
of regression tests and the second was the fact that I had written or
maintained several such tools at different companies over the years. I
thought a single open source tool, flexible enough to meet the needs
of any team doing continuous integrating and or running a complex
suite of tests for release qualification would solve some problems for
me and for others.</p></div>
<div class="literalblock">
<div class="content monospaced">
<pre>-- Matt Welland, original author of the Megatest tool suite.</pre>
</div></div>
</div>
</div>
<div class="sect1">
<h2 id="_megatest_design_philosophy">Megatest Design Philosophy</h2>
<div class="sectionbody">
<div class="paragraph"><p>Megatest is a distributed system intended to provide the minimum needed
resources to make writing a suite of tests and tasks for implementing
continuous build for software, design engineering or process control (via
owlfs for example) without being specialized for any specific problem
space. Megatest in of itself does not know what constitutes a PASS or FAIL
of a test or task. In most cases megatest is best used in conjunction with
logpro or a similar tool to parse, analyze and decide on the test outcome.</p></div>
<div class="ulist"><ul>
<li>
<p>
Self-checking - make it as easy as possible to write self-checking
   tests (as opposed to using deltas, i.e. tests that compare with a
   previous measurement to deterine PASS/FAIL).
</p>
</li>
<li>
<p>
Traceable - environment variables, host OS and other possibly
   influential variables are captured and kept recorded.
</p>
</li>
<li>
<p>
Immutable - once a test is run it cannot be easily overwritten or
   modified accidentally.
</p>
</li>
<li>
<p>
Repeatable - test results can be recreated in the future using all
   the original variables.
</p>
</li>
<li>
<p>











Relocatable - the testsuite or automation area can be checked out
   and the tests run anywhere in the disk hierarchy.
</p>
</li>
<li>
<p>
Encapsulated - the tests run in self-contained directories and all
   inputs and outputs to the process can be found in the run areas.
</p>
</li>
<li>
<p>
Deployable - a testsuite is self-contained and can be bundled with
   a software project and easily used by others with little to no
   setup burden.
</p>
</li>
</ul></div>
</div>
</div>
<div class="sect1">
<h2 id="_megatest_architecture">Megatest Architecture</h2>
<div class="sectionbody">
<div class="sect2">
<h3 id="_data_separation">Data separation</h3>
<div class="paragraph"><p>All data to specify the tests and configure the system is stored in
plain text config files. All system state is stored in an sqlite3
database.</p></div>
</div>
<div class="sect2">
<h3 id="_distributed_compute">Distributed Compute</h3>
<div class="paragraph"><p>Tests are launched using the launching system available for the
distributed compute platform in use. A template script is provided
which can launch jobs on local and remote Linux hosts. Currently
megatest uses the network filesystem to call home to your master
sqlite3 database. Megatest has been used with the Intel Netbatch and
lsf (also known as openlava) batch systems and it should be
straightforward to use it with other similar systems.</p></div>
</div>
</div>












</div>
<div class="sect1">








<h2 id="_overview">Overview</h2>


































<div class="sectionbody">
<div class="sect2">



<h3 id="_stand_alone_megatest_area">Stand-alone Megatest Area</h3>
<div class="paragraph"><p>A single, stand-alone, Megatest based testsuite or "area" is
sufficient for most validation, automation and build problems.</p></div>
<div class="imageblock">




<div class="content">






<img src="megatest-stand-alone-area.png" alt="Static">




</div>








</div>





<div class="paragraph"><p>Megatest is designed as a distributed or decoupled system. This means
















































you can run the areas stand-alone with no additional


infrastructure. I.e. there are no databases, web servers or other
centralized resources needed. However as your needs grow you can
integrate multiple areas into a bigger system.</p></div>






























































<div class="sect3">




<h4 id="_component_descriptions">Component Descriptions</h4>
<div class="olist arabic"><ol class="arabic">
<li>
<p>


Multi-area dashboard and xterm. A gui (the dashboard) is usually the
  best option for controlling and launching runs but all operations
  can also be done from the commandline. Note: The not yet released
  multi-area dashboard replaces the old dashboard for browsing and
  controlling runs but for managing a single area the old dashboard
  works very well.



</p>
</li>
<li>





<p>
























Area/testsuite. This is your testsuite or automation definition and






















  consists of the information in megatest.config, runconfigs.config






  and your testconfigs along with any custom scripting that can&#8217;t be
  done with the native Megatest features.
</p>
</li>
<li>
<p>
If your testsuite or build automation is too large to run on a
  single instance you can distribute your jobs into a compute server
  pool. The only current requirements are password-less ssh access and











  a network filesystem.





</p>
</li>
</ol></div>
</div>
</div>
<div class="sect2">
<h3 id="_full_system_architecture">Full System Architecture</h3>
<div class="imageblock">

<div class="content">


























<img src="megatest-system-architecture.png" alt="Static">
</div>



</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_installation">Installation</h2>
<div class="sectionbody">
<div class="sect2">
<h3 id="_dependencies">Dependencies</h3>
<div class="paragraph"><p>Chicken scheme and a number of "eggs" are required for building
Megatest. See the script installall.sh in the utils directory of the
source distribution for an automated way to install everything
needed for building Megatest on Linux.</p></div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_getting_started">Getting Started</h2>
<div class="sectionbody">
<div class="listingblock">
<div class="title">Getting started with Megatest</div>
<div class="content monospaced">
<pre>Creating a testsuite or flow and your first test or task.</pre>
</div></div>
<div class="paragraph"><p>After installing Megatest you can create a flow or testsuite and add some
tests using the helpers. Here is a quickstart sequence to get you up and
running your first automated testsuite.</p></div>
<div class="sect2">
<h3 id="_creating_a_megatest_area">Creating a Megatest Area</h3>
<div class="sect3">

<div class="paragraph"><p>Start the dashboard and browse your run in the "Runs" tab.</p></div>
<div class="listingblock">
<div class="title">Starting dashboard</div>
<div class="content monospaced">
<pre>dashboard -rows 24</pre>
</div></div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_study_plan">Study Plan</h2>
<div class="sectionbody">
<div class="paragraph"><p>Megatest is an extensive program with a lot to learn. Following are some paths through the material to smooth the learning path.</p></div>
<div class="sect2">
<h3 id="_basic_concepts_suggest_you_pick_these_up_on_the_way">Basic Concepts (suggest you pick these up on the way)</h3>
<div class="ulist"><ul>
<li>
<p>
Components of automation; run, test, iteration
</p>
</li>
<li>
<p>
Selectors; target, runname, and testpatt
</p>
</li>
</ul></div>
</div>
<div class="sect2">
<h3 id="_running_testsuites_or_automation">Running Testsuites or Automation</h3>
<div class="ulist"><ul>
<li>
<p>
Using the dashboard gui (recommended)
</p>
<div class="ulist"><ul>
<li>
<p>
Using the "Runs" panel.
</p>
</li>
<li>
<p>
Using the "Run Control" panel.
</p>
</li>
<li>
<p>
Using a test control panel
</p>
</li>
<li>
<p>
The Right Mouse Button menu
</p>
</li>
<li>
<p>
Debug features
</p>
<div class="ulist"><ul>
<li>
<p>
xterm
</p>
<div class="ulist"><ul>
<li>
<p>
pstree
</p>
</li>
<li>
<p>
log files; mt_copy.log, mt_launch.log
</p>
</li>
<li>
<p>
variables; megatest.csh, megatest.sh
</p>
</li>
<li>
<p>
testconfig dump, *testconfig
</p>
</li>
</ul></div>
</li>
<li>
<p>
State/status buttons
</p>
</li>
<li>
<p>
Run, Clean, KillReq
</p>
</li>
<li>
<p>
ReRunClean
</p>
</li>
</ul></div>
</li>
</ul></div>
</li>
<li>
<p>
Using the command line
</p>
<div class="ulist"><ul>
<li>
<p>
Getting help; megatest -h, megatest -manual
</p>
</li>
<li>
<p>
Starting runs; megatest -run
</p>
<div class="ulist"><ul>
<li>
<p>
Selection controls; -target, -runname and -testpatt
</p>
</li>
</ul></div>
</li>
</ul></div>
</li>
</ul></div>
</div>
<div class="sect2">
<h3 id="_writing_tests_and_flows">Writing Tests and Flows</h3>
<div class="ulist"><ul>
<li>
<p>
environment variables (table 5)
</p>
</li>
<li>
<p>
tests/<em>testname</em>/testconfig <a href="megatest_manual.html#_the_testconfig_file">testconfig details</a>
</p>
<div class="ulist"><ul>
<li>
<p>
ezsteps and logpro section
</p>
</li>
<li>
<p>
iteration (one test applied to many inputs), items, itemstable <a href="megatest_manual.html#_iteration">test iteration</a>
</p>
</li>
<li>
<p>
dependencies, waiton, itemmatch, itemwait <a href="megatest_manual.html#_requirements_section">test requirements</a>
</p>
</li>
<li>
<p>
miscellaneous; mode toplevel, runtimelim, skip on file or on running, waiver propagation
</p>
</li>
</ul></div>
</li>
<li>
<p>
megatest areas
</p>
<div class="ulist"><ul>
<li>
<p>
megatest.config
</p>
</li>
<li>
<p>
runconfigs.config
</p>
</li>
<li>
<p>
config language features; include, shell, system, scheme, rp|realpath, getenv, get, rget, scriptinc <a href="megatest.html#_config_file_helpers">config file helpers</a>
</p>
</li>
</ul></div>
</li>
</ul></div>
</div>
<div class="sect2">
<h3 id="_advanced_topics">Advanced Topics</h3>
<div class="ulist"><ul>
<li>
<p>
Removing and keeping runs selectively <a href="megatest_manual.html#_managing_old_runs">managing runs</a>
</p>
</li>
<li>
<p>
Subruns <a href="megatest_manual.html#_nested_runs">nested runs</a>
</p>
</li>
<li>
<p>
Config file features <a href="megatest_manual.html#_config_file_helpers">config file features</a>
</p>
</li>
<li>
<p>
HTML output with -generate-html
</p>
</li>
<li>
<p>
Triggers, post run, state/status
</p>
</li>
<li>
<p>
MTLOWESTLOAD
</p>
</li>
<li>
<p>
flexilauncher
</p>
</li>
<li>
<p>
env delta and testconfig
</p>
</li>
<li>
<p>
capturing test data, extracting values from logpro and using them for pass/fail
</p>
</li>
<li>
<p>
mtutil, postgres connection, packets for cross-site/cross-user control (e.g. mcrun).
</p>
</li>
</ul></div>
</div>
<div class="sect2">
<h3 id="_maintenance_and_troubleshooting">Maintenance and Troubleshooting</h3>
<div class="ulist"><ul>
<li>
<p>
cleanup-db, database structure of Megatest 1.6x
</p>
</li>
<li>
<p>
archiving
</p>
</li>
<li>
<p>
homehost management
</p>
</li>
<li>
<p>
show-runconfig
</p>
</li>
<li>
<p>
show-config
</p>
</li>
<li>
<p>
show with -debug 0,9
</p>
</li>
<li>
<p>
load management
</p>
</li>
</ul></div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_writing_tests">Writing Tests</h2>
<div class="sectionbody">
<div class="sect2">
<h3 id="_creating_a_new_test">Creating a new Test</h3>

VAR1 VAL2</pre>
</div></div>
<div class="paragraph"><p>Can use either wildcard of "%" or a regular expression:</p></div>
<div class="listingblock">
<div class="content monospaced">
<pre>[/abc.*def/]</pre>
</div></div>
</div>
<div class="sect2">
<h3 id="_disk_space_checks">Disk Space Checks</h3>
<div class="paragraph"><p>Some parameters you can put in the [setup] section of megatest.config:</p></div>
<div class="listingblock">
<div class="content monospaced">
<pre># minimum space required in a run disk
minspace 10000000

# minimum space required in dbdir:
dbdir-space-required 100000

# script that takes path as parameter and returns number of bytes available:
free-space-script check-space.sh</pre>
</div></div>
</div>
<div class="sect2">
<h3 id="_trim_trailing_spaces">Trim trailing spaces</h3>
<div class="listingblock">
<div class="content monospaced">
<pre>[configf:settings trim-trailing-spaces yes]</pre>
</div></div>
</div>
<div class="sect2">
<h3 id="_job_submission_control">Job Submission Control</h3>
<div class="sect3">
<h4 id="_submit_jobs_to_host_types_based_on_test_name_2">Submit jobs to Host Types based on Test Name</h4>
<div class="listingblock">
<div class="title">In megatest.config</div>
<div class="content monospaced">
<pre>[host-types]
general   nbfake
remote    bsub

[launchers]
runfirst/sum% remote
% general

[jobtools]
launcher bsub
# if defined and not "no" flexi-launcher will bypass launcher unless
# there is no host-type match.
flexi-launcher yes</pre>
</div></div>

<div class="sect4">
<h5 id="_host_types">host-types</h5>
<div class="paragraph"><p>List of host types and the commandline to run a job on that host type.</p></div>
<div class="listingblock">
<div class="title">host-type &#8658; launch command</div>
<div class="content monospaced">
<pre>general nbfake</pre>

<div class="paragraph"><p>The runscript method is a brute force way to run scripts where the
user is responsible for setting STATE and STATUS</p></div>
<div class="listingblock">
<div class="content monospaced">
<pre>runscript main.csh</pre>
</div></div>
</div>
</div>
<div class="sect2">
<h3 id="_iteration">Iteration</h3>
<div class="listingblock">
<div class="title">Sections for iteration</div>
<div class="content monospaced">
<pre># full combinations
[items]
A x y
B 1 2

# Yields: x/1 x/2 y/1 y/2

# tabled
[itemstable]
A x y
B 1 2

# Yields x/1 y/2</pre>
</div></div>
</div>
<div class="sect2">
<h3 id="_requirements_section">Requirements section</h3>
<div class="listingblock">
<div class="title">Header</div>
<div class="content monospaced">
<pre>[requirements]</pre>
</div></div>
</div>
<div class="sect2">
<h3 id="_wait_on_other_tests">Wait on Other Tests</h3>
<div class="listingblock">
<div class="content monospaced">
<pre># A normal waiton waits for the prior tests to be COMPLETED
# and PASS, CHECK or WAIVED
waiton test1 test2</pre>
</div></div>
</div>
<div class="sect2">
<h3 id="_mode">Mode</h3>
<div class="paragraph"><p>The default (i.e. if mode is not specified) is normal. All pre-dependent tests
must be COMPLETED and PASS, CHECK or WAIVED before the test will start</p></div>
<div class="listingblock">
<div class="content monospaced">
<pre>[requirements]
mode   normal</pre>
</div></div>

was historically called "itemwait" mode. The terms "itemwait" and
"itemmatch" are synonyms.</p></div>
<div class="listingblock">
<div class="content monospaced">
<pre>[requirements]
mode itemmatch</pre>
</div></div>

</div>
<div class="sect2">
<h3 id="_overriding_enviroment_variables">Overriding Enviroment Variables</h3>
<div class="paragraph"><p>Override variables before starting the test. Can include files (perhaps generated by megatest -envdelta or similar).</p></div>
<div class="listingblock">
<div class="content monospaced">
<pre>[pre-launch-env-vars]

# The following would map:
#   a123b321 to b321fooa123 then to 321fooa123p
#
[requirements]
itemmap (a\d+)(b\d+) \2foo\1
  b(.*) \1p</pre>
</div></div>
</div>
<div class="sect2">
<h3 id="_complex_mapping">Complex mapping</h3>
<div class="paragraph"><p>Complex mappings can be handled with a separate [itemmap] section (instead if an itemmap line in the [requirements] section)</p></div>
<div class="paragraph"><p>Each line in an itemmap section starts with a waiton test name followed by an itemmap expression</p></div>
<div class="listingblock">
<div class="title">eg: The following causes waiton test A item <span class="monospaced">bar/1234</span> to run when our test&#8217;s <span class="monospaced">foo/1234</span> item is requested as well as causing waiton test B&#8217;s <span class="monospaced">blah</span> item to run when our test&#8217;s <span class="monospaced">stuff/blah</span> item is requested</div>
<div class="content monospaced">
<pre>[itemmap]
A foo/ bar/
B stuff/</pre>
</div></div>
</div>
<div class="sect2">
<h3 id="_complex_mapping_example">Complex mapping example</h3>
<div class="imageblock">
<div class="content">
<img src="complex-itemmap.png" alt="complex-itemmap.png">
</div>
</div>
<div class="paragraph"><p>We accomplish this by configuring the testconfigs of our tests C D and E as follows:</p></div>
<div class="listingblock">

<li>
<p>
Test A has no waitons.  All waitons of all tests in full list have been processed.  Full list is finalized.
</p>
</li>
</ol></div>
</div>
<div class="sect2">
<h3 id="_itemstable">itemstable</h3>
<div class="paragraph"><p>An alternative to defining items is the itemstable section.  This lets you define the itempath in a table format rather than specifying components and relying on getting all permutations of those components.</p></div>
</div>
<div class="sect2">
<h3 id="_dynamic_flow_dependency_tree">Dynamic Flow Dependency Tree</h3>
<div class="listingblock">
<div class="title">Autogeneration waiton list for dynamic flow dependency trees</div>
<div class="content monospaced">
<pre>[requirements]
# With a toplevel test you may wish to generate your list
# of tests to run dynamically
#
waiton #{shell get-valid-tests-to-run.sh}</pre>
</div></div>
</div>
<div class="sect2">
<h3 id="_run_time_limit_2">Run time limit</h3>
<div class="listingblock">
<div class="content monospaced">
<pre>[requirements]
runtimelim 1h 2m 3s  # this will automatically kill the test if it runs for more than 1h 2m and 3s</pre>
</div></div>
</div>
<div class="sect2">
<h3 id="_skip">Skip</h3>
<div class="paragraph"><p>A test with a skip section will conditional skip running.</p></div>
<div class="listingblock">
<div class="title">Skip section example</div>
<div class="content monospaced">
<pre>[skip]
prevrunning x
# rundelay 30m 15s</pre>
</div></div>
</div>
<div class="sect2">
<h3 id="_skip_on_still_running_tests">Skip on Still-running Tests</h3>
<div class="listingblock">
<div class="content monospaced">
<pre># NB// If the prevrunning line exists with *any* value the test will
# automatically SKIP if the same-named test is currently RUNNING. The
# "x" can be any string. Comment out the prevrunning line to turn off
# skip.

[skip]
prevrunning x</pre>
</div></div>
</div>
<div class="sect2">
<h3 id="_skip_if_a_file_exists">Skip if a File Exists</h3>
<div class="listingblock">
<div class="content monospaced">
<pre>[skip]
fileexists /path/to/a/file # skip if /path/to/a/file exists</pre>
</div></div>
</div>
<div class="sect2">
<h3 id="_skip_if_test_ran_more_recently_than_specified_time">Skip if test ran more recently than specified time</h3>
<div class="listingblock">
<div class="title">Skip if this test has been run in the past fifteen minutes and 15 seconds.</div>
<div class="content monospaced">
<pre>[skip]
rundelay 15m 15s</pre>
</div></div>
</div>
<div class="sect2">
<h3 id="_disks">Disks</h3>
<div class="paragraph"><p>A disks section in testconfig will override the disks section in
megatest.config. This can be used to allocate disks on a per-test or per item
basis.</p></div>
</div>
<div class="sect2">
<h3 id="_controlled_waiver_propagation">Controlled waiver propagation</h3>
<div class="paragraph"><p>If test is FAIL and previous test in run with same MT_TARGET is WAIVED then apply the following rules from the testconfig:
If a waiver check is specified in the testconfig apply the check and if it passes then set this FAIL to WAIVED</p></div>
<div class="paragraph"><p>Waiver check has two parts, 1) a list of waiver, rulename, filepatterns and 2) the rulename script spec (note that "diff" and "logpro" are predefined)</p></div>
<div class="listingblock">
<div class="content monospaced">
<pre>###### EXAMPLE FROM testconfig #########
# matching file(s) will be diff'd with previous run and logpro applied

# This builtin rule is the default if there is no <waivername>.logpro file
# diff   diff %file1% %file2%

# This builtin rule is applied if a <waivername>.logpro file exists
# logpro diff %file1% %file2% | logpro %waivername%.logpro %waivername%.html</pre>
</div></div>
</div>

<div class="sect2">
<h3 id="_ezsteps">Ezsteps</h3>
<div class="listingblock">
<div class="title">Example ezsteps with logpro rules</div>
<div class="content monospaced">
<pre>[ezsteps]
lookittmp   ls /tmp

[logpro]
lookittmp ;; Note: config file format supports multi-line entries where leading whitespace is removed from each line
  ;;     a blank line indicates the end of the block of text
  (expect:required in "LogFileBody" &gt; 0 "A file name that should never exist!" #/This is a awfully stupid file name that should never be found in the temp dir/)</pre>
</div></div>
<div class="paragraph"><p>To transfer the environment to the next step you can do the following:</p></div>
<div class="listingblock">
<div class="title">Propagate environment to next step</div>
<div class="content monospaced">
<pre>$MT_MEGATEST -env2file .ezsteps/${stepname}</pre>
</div></div>
</div>
<div class="sect2">
<h3 id="_triggers">Triggers</h3>
<div class="paragraph"><p>In your testconfig or megatest.config triggers can be specified</p></div>
<div class="listingblock">
<div class="title">Triggers spec</div>
<div class="content monospaced">
<pre>[triggers]

# Call script running.sh when test goes to state=RUNNING, status=PASS
RUNNING/PASS running.sh

# Call script running.sh any time state goes to RUNNING
RUNNING/ running.sh

# Call script onpass.sh any time status goes to PASS
PASS/ onpass.sh</pre>
</div></div>
<div class="paragraph"><p>Scripts called will have; test-id test-rundir trigger test-name item-path state status event-time, added to the commandline.</p></div>
<div class="paragraph"><p>HINT</p></div>
<div class="paragraph"><p>To start an xterm (useful for debugging), use a command line like the following:</p></div>
<div class="listingblock">
<div class="title">Start an xterm using a trigger for test completed.</div>
<div class="content monospaced">
<pre>[triggers]
COMPLETED/ xterm -e bash -s --</pre>
</div></div>
<div class="admonitionblock">
<table><tr>
<td class="icon">
<img src="/nfs/pdx/disks/ice.disk.002/icfadm/pkgs/asciidoc/8.6.7/images/icons/note.png" alt="Note">
</td>
<td class="content">There is a trailing space after the double-dash</td>
</tr></table>
</div>
<div class="paragraph"><p>There are a number of environment variables available to the trigger script
but since triggers can be called in various contexts not all variables are
available at all times. The trigger script should check for the variable and
fail gracefully if it doesn&#8217;t exist.</p></div>
<table class="tableblock frame-topbot grid-all"
style="
width:90%;
">
<caption class="title">Table 5. Environment variables visible to the trigger script</caption>
<col style="width:50%;">
<col style="width:50%;">
<thead>
<tr>
<th class="tableblock halign-left valign-top" > Variable            </th>
<th class="tableblock halign-left valign-top" > Purpose</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top" ><p class="tableblock">MT_TEST_RUN_DIR</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock">The directory where Megatest ran this test</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" ><p class="tableblock">MT_CMDINFO</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock">Encoded command data for the test</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" ><p class="tableblock">MT_DEBUG_MODE</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock">Used to pass the debug mode to nested calls to Megatest</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" ><p class="tableblock">MT_RUN_AREA_HOME</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock">Megatest home area</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" ><p class="tableblock">MT_TESTSUITENAME</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock">The name of this testsuite or area</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" ><p class="tableblock">MT_TEST_NAME</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock">The name of this test</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" ><p class="tableblock">MT_ITEM_INFO</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock">The variable and values for the test item</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" ><p class="tableblock">MT_MEGATEST</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock">Which Megatest binary is being used by this area</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" ><p class="tableblock">MT_TARGET</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock">The target variable values, separated by <em>/</em></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" ><p class="tableblock">MT_LINKTREE</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock">The base of the link tree where all run tests can be found</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" ><p class="tableblock">MT_ITEMPATH</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock">The values of the item path variables, separated by <em>/</em></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" ><p class="tableblock">MT_RUNNAME</p></td>
<td class="tableblock halign-left valign-top" ><p class="tableblock">The name of the run</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect2">
<h3 id="_override_the_toplevel_html_file">Override the Toplevel HTML File</h3>
<div class="paragraph"><p>Megatest generates a simple html file summary for top level tests of

# Within the archive the data is structured like this:
#  <target>/<runname>/<test>/
archive0 /mfs/myarchive-data/adisk1</pre>
</div></div>
</div>
</div>
<div class="sect1">
<h2 id="_environment_variables">Environment Variables</h2>
<div class="sectionbody">
<div class="paragraph"><p>It is often necessary to capture and or manipulate environment
variables. Megatest has some facilities built in to help.</p></div>
<div class="sect2">
<h3 id="_capture_variables">Capture variables</h3>
<div class="listingblock">
<div class="title">Commands</div>
<div class="content monospaced">
<pre># capture the current enviroment into a db called envdat.db under
# the context "before"
megatest -envcap before

# capture the current environment into a db called startup.db with
# context "after"
megatest -envcap after startup.db

# write the diff from before to after
megatest -envdelta before-after -dumpmode bash</pre>
</div></div>
<div class="paragraph"><p>Dump modes include bash, csh and config. You can include config data
into megatest.config, runconfigs.config and testconfig files. This is
useful for capturing a complex environment in a special-purpose test
and then utilizing that environment in downstream tests.</p></div>
<div class="listingblock">
<div class="title">Example of generating and using config data</div>
<div class="content monospaced">
<pre>megatest -envcap original
# do some stuff here
megatest -envcap munged
megatest -envdelta original-munged -dumpmode ini -o modified.config</pre>
</div></div>
<div class="paragraph"><p>Then in runconfigs.config</p></div>
<div class="listingblock">
<div class="title">Example of using modified.config in a testconfig</div>
<div class="content monospaced">

<pre>[pre-launch-env-vars]
[include modified.config]</pre>
</div></div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_managing_old_runs">Managing Old Runs</h2>

<div class="content">
<img src="server.png" alt="server.png">
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_road_map">Road Map</h2>
<div class="sectionbody">
<div class="paragraph"><p>Note 1: This road-map continues to evolve and subject to change without notice.</p></div>
<div class="sect2">
<h3 id="_here_is_a_smattering_of_ideas_for_megatest_2_0">Here is a smattering of ideas for Megatest 2.0</h3>
<div class="olist arabic"><ol class="arabic">
<li>
<p>
Add variable $MT_RUNPATH = $MT_LINKTREE/$MT_TARGET/$MT_RUNNAME
</p>
</li>
</ol></div>
<div class="listingblock">
<div class="content monospaced">
<pre>[db]
api legacy|new</pre>
</div></div>
<div class="olist arabic"><ol class="arabic">
<li>
<p>
One big lesson from the 1.63-1.65 generation was that the main.db, 1.db … model was really good at scaling. I’d like to combine that model with the current also-very-good model. Obviously this is a disruptive change. I think making the old model the default and the new model an option for at least one generation would be fair.
</p>
</li>
<li>
<p>
Rigorous megatest.config and runconfig.config caching.
</p>
<div class="olist loweralpha"><ol class="loweralpha">
<li>
<p>
Cache the configs in $MT_RUNPATH
</p>
</li>
<li>
<p>
Following invocations of –run, -rerun* will calculate the new config but only overwrite the cached file IF changed
</p>
</li>
</ol></div>
</li>
<li>
<p>
If the cached file changes ALL existing tests go from COMPLETED &#8594; STALE, I’m not sure what to do about RUNNING tests
</p>
</li>
<li>
<p>
!VARS in runconfigs are not exported to the environment. They are accessed via rget as if the ! was not there.
</p>
</li>
<li>
<p>
Per test copy commands (crude example below is not correct).
</p>
</li>
</ol></div>
<div class="listingblock">
<div class="content monospaced">
<pre>[testcopy]
%/iind% unison SRC DEST
% cp –r SRC DEST</pre>
</div></div>
<div class="olist arabic"><ol class="arabic">
<li>
<p>
Test management via pkts (optional?)
</p>
<div class="olist loweralpha"><ol class="loweralpha">
<li>
<p>
Control pkt types: run, kill, rerunclean, clean, archive, status?
</p>
</li>
<li>
<p>
Status pkt types: ack, step, status_change
</p>
</li>
</ol></div>
</li>
<li>
<p>
Add nanomsg as a transport option
.
</p>
</li>
</ol></div>
</div>
<div class="sect2">
<h3 id="_rfc_m01_add_ability_to_move_runs_to_other_areas">RFC M01: Add ability to move runs to other Areas</h3>
<div class="paragraph"><p><strong>Purpose</strong>: allow shrinking megatest.db data by moving runs to an alternate
 Megatest area with same keys.</p></div>
<div class="paragraph"><p><strong>Method</strong>: extend db sync to take a different megatest area as a destination.</p></div>
<div class="paragraph"><p><strong>Design</strong>:</p></div>
<div class="olist arabic"><ol class="arabic">
<li>
<p>
add param -destination [area|path]. when specified runs are copied to new
  area and removed from local db.
</p>
</li>
<li>
<p>
the data move would involve these steps
</p>
<div class="olist loweralpha"><ol class="loweralpha">
<li>
<p>
copy the run data to destination area megatest.db
</p>
</li>
<li>
<p>
mark the run records as deleted, do not remove the run data on disk
</p>
</li>
</ol></div>
</li>
<li>
<p>
accessing the data would be by running dashboard in the satellite area
</p>
</li>
<li>
<p>
future versions of Megatest dashboard should support displaying areas in a
  merged way.
</p>
</li>
<li>
<p>
some new controls would be supported in the config
</p>
<div class="olist loweralpha"><ol class="loweralpha">
<li>
<p>
[setup] &#8658; allow-runs [no|yes]  &#8656;= used to disallow runs
</p>
</li>
<li>
<p>
[setup] &#8658; auto-migrate=[areaname|path]  &#8656;= used to automatically
   migrate data to a satellite area.
</p>
</li>
</ol></div>
</li>
</ol></div>
<div class="paragraph"><p><strong>Branch</strong>: This work is taking place on branch v1.65-reduce-records</p></div>
</div>
<div class="sect2">
<h3 id="_rfc_m02_move_data_into_completed_runs_db">RFC M02: Move data into completed-runs.db</h3>
<div class="paragraph"><p><strong>Purpose</strong>: shrink megatest.db data to enable lower load and higher performance.</p></div>
<div class="paragraph"><p><strong>Method</strong>: add a completed-runs.db and automatically move runs data from megatest.db to that db</p></div>
<div class="paragraph"><p><strong>Design</strong>:</p></div>
<div class="olist arabic"><ol class="arabic">
<li>
<p>
completed-runs.db is a full megatest database with complete schema
</p>
</li>
<li>
<p>
the data move would involve these steps
</p>
<div class="olist loweralpha"><ol class="loweralpha">
<li>
<p>
copy the run data to completed-runs.db
</p>
</li>
<li>
<p>
remove the run data, first from /tmp/…/megatest.db and /tmp/…/megatest_ref.db, followed by megatest.db
</p>
</li>
</ol></div>
</li>
<li>
<p>
accessing the data would be unchanged for most operations.
</p>
</li>
<li>
<p>
a mode -full-db will be added which when specified would attach the completed-runs.db to megatest.db before doing the query
</p>
</li>
<li>
<p>
mechanisms for moving runs to/from the megatest.db would be added
</p>
<div class="olist loweralpha"><ol class="loweralpha">
<li>
<p>
-reduce-records &#8658; move runs to completed-runs.db
</p>
</li>
<li>
<p>
-restore-records &#8658; move runs from completed-runs.db to megatest.db
</p>
</li>
</ol></div>
</li>
</ol></div>
<div class="paragraph"><p><strong>Branch</strong>: This work is taking place on branch v1.65-reduce-records</p></div>
</div>
<div class="sect2">
<h3 id="_rfc_m03_automatic_homehost_migrations">RFC M03: Automatic homehost migrations</h3>
<div class="paragraph"><p><strong>Purpose</strong>: Automatically migrate homehost.</p></div>
<div class="paragraph"><p><strong>Method</strong>: Check that there are no tests running, launched or remotehoststart in past ½ hour then if not on homehost migrate the db to current host</p></div>
<div class="paragraph"><p><strong>Design</strong>:</p></div>
<div class="olist arabic"><ol class="arabic">
<li>
<p>
Check that the system is quiescent, i.e. that there are no runs in flight or recently run
</p>
</li>
<li>
<p>
Create a lock
</p>
</li>
<li>
<p>
Migrate the /tmp cache db to the current host
</p>
</li>
<li>
<p>
Update the .homehost file
</p>
</li>
<li>
<p>
Remove the lock
</p>
</li>
</ol></div>
<div class="paragraph"><p><strong>Branch</strong>: This work not yet started</p></div>
</div>
<div class="sect2">
<h3 id="_architecture_refactor">Architecture Refactor</h3>
<div class="sect3">
<h4 id="_goals">Goals</h4>
<div class="olist arabic"><ol class="arabic">
<li>
<p>
Reduce load on the file system. Sqlite3 files on network filesystem can be
  a burden. <span class="green">[DONE]</span>
</p>
</li>
<li>
<p>
Reduce number of servers and frequency of start/stop. This is mostly an
  issue of clutter but also a reduction in "moving parts". <span class="green">[DONE]</span>
</p>
</li>
<li>
<p>
Coalesce activities to a single home host where possible. Give the user
  feedback that they have started the dashboard on a host other than the
  home host. <span class="green">[DONE]</span>
</p>
</li>
<li>
<p>
Reduce number of processes involved in managing running tests.
</p>
</li>
</ol></div>
</div>
<div class="sect3">
<h4 id="_changes_needed">Changes Needed</h4>
<div class="olist arabic"><ol class="arabic">
<li>
<p>
ACID compliant db will be on /tmp and synced to megatest.db with a five
  second max delay. <span class="green">[DONE]</span>
</p>
</li>
<li>
<p>
Read/writes to db for processes on homehost will go direct to /tmp
  megatest.db file. <span class="green">[DONE]</span>
</p>
</li>
<li>
<p>
Read/wites fron non-homehost processes will go through one server. Bulk
  reads (e.g. for dashboard or list-runs) will be cached on the current host
  in /tmp and synced from the home megatest.db in the testsuite area. <span class="green">[DONE]</span>
</p>
</li>
<li>
<p>
Db syncs rely on the target db file timestame minus some margin. <span class="green">[DONE]</span>
</p>
</li>
<li>
<p>
Since bulk reads do not use the server we can switch to simple RPC for the
  network transport. <span class="green">[DONE]</span>
</p>
</li>
<li>
<p>
Test running manager process extended to manage multiple running tests.
</p>
</li>
</ol></div>
</div>
</div>
<div class="sect2">
<h3 id="_current_items">Current Items</h3>
<div class="sect3">
<h4 id="_ww05_migrate_to_inmem_db">ww05 - migrate to inmem-db</h4>
<div class="olist arabic"><ol class="arabic">
<li>
<p>
Switch to inmem db with fast sync to on disk db&#8217;s <span class="green">[DONE]</span>
</p>
</li>
<li>
<p>
Server polls tasks table for next action
</p>
<div class="olist loweralpha"><ol class="loweralpha">
<li>
<p>
Task table used for tracking runner process <span class="red">[Replaced by mtutil]</span>
</p>
</li>
<li>
<p>
Task table used for jobs to run <span class="red">[Replaced by mtutil]</span>
</p>
</li>
<li>
<p>
Task table used for queueing runner actions (remove runs,
   cleanRunExecute, etc)  <span class="red">[Replaced by mtutil</span>]
</p>
</li>
</ol></div>
</li>
</ol></div>
<div class="paragraph"><p>shifting, note that the preceding blank line is needed.</p></div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_index">Index</h2>
<div class="sectionbody">
</div>
</div>
</div>
<div id="footnotes"><hr></div>
<div id="footer">
<div id="footer-text">
Version 1.0<br>
Last updated 2018-11-29 09:32:52 PST
</div>
</div>
</body>
</html>