Megatest

Diff
Login

Differences From Artifact [bb97544027]:

To Artifact [2589d78a96]:


53
54
55
56
57
58
59
60
61



62
63
64


65
66
67
68
69


70
71


72
73
74


75
76



77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97





98
99
100
101
102
103
104
105
106
107
108
109
110

111
112
113
114
115
116
117
53
54
55
56
57
58
59


60
61
62
63


64
65
66
67
68
69

70
71
72

73
74
75


76
77
78

79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99



100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125







-
-
+
+
+

-
-
+
+




-
+
+

-
+
+

-
-
+
+

-
+
+
+


















-
-
-
+
+
+
+
+













+







resources to make writing a suite of tests and tasks for implementing
continuous build for software, design engineering or process control (via
owlfs for example) without being specialized for any specific problem
space. Megatest in of itself does not know what constitutes a PASS or FAIL
of a test or task. In most cases megatest is best used in conjunction with
logpro or a similar tool to parse, analyze and decide on the test outcome.

 * Self-checking -Repeatable strive for directed or self-checking test
   as opposed to delta based tests.
 * Self-checking - make it as easy as possible to write self-checking
   tests (as opposed to using deltas, i.e. tests that compare with a
   previous measurement to deterine PASS/FAIL).

 * Traceable - environment variables, host OS and other possibly influential
   variables are captured and kept recorded.
 * Traceable - environment variables, host OS and other possibly
   influential variables are captured and kept recorded.

 * Immutable - once a test is run it cannot be easily overwritten or
   modified accidentally.

 * Repeatable - test results can be recreated in the future using all the original variables.
 * Repeatable - test results can be recreated in the future using all
   the original variables.

 * Relocatable - the testsuite or automation area can be checked out and the tests run anywhere in the disk hierarchy.
 * Relocatable - the testsuite or automation area can be checked out
   and the tests run anywhere in the disk hierarchy.

 * Encapsulated - the tests run in self-contained directories and all inputs
   and outputs to the process can be found in the run areas.
 * Encapsulated - the tests run in self-contained directories and all
   inputs and outputs to the process can be found in the run areas.

 * Deployable - a testsuite is self-contained and can be bundled with a software project.
 * Deployable - a testsuite is self-contained and can be bundled with
   a software project and easily used by others with little to no
   setup burden.

Megatest Architecture
---------------------

Data separation
~~~~~~~~~~~~~~~

All data to specify the tests and configure the system is stored in
plain text config files. All system state is stored in an sqlite3
database.

Distributed Compute
~~~~~~~~~~~~~~~~~~~

Tests are launched using the launching system available for the
distributed compute platform in use. A template script is provided
which can launch jobs on local and remote Linux hosts. Currently
megatest uses the network filesystem to call home to your master
sqlite3 database. Megatest has been used with the Intel Netbatch
system, the lsf batch system and it should be possible to use it with
other similar systems.
sqlite3 database. Megatest has been used with the Intel Netbatch and
lsf (also known as openlava) batch systems and it should be
straightforward to use it with other similar systems.

include::overview.txt[]

include::installation.txt[]

include::getting_started.txt[]

include::study_plan.txt[]

// :leveloffset: 0

include::writing_tests.txt[]
include::howto.txt[]

include::reference.txt[]


Megatest Internals
------------------

["graphviz", "server.png"]
----------------------------------------------------------------------
include::server.dot[]