Megatest

Check-in [04bf62110e]
Login
☰Home Timeline Files Branches Tags Tickets Wiki Login
Overview
Comment:Added starting docs for dashboard
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA1: 04bf62110e86e221f9c44fc32fcef1e082aa479a
User & Date: matt on 2011-09-21 00:12:56
Other Links: manifest | tags
Context
2011-09-21
00:45
Adding html file as .wiki check-in: 513285e117 user: matt tags: trunk
00:12
Added starting docs for dashboard check-in: 04bf62110e user: matt tags: trunk
2011-09-20
23:23
Move docs to new template check-in: 58d65d4e70 user: matt tags: trunk
Changes
Hide Diffs Unified Diffs Ignore Whitespace Patch

Modified dashboard.scm from [1ae7a38415] to [06f47be6c4].

︙︙
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

(include "gui.scm")
(include "dashboard-tests.scm")
(include "megatest-version.scm")

(define help (conc 
"Megatest Dashboard, documentation at http://www.kiatoa.com/fossils/megatest
  version " megatest-version "
  license GPL, Copyright Matt Welland 2011

Usage: dashboard [options]
  -h              : this help
  -run runid      : control run identified by runid
  -test testid    : control test identified by testid

Misc








|








31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

(include "gui.scm")
(include "dashboard-tests.scm")
(include "megatest-version.scm")

(define help (conc 
"Megatest Dashboard, documentation at http://www.kiatoa.com/fossils/megatest
  version " megatest-version "
  license GPL, Copyright (C) Matt Welland 2011

Usage: dashboard [options]
  -h              : this help
  -run runid      : control run identified by runid
  -test testid    : control test identified by testid

Misc
︙︙

Added docs/dashboard-test.png version [14d4ee0830].

cannot compute difference between binary files

Added docs/dashboard.png version [5ee6b0359b].

cannot compute difference between binary files

Modified docs/megatest.html from [1024506222] to [50f0ba7bf3]. 

1
2
3
4
5
6
7
8
9
10
11
12









13
14
15
16
17

18
19
20
21












22
23








24








25
26
















































27
28
29
30
31
32
33
34
35
36
37
38
39

40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103

104
105
106
107
108
109
110

111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128


129

130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149

150
151
152
153
154
155
156
157



158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178

179


180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
<meta name="generator" content="http://www.nongnu.org/elyxer/"/>
<meta name="create-date" content="2011-08-03"/>
<link rel="stylesheet" href="http://www.nongnu.org/elyxer/lyx.css" type="text/css" media="screen"/>
<title>Megatest</title>
</head>
<body>
<div id="globalWrapper">









<h1 class="title">
Megatest
</h1>
<h2 class="Date">
5/5/2011 

</h2>
<h2 class="author">
Matt Welland
</h2>












<div class="abstract">
<p class="abstract-message">








Abstract








</p>
Megatest is a minimalistic tool to help automate and manage the running of a suite of regression tests or other processes and to roll up the results for display on a web page or dashboard.
















































</div>
<div class="fulltoc">
<div class="tocheader">
Table of Contents
</div>
<div class="tocindent">
<div class="toc">
<a class="Link" href="#toc-Chapter-1">Chapter 1: About Megatest</a>
</div>
<div class="tocindent">
<div class="toc">
<a class="Link" href="#toc-Section-1.1">Section 1.1: Megatest design philosophy</a>
</div>

<div class="toc">
<a class="Link" href="#toc-Section-1.2">Section 1.2: Megatest architecture</a>
</div>
</div>
<div class="toc">
<a class="Link" href="#toc-Chapter-2">Chapter 2: Installation</a>
</div>
<div class="tocindent">
<div class="toc">
<a class="Link" href="#toc-Section-2.1">Section 2.1: Compilation</a>
</div>
<div class="tocindent">
<div class="toc">
<a class="Link" href="#toc-Subsection-2.1.1">Subsection 2.1.1: Dependencies</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-2.1.2">Subsection 2.1.2: Build and install</a>
</div>
</div>
<div class="toc">
<a class="Link" href="#toc-Section-2.2">Section 2.2: Setup</a>
</div>
<div class="tocindent">
<div class="toc">
<a class="Link" href="#toc-Subsection-2.2.1">Subsection 2.2.1: Create megatest.config</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-2.2.2">Subsection 2.2.2: Create runconfigs.config</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-2.2.3">Subsection 2.2.3: Create the tests directory and your first test</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-2.2.4">Subsection 2.2.4: Create the testconfig file for your test</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-2.2.5">Subsection 2.2.5: Create your test running script, main.sh</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-2.2.6">Subsection 2.2.6: Run megatest and watch your run progress</a>
</div>
</div>
</div>
<div class="toc">
<a class="Link" href="#toc-Chapter-3">Chapter 3: How to Write Tests</a>
</div>
<div class="tocindent">
<div class="toc">
<a class="Link" href="#toc-Section-3.1">Section 3.1: A Simple Test with one Step</a>
</div>
<div class="tocindent">
<div class="toc">
<a class="Link" href="#toc-Subsection-3.1.1">Subsection 3.1.1: Create your test directory. The directory name will be the name of the test</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-3.1.2">Subsection 3.1.2: Create your testconfig file. This file contains various specifications for your test. For our example the test author has chosen to write the test using csh scripting.</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-3.1.3">Subsection 3.1.3: Create your main.csh script</a>
</div>
</div>
<div class="toc">
<a class="Link" href="#toc-Section-3.2">Section 3.2: Simple Test with Multiple Steps </a>
</div>

<div class="toc">
<a class="Link" href="#toc-Section-3.3">Section 3.3: Simple Test with Multiple Steps, Some in Parallel </a>
</div>
<div class="tocindent">
<div class="toc">
<a class="Link" href="#toc-Subsection-3.3.1">Subsection 3.3.1: The Makefile</a>
</div>

<div class="toc">
<a class="Link" href="#toc-Subsection-3.3.2">Subsection 3.3.2: The main.csh file</a>
</div>
</div>
<div class="toc">
<a class="Link" href="#toc-Section-3.4">Section 3.4: Simple Test with Iteration</a>
</div>
<div class="tocindent">
<div class="toc">
<a class="Link" href="#toc-Subsection-3.4.1">Subsection 3.4.1: Update you testconfig file for iteration</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-3.4.2">Subsection 3.4.2: Rewrite your main.csh for iteration</a>
</div>
</div>
<div class="toc">
<a class="Link" href="#toc-Section-3.5">Section 3.5: Tests with Inter-test dependencies</a>
</div>


</div>

<div class="toc">
<a class="Link" href="#toc-Chapter-4">Chapter 4: Reference</a>
</div>
<div class="tocindent">
<div class="toc">
<a class="Link" href="#toc-Section-4.1">Section 4.1: Configuration file Syntax</a>
</div>
<div class="tocindent">
<div class="toc">
<a class="Link" href="#toc-Subsection-4.1.1">Subsection 4.1.1: Sections</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-4.1.2">Subsection 4.1.2: Variables</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-4.1.3">Subsection 4.1.3: Includes</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-4.1.4">Subsection 4.1.4: Setting a variable by running a command</a>
</div>

<div class="toc">
<a class="Link" href="#toc-Subsection-4.1.5">Subsection 4.1.5: Notes</a>
</div>
</div>
<div class="toc">
<a class="Link" href="#toc-Section-4.2">Section 4.2: Environment variables</a>
</div>
<div class="toc">



<a class="Link" href="#toc-Section-4.3">Section 4.3: Configuration files</a>
</div>
<div class="tocindent">
<div class="toc">
<a class="Link" href="#toc-Subsection-4.3.1">Subsection 4.3.1: megatest.config</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-4.3.2">Subsection 4.3.2: runconfigs.config file</a>
</div>
</div>
<div class="toc">
<a class="Link" href="#toc-Section-4.4">Section 4.4: Writing tests</a>
</div>
<div class="tocindent">
<div class="toc">
<a class="Link" href="#toc-Subsection-4.4.1">Subsection 4.4.1: testconfig file</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-4.4.2">Subsection 4.4.2: Command line</a>
</div>
</div>

</div>


</div>

</div>
<h1 class="Chapter">
<a class="toc" name="toc-Chapter-1">1</a> About Megatest
</h1>
<h2 class="Section">
<a class="toc" name="toc-Section-1.1">1.1</a> Megatest design philosophy
</h2>
<div class="Unindented">
Megatest is intended to provide the minimum needed resources to make writing a suite of tests for software, design engineering or process control (via owlfs for example) without being specialized for any specific problem space. Megatest in of itself does not know what constitutes a PASS or FAIL of a test. In most cases megatest is best used in conjunction with logpro or a similar tool to parse, analyze and decide on the test outcome. A call to megatest can then be made to record the result. 
</div>
<h2 class="Section">
<a class="toc" name="toc-Section-1.2">1.2</a> Megatest architecture
</h2>
<div class="Unindented">
All data to specify the tests and configure the system is stored in plain text files. All system state is stored in an sqlite3 database. Tests are launched using the launching system available for the distributed compute platform in use. A template script is provided which can launch jobs on local and remote Linux hosts. Currently megatest uses the network filesystem to “call home” to your master sqlite3 database.
</div>
<h1 class="Chapter">
<a class="toc" name="toc-Chapter-2">2</a> Installation
</h1>
<h2 class="Section">
<a class="toc" name="toc-Section-2.1">2.1</a> Compilation
</h2>
<h3 class="Subsection">
<a class="toc" name="toc-Subsection-2.1.1">2.1.1</a> Dependencies
</h3>
<div class="Unindented">
Chicken scheme and a number of eggs are required for building megatest. See the file utils/installall.sh for an automated way to install the dependencies on Linux.
</div>
<h3 class="Subsection">
<a class="toc" name="toc-Subsection-2.1.2">2.1.2</a> Build and install
</h3>
<div class="Unindented">
Run “make test” to create the megatest executable. You may wish to copy the executable to a centrally accessible location.
</div>
<h2 class="Section">
<a class="toc" name="toc-Section-2.2">2.2</a> Setup
</h2>
<h3 class="Subsection">
<a class="toc" name="toc-Subsection-2.2.1">2.2.1</a> Create megatest.config
</h3>
<div class="Unindented">
Create the file megatest.config using the megatest.config template from the tests directory. At a minimum you need the following:
</div>
<div class="Paragraph">
<a class="toc" name="toc-Paragraph-1"></a><div class="listing">
<pre class="listing"># Fields are the keys under which your test runs are organized
[fields]
field1 TEXT
field2 TEXT
​
[jobtools]
# The launcher launches jobs to the local or remote hosts,







|

|



>
>
>
>
>
>
>
>
>

|

|
<
>

|
|

>
>
>
>
>
>
>
>
>
>
>
>
|
|
>
>
>
>
>
>
>
>
|
>
>
>
>
>
>
>
>

|
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







|

<

|

>

|

<

|

|

|



|


|



|



|


|


|


|


|


|


<

|



|

<

|


|


|



|

>

|

<

|

>

|

|

|

<

|


|



|

>
>

>

|



|

<

|


|


|


|

>

|

<

|


>
>
>
|



|


|


<
<

<

|


|


>

>
>
|


|
|

|
|

|


|
|

|


<
<
<
|
|
|
|
|
|
|
|

|
|
|
|


|
|
|
|
|
|
|


|
|








1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120

121
122
123
124
125
126
127

128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165

166
167
168
169
170
171
172

173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189

190
191
192
193
194
195
196
197
198
199
200

201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221

222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237

238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254


255

256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284



285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
<meta name="generator" content="http://www.nongnu.org/elyxer/"/>
<meta name="create-date" content="2011-09-20"/>
<link rel="stylesheet" href="http://www.nongnu.org/elyxer/lyx.css" type="text/css" media="screen"/>
<title>Converted document</title>
</head>
<body>
<div id="globalWrapper">
<div class="Standard">

</div>
<div class="Standard">

</div>
<div class="Standard">

</div>
<h1 class="title">

</h1>
<h2 class="author">

Matthew Welland
</h2>
<h2 class="Date">
Sept. 20, 
</h2>
<div class="Standard">

</div>
<div class="Standard">
<p><br/>
</p>
2011 Matthew Welland. All rights reserved. 
</div>
<div class="Standard">
Megatest is free software released under the General Public License v2.0. Please see the file COPYING in the source distribution for details. 
</div>
<div class="Standard">
<div class="medskip">

</div>
Email: matt@kiatoa.com. 
</div>
<div class="Standard">
Web: www.kiatoa.com/fossils/megatest
</div>
<div class="Standard">
<div class="medskip">

</div>
This document is believed to be acurate at the time of writing but as with any opensource project the source code itself is the final arbiter of the softwares behaviour. It is the responsibility of the end user to validate that the code will perform as they expect. The author assumes no responsibility for any inaccuracies that this document may contain. In no event will Matthew Welland be liable for direct, indirect, special, exemplary, incidental, or consequential damages resulting from any defect or omission in this document, even if advised of the possibility of such damages. 
</div>
<div class="Standard">
This document is a snapshot in time and the Megatest software has likely been changed since publication. This document and the product that it describes may be improved at any time, without notice or obligation. 
</div>
<div class="Standard">
<p><br/>
</p>

</div>
<h1 class="Section-">
<a class="toc" name="toc-Section--1"></a>Document Revision History
</h1>
<div class="Standard">
<i>Notable revisions of the software are occasionally documented here</i>.
</div>
<div class="Standard">
<div class="center">
<table>
<tr>
<td align="center" valign="top">
Version
</td>
<td align="center" valign="top">
Author
</td>
<td align="justify" valign="top">
Description
</td>
<td align="center" valign="top">
Date
</td>

</tr>
<tr>
<td align="center" valign="top">
v1.25
</td>
<td align="center" valign="top">
matt
</td>
<td align="justify" valign="top">
converted to new document format
</td>
<td align="center" valign="top">
<div class="PlainVisible">

</div>

</td>

</tr>

</table>

</div>

</div>
<div class="fulltoc">
<div class="tocheader">
Table of Contents
</div>
<div class="tocindent">
<div class="toc">
<a class="Link" href="#toc-Section--1">Section: Document Revision History</a>
</div>

<div class="toc">
<a class="Link" href="#toc-Section-1">Section 1: Introduction</a>
</div>
<div class="tocindent">
<div class="toc">
<a class="Link" href="#toc-Subsection-1.1">Subsection 1.1: Megatest design philosophy</a>
</div>

<div class="toc">
<a class="Link" href="#toc-Subsection-1.2">Subsection 1.2: Megatest architecture</a>
</div>
</div>
<div class="toc">
<a class="Link" href="#toc-Section-2">Section 2: Installation</a>
</div>
<div class="tocindent">
<div class="toc">
<a class="Link" href="#toc-Subsection-2.1">Subsection 2.1: Dependencies</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-2.2">Subsection 2.2: Build and Install</a>
</div>
</div>
<div class="toc">
<a class="Link" href="#toc-Section-3">Section 3: Setup</a>
</div>
<div class="tocindent">
<div class="toc">
<a class="Link" href="#toc-Subsection-3.1">Subsection 3.1: Create megatest.config</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-3.2">Subsection 3.2: Create runconfigs.config</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-3.3">Subsection 3.3: Create the tests directory and your first test</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-3.4">Subsection 3.4: Create the testconfig file for your test</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-3.5">Subsection 3.5: Create your test running script, main.sh</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-3.6">Subsection 3.6: Run megatest and watch your run progress</a>
</div>
</div>

<div class="toc">
<a class="Link" href="#toc-Section-4">Section 4: How to Write Tests</a>
</div>
<div class="tocindent">
<div class="toc">
<a class="Link" href="#toc-Subsection-4.1">Subsection 4.1: A Simple Test with one Step</a>
</div>

<div class="toc">
<a class="Link" href="#toc-Subsection-4.2">Subsection 4.2: Create your testconfig file</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-4.3">Subsection 4.3: Create the main.csh script</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-4.4">Subsection 4.4: Simple Test with Multiple Steps </a>
</div>
</div>
<div class="toc">
<a class="Link" href="#toc-Section-5">Section 5: Simple Test with Multiple Steps, Some in Parallel</a>
</div>
<div class="tocindent">
<div class="toc">
<a class="Link" href="#toc-Subsection-5.1">Subsection 5.1: The Makefile</a>
</div>

<div class="toc">
<a class="Link" href="#toc-Subsection-5.2">Subsection 5.2: The main.csh file</a>
</div>
</div>
<div class="toc">
<a class="Link" href="#toc-Section-6">Section 6: Simple Test with Iteration</a>
</div>
<div class="tocindent">
<div class="toc">
<a class="Link" href="#toc-Subsection-6.1">Subsection 6.1: Update your testconfig file for iteration</a>
</div>

<div class="toc">
<a class="Link" href="#toc-Subsection-6.2">Subsection 6.2: Rewrite your main.csh for iteration</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-6.3">Subsection 6.3: Tests with Inter-test dependencies</a>
</div>
</div>
<div class="toc">
<a class="Link" href="#toc-Section-7">Section 7: Dashboard</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Section-8">Section 8: Reference</a>
</div>
<div class="tocindent">
<div class="toc">
<a class="Link" href="#toc-Subsection-8.1">Subsection 8.1: Configuration file Syntax</a>
</div>
<div class="tocindent">
<div class="toc">
<a class="Link" href="#toc-Subsubsection-8.1.1">Subsubsection 8.1.1: Sections</a>
</div>

<div class="toc">
<a class="Link" href="#toc-Subsubsection-8.1.2">Subsubsection 8.1.2: Variables</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsubsection-8.1.3">Subsubsection 8.1.3: Includes</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsubsection-8.1.4">Subsubsection 8.1.4: Setting a variable by running a command</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsubsection-8.1.5">Subsubsection 8.1.5: Notes</a>
</div>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-8.2">Subsection 8.2: Environment variables</a>
</div>

<div class="toc">
<a class="Link" href="#toc-Subsection-8.3">Subsection 8.3: megatest.config</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-8.4">Subsection 8.4: runconfigs.config file</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-8.5">Subsection 8.5: Writing tests</a>
</div>
<div class="tocindent">
<div class="toc">
<a class="Link" href="#toc-Subsubsection-8.5.1">Subsubsection 8.5.1: testconfig file</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsubsection-8.5.2">Subsubsection 8.5.2: Command line</a>
</div>
</div>


</div>

<div class="toc">
<a class="Link" href="#toc-Section-A">Section A: Data</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Section-B">Section B: References</a>
</div>
</div>

</div>
<div class="Standard">
<p><br/>
</p>

</div>
<h1 class="Section">
<a class="toc" name="toc-Section-1">1</a> Introduction
</h1>
<h2 class="Subsection">
<a class="toc" name="toc-Subsection-1.1">1.1</a> Megatest design philosophy
</h2>
<div class="Standard">
Megatest is intended to provide the minimum needed resources to make writing a suite of tests for software, design engineering or process control (via owlfs for example) without being specialized for any specific problem space. Megatest in of itself does not know what constitutes a PASS or FAIL of a test. In most cases megatest is best used in conjunction with logpro or a similar tool to parse, analyze and decide on the test outcome. A call to megatest can then be made to record the result. 
</div>
<h2 class="Subsection">
<a class="toc" name="toc-Subsection-1.2">1.2</a> Megatest architecture
</h2>
<div class="Standard">
All data to specify the tests and configure the system is stored in plain text files. All system state is stored in an sqlite3 database. Tests are launched using the launching system available for the distributed compute platform in use. A template script is provided which can launch jobs on local and remote Linux hosts. Currently megatest uses the network filesystem to “call home” to your master sqlite3 database.
</div>



<h1 class="Section">
<a class="toc" name="toc-Section-2">2</a> Installation
</h1>
<h2 class="Subsection">
<a class="toc" name="toc-Subsection-2.1">2.1</a> Dependencies
</h2>
<div class="Standard">
Chicken scheme and a number of “eggs” are required for building megatest. See the file utils/installall.sh for an automated way to install the dependencies on Linux.
</div>
<h2 class="Subsection">
<a class="toc" name="toc-Subsection-2.2">2.2</a> Build and Install
</h2>
<div class="Standard">
Run “make test” to create the megatest executable. You may wish to copy the executable to a centrally accessible location.
</div>
<h1 class="Section">
<a class="toc" name="toc-Section-3">3</a> Setup
</h1>
<h2 class="Subsection">
<a class="toc" name="toc-Subsection-3.1">3.1</a> Create megatest.config
</h2>
<div class="Standard">
Create the file megatest.config using the megatest.config template from the tests directory. At a minimum you need the following:
</div>
<div class="Standard">
<div class="listing">
<pre class="listing"># Fields are the keys under which your test runs are organized
[fields]
field1 TEXT
field2 TEXT
​
[jobtools]
# The launcher launches jobs to the local or remote hosts,
︙︙
243
244
245
246
247
248
249
250
251
252
253
254



255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431

432


433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450

451


452
453
454
455
456
457
458
459
460
461
462
463


464





465



466

467





468



469
470
471
472
473
474
475


476
477
478
479
480















481
482
483
484

485



486



487



488
489
490
491

492


493
494
495








496
497



498
499
500
501

502
503

504
505


506
507
508
509
510

511
512

513
514


515
516
517
518
519

520
521

522
523


524
525
526
527
528

529
530

531
532


533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677

# name /path/to/area 
[disks]
disk1 /tmp 
</pre>
</div>

</div>
<h3 class="Subsection">
<a class="toc" name="toc-Subsection-2.2.2">2.2.2</a> Create runconfigs.config
</h3>
<div class="Unindented">
This file is used to set environment variables that are run specific. You can simply create an empty file to start.<div class="listing">



<pre class="listing"># runconfigs.config
​
</pre>
</div>

</div>
<h3 class="Subsection">
<a class="toc" name="toc-Subsection-2.2.3">2.2.3</a> Create the tests directory and your first test
</h3>
<div class="Unindented">
The structure should look like this:
</div>
<div class="Paragraph">
<a class="toc" name="toc-Paragraph-2"></a><div class="listing">
<pre class="listing">../tests
  ├── megatest.config
  ├── runconfigs.config
  └── tests
     └── mytest
           ├── main.sh
           └── testconfig
</pre>
</div>

</div>
<h3 class="Subsection">
<a class="toc" name="toc-Subsection-2.2.4">2.2.4</a> Create the testconfig file for your test
</h3>
<div class="Paragraph">
<a class="toc" name="toc-Paragraph-3"></a><div class="listing">
<pre class="listing">[setup]
runscript main.sh
</pre>
</div>

</div>
<h3 class="Subsection">
<a class="toc" name="toc-Subsection-2.2.5">2.2.5</a> Create your test running script, main.sh
</h3>
<div class="Paragraph">
<a class="toc" name="toc-Paragraph-4"></a><div class="listing">
<pre class="listing">#!/bin/bash
​
$MT_MEGATEST -runstep mystep1 "sleep 20;echo Done" -m "mystep1 is done"
​
$MT_MEGATEST -test-status :state COMPLETED :status PASS -m "This is a comment"
</pre>
</div>

</div>
<h3 class="Subsection">
<a class="toc" name="toc-Subsection-2.2.6">2.2.6</a> Run megatest and watch your run progress
</h3>
<div class="Paragraph">
<a class="toc" name="toc-Paragraph-5"></a><div class="listing">
<pre class="listing">megatest :field1 abc :field2 def :runname 2011week08.4a -runall
​
watch megatest -list-runs %
​
# OR use the dashboard
​
dashboard &amp;
</pre>
</div>

</div>
<div class="Code">

</div>
<h1 class="Chapter">
<a class="toc" name="toc-Chapter-3">3</a> How to Write Tests
</h1>
<h2 class="Section">
<a class="toc" name="toc-Section-3.1">3.1</a> A Simple Test with one Step
</h2>
<h3 class="Subsection">
<a class="toc" name="toc-Subsection-3.1.1">3.1.1</a> Create your test directory. The directory name will be the name of the test
</h3>
<div class="Paragraph">
<a class="toc" name="toc-Paragraph-6"></a><div class="listing">
<pre class="listing">mkdir simpletest
cd simpletest
</pre>
</div>

</div>
<h3 class="Subsection">
<a class="toc" name="toc-Subsection-3.1.2">3.1.2</a> Create your testconfig file. This file contains various specifications for your test. For our example the test author has chosen to write the test using csh scripting.
</h3>
<div class="Paragraph">
<a class="toc" name="toc-Paragraph-7"></a><div class="listing">
<pre class="listing"># testconfig
​
[setup]
runscript main.csh
</pre>
</div>

</div>
<h3 class="Subsection">
<a class="toc" name="toc-Subsection-3.1.3">3.1.3</a> Create your main.csh script
</h3>
<div class="Unindented">
Note: Using csh is NOT recommended. Use bash, perl, ruby, zsh or anything other than csh. We use csh here because it is popular in some industries. 
</div>
<div class="Paragraph">
<a class="toc" name="toc-Paragraph-8"></a><div class="listing">
<pre class="listing">#!/bin/tcsh -x
​
# run the cpu1 simulation.
#   The step name is "run_simulation"
#   The commandline being run for this step is "runsim cpu1"
#   The logpro file to validate the output from the run is "runsim.logpro"
​
$MT_MEGATEST -runstep run_simulation -logpro runsim.logpro "runsim cpu1"
$MT_MEGATEST -test-status :state COMPLETED :status $?
</pre>
</div>

</div>
<div class="Unindented">
You can now run megatest and the created test directory will contain the new files “run_simulation.html” and “run_simulation.log”. If you are using the dashboard you can click on the run and then push the “View log” button to view the log file in firefox.
</div>
<h2 class="Section">
<a class="toc" name="toc-Section-3.2">3.2</a> Simple Test with Multiple Steps 
</h2>
<div class="Unindented">
To run multiple steps simply add them to the main.csh file. Here we add a step to test “cpu2”. The second step that tests cpu2 will only run after the step that tested “cpu1” completes.
</div>
<div class="Paragraph">
<a class="toc" name="toc-Paragraph-9"></a><div class="listing">
<pre class="listing">#!/bin/tcsh -x
​
# run the cpu1 simulation.
#   The step name is "run_simulation"
#   The commandline being run for this step is "runsim cpu1"
#   The logpro file to validate the output from the run is "runsim.logpro"
​
$MT_MEGATEST -runstep run_simulation_cpu1 -logpro runsim.logpro "runsim cpu1" &amp;&amp; \
$MT_MEGATEST -runstep run_simulation_cpu2 -logpro runsim.logpro "runsim cpu2"
$MT_MEGATEST -test-status :state COMPLETED :status $?
</pre>
</div>

</div>
<h2 class="Section">
<a class="toc" name="toc-Section-3.3">3.3</a> Simple Test with Multiple Steps, Some in Parallel 
</h2>
<h3 class="Subsection">
<a class="toc" name="toc-Subsection-3.3.1">3.3.1</a> The Makefile
</h3>
<div class="Unindented">
A good way to run steps in parallel within a single test, especially when there are following steps, is to use the Unix Make utility. Writing Makefiles is beyond the scope of this document but here is a minimal example that will run “runsim cpu1” and “runsim cpu2” in parallel. For more information on make try “info make” at the Linux command prompt.
</div>
<div class="Paragraph">
<a class="toc" name="toc-Paragraph-10"></a><div class="listing">
<pre class="listing"># Example Makefile to run two steps in parallel
​
RTLDIR=/path/to/rtl
CPUS = cpu1 cpu2
​
run_simulation_$(CPUS).html : $(RTLDIR)/$(CPUS)
	$(MT_MEGATEST) -runstep run_simulation_$(CPUS) -logpro runsim.logpro "runsim $(CPUS)
</pre>
</div>

</div>
<h3 class="Subsection">
<a class="toc" name="toc-Subsection-3.3.2">3.3.2</a> The main.csh file
</h3>
<div class="Unindented">
<div class="listing">
<pre class="listing">#!/bin/tcsh -x
​
# run the cpu1 and cpu2 simulations in parallel. 
# The -j parameter tells make how many jobs it may run in parallel
​

make -j 2 


$MT_MEGATEST -test-status :state COMPLETED :status $?
</pre>
</div>

</div>
<h2 class="Section">
<a class="toc" name="toc-Section-3.4">3.4</a> Simple Test with Iteration
</h2>
<div class="Unindented">
Since no jobs run after the cpu1 and cpu2 simulations in this test it is possible to use iterated mode.
</div>
<h3 class="Subsection">
<a class="toc" name="toc-Subsection-3.4.1">3.4.1</a> Update you testconfig file for iteration
</h3>
<div class="Paragraph">
<a class="toc" name="toc-Paragraph-11"></a><div class="listing">
<pre class="listing">[setup]
runscript main.csh

​


[items]
CPU cpu1 cpu2
</pre>
</div>

</div>
<h3 class="Subsection">
<a class="toc" name="toc-Subsection-3.4.2">3.4.2</a> Rewrite your main.csh for iteration
</h3>
<div class="Paragraph">
<a class="toc" name="toc-Paragraph-12"></a><div class="listing">
<pre class="listing">#!/bin/tcsh -x


​





# run the cpu simulation but now use the environment variable $CPU



# to select what cpu to run the simulation against

​





$MT_MEGATEST -runstep run_simulation -logpro runsim.logpro "runsim $CPU"



# As of version 1.07 Megatest automatically converts a status of "0"
# to "PASS", any other number to "FAIL" and directly uses the value of
# a string passed in.
$MT_MEGATEST -test-status :state COMPLETED :status $?
</pre>
</div>



</div>
<h2 class="Section">
<a class="toc" name="toc-Section-3.5">3.5</a> Tests with Inter-test dependencies
</h2>
<div class="Unindented">















Sometimes a test depends on the output from a previous test or it may not make sense to run a test is another test does not complete with status “PASS”. In either of these scenarios you can use the “waiton” keyword in your testconfig file to indicate that this test must wait on one or more tests to complete before being launched. In this example there is no point in running the “system” test if the “cpu” and “mem” tests either do not complete or complete but with status “FAIL”.
</div>
<div class="Paragraph">
<a class="toc" name="toc-Paragraph-13"></a><div class="listing">

<pre class="listing"># testconfig for the "system" test



[setup]



runscript main.csh



waiton cpu mem
</pre>
</div>


</div>


<h1 class="Chapter">
<a class="toc" name="toc-Chapter-4">4</a> Reference
</h1>








<h2 class="Section">
<a class="toc" name="toc-Section-4.1">4.1</a> Configuration file Syntax



</h2>
<h3 class="Subsection">
<a class="toc" name="toc-Subsection-4.1.1">4.1.1</a> Sections
</h3>

<div class="Unindented">
[section name]

</div>
<blockquote class="Quotation">


This creates a section named “section name”
</blockquote>
<h3 class="Subsection">
<a class="toc" name="toc-Subsection-4.1.2">4.1.2</a> Variables
</h3>

<div class="Unindented">
VARX has this value

</div>
<blockquote class="Quotation">


The variable “VARX” will have the value “has this value”
</blockquote>
<h3 class="Subsection">
<a class="toc" name="toc-Subsection-4.1.3">4.1.3</a> Includes
</h3>

<div class="Unindented">
[include filename]

</div>
<blockquote class="Quotation">


The file named “filename” will be included as if part of the calling file. NOTE: This means no section can be named “include “ (with the whitespace).
</blockquote>
<h3 class="Subsection">
<a class="toc" name="toc-Subsection-4.1.4">4.1.4</a> Setting a variable by running a command
</h3>

<div class="Unindented">
VARNAME [system ls /tmp]

</div>
<blockquote class="Quotation">


The variable “VARNAME” will get a value created by the Unix command “ls /tmp”. All lines of output from the command will be joined with a space.
</blockquote>
<h3 class="Subsection">
<a class="toc" name="toc-Subsection-4.1.5">4.1.5</a> Notes
</h3>
<ul>
<li>
Some variables are infered as lists. Each token on the line separated by whitespace will be member of the list.
</li>
<li>
Comments (lines starting with #) and blank lines are ignored.
</li>

</ul>
<h2 class="Section">
<a class="toc" name="toc-Section-4.2">4.2</a> Environment variables
</h2>
<div class="Unindented">
<table>
<tr>
<td align="left" valign="top">
Variable
</td>
<td align="left" valign="top">
Purpose
</td>
<td align="left" valign="top">
Default value/Comment
</td>

</tr>
<tr>
<td align="left" valign="top">
MT_CMDINFO
</td>
<td align="left" valign="top">
Conveys test variables to the megatest test runner.
</td>
<td align="left" valign="top">
Not for use by the end user
</td>

</tr>
<tr>
<td align="left" valign="top">
MT_TEST_RUN_DIR
</td>
<td align="left" valign="top">
Directory assigned by megatest for the test to run.
</td>
<td align="left" valign="top">
Tests can consider this their “home” directory
</td>

</tr>
<tr>
<td align="left" valign="top">
MT_TEST_NAME
</td>
<td align="left" valign="top">
Name of the test, corrosponds to the directory name under tests.
</td>
<td align="left" valign="top">

</td>

</tr>
<tr>
<td align="left" valign="top">
MT_ITEM_INFO
</td>
<td align="left" valign="top">
Iterated tests will set this to a sequence of key/values ((KEY val) ...) 
</td>
<td align="left" valign="top">

</td>

</tr>
<tr>
<td align="left" valign="top">
MT_RUN_AREA_HOME
</td>
<td align="left" valign="top">
Directory where megatest was launched from and where the tests code can be found
</td>
<td align="left" valign="top">

</td>

</tr>
<tr>
<td align="left" valign="top">
MT_RUNNAME
</td>
<td align="left" valign="top">
Name of this run as set by the :runname parameter
</td>
<td align="left" valign="top">

</td>

</tr>
<tr>
<td align="left" valign="top">
MT_MEGATEST
</td>
<td align="left" valign="top">
Path/Filename to megatest executable. Found either from called path or but using the “exectuable” keyword in the [setup] section.
</td>
<td align="left" valign="top">

</td>

</tr>
<tr>
<td align="left" valign="top">
&lt;field1&gt; ....
</td>
<td align="left" valign="top">
The field values as set on the megatest -runall command line (e.g. :field1 abc)
</td>
<td align="left" valign="top">
Planning ahead your field names is important so as to avoid collisions with other environment variables
</td>

</tr>

</table>

</div>
<h2 class="Section">
<a class="toc" name="toc-Section-4.3">4.3</a> Configuration files
</h2>
<h3 class="Subsection">
<a class="toc" name="toc-Subsection-4.3.1">4.3.1</a> megatest.config
</h3>
<div class="Unindented">
<table>
<tr>
<td align="left" valign="top">
section
</td>
<td align="left" valign="top">
variable








|
|
|
|
|
>
>
>

<




|
|
|
|
<
<
|
<











|
|
|
|
|






|
|
|
|
|



<





|
|
|
|
|











<
<
<
|
|

|
|

<
<
<
|
|






|
|
|
|
|








|
|
|
|
|

|
|













|


|
|

|


|
|














|
|
|
|
|
|
|


|
|











|
|
|
|






>

>
>





|
|
|
|


|
|
|
|
|


>
|
>
>






|
|
|
|
|
|
>
>
|
>
>
>
>
>

>
>
>

>
|
>
>
>
>
>

>
>
>

<
<
<
<


>
>

|
<
<
|
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>


|
|
>
|
>
>
>

>
>
>

>
>
>

<


>

>
>
|
|

>
>
>
>
>
>
>
>
|
|
>
>
>

|
|

>
|
|
>

|
>
>

|
|
|

>
|
|
>

|
>
>

|
|
|

>
|
|
>

|
>
>

|
|
|

>
|
|
>

|
>
>

|
|
|










|
|

|








<
<
<









<
<
<









<
<
<








<
<
<









<
<
<









<
<
<









<
<
<









<
<
<










<
<
<






|
|

<
<
<
|








327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342

343
344
345
346
347
348
349
350


351

352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381

382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402



403
404
405
406
407
408



409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570




571
572
573
574
575
576


577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610

611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709



710
711
712
713
714
715
716
717
718



719
720
721
722
723
724
725
726
727



728
729
730
731
732
733
734
735



736
737
738
739
740
741
742
743
744



745
746
747
748
749
750
751
752
753



754
755
756
757
758
759
760
761
762



763
764
765
766
767
768
769
770
771



772
773
774
775
776
777
778
779
780
781



782
783
784
785
786
787
788
789
790



791
792
793
794
795
796
797
798

# name /path/to/area 
[disks]
disk1 /tmp 
</pre>
</div>

</div>
<h2 class="Subsection">
<a class="toc" name="toc-Subsection-3.2">3.2</a> Create runconfigs.config
</h2>
<div class="Standard">
This file is used to set environment variables that are run specific. You can simply create an empty file to start.
</div>
<div class="Standard">
<div class="listing">
<pre class="listing"># runconfigs.config

</pre>
</div>

</div>
<h2 class="Subsection">
<a class="toc" name="toc-Subsection-3.3">3.3</a> Create the tests directory and your first test
</h2>
<div class="Standard">


<div class="listing">

<pre class="listing">../tests
  ├── megatest.config
  ├── runconfigs.config
  └── tests
     └── mytest
           ├── main.sh
           └── testconfig
</pre>
</div>

</div>
<h2 class="Subsection">
<a class="toc" name="toc-Subsection-3.4">3.4</a> Create the testconfig file for your test
</h2>
<div class="Standard">
<div class="listing">
<pre class="listing">[setup]
runscript main.sh
</pre>
</div>

</div>
<h2 class="Subsection">
<a class="toc" name="toc-Subsection-3.5">3.5</a> Create your test running script, main.sh
</h2>
<div class="Standard">
<div class="listing">
<pre class="listing">#!/bin/bash
​
$MT_MEGATEST -runstep mystep1 "sleep 20;echo Done" -m "mystep1 is done"

$MT_MEGATEST -test-status :state COMPLETED :status PASS -m "This is a comment"
</pre>
</div>

</div>
<h2 class="Subsection">
<a class="toc" name="toc-Subsection-3.6">3.6</a> Run megatest and watch your run progress
</h2>
<div class="Standard">
<div class="listing">
<pre class="listing">megatest :field1 abc :field2 def :runname 2011week08.4a -runall
​
watch megatest -list-runs %
​
# OR use the dashboard
​
dashboard &amp;
</pre>
</div>

</div>



<h1 class="Section">
<a class="toc" name="toc-Section-4">4</a> How to Write Tests
</h1>
<h2 class="Subsection">
<a class="toc" name="toc-Subsection-4.1">4.1</a> A Simple Test with one Step
</h2>



<div class="Standard">
<div class="listing">
<pre class="listing">mkdir simpletest
cd simpletest
</pre>
</div>

</div>
<h2 class="Subsection">
<a class="toc" name="toc-Subsection-4.2">4.2</a> Create your testconfig file
</h2>
<div class="Standard">
<div class="listing">
<pre class="listing"># testconfig
​
[setup]
runscript main.csh
</pre>
</div>

</div>
<h2 class="Subsection">
<a class="toc" name="toc-Subsection-4.3">4.3</a> Create the main.csh script
</h2>
<div class="Standard">
Note: Using csh is NOT recommended. Use bash, perl, ruby, zsh or anything other than csh. We use csh here because it is popular in the EDA industry for which Megatest was originally created. 
</div>
<div class="Standard">
<div class="listing">
<pre class="listing">#!/bin/tcsh -x
​
# run the cpu1 simulation.
#   The step name is "run_simulation"
#   The commandline being run for this step is "runsim cpu1"
#   The logpro file to validate the output from the run is "runsim.logpro"
​
$MT_MEGATEST -runstep run_simulation -logpro runsim.logpro "runsim cpu1"
$MT_MEGATEST -test-status :state COMPLETED :status $?
</pre>
</div>

</div>
<div class="Standard">
You can now run megatest and the created test directory will contain the new files “run_simulation.html” and “run_simulation.log”. If you are using the dashboard you can click on the run and then push the “View log” button to view the log file in firefox.
</div>
<h2 class="Subsection">
<a class="toc" name="toc-Subsection-4.4">4.4</a> Simple Test with Multiple Steps 
</h2>
<div class="Standard">
To run multiple steps simply add them to the main.csh file. Here we add a step to test “cpu2”. The second step that tests cpu2 will only run after the step that tested “cpu1” completes.
</div>
<div class="Standard">
<div class="listing">
<pre class="listing">#!/bin/tcsh -x
​
# run the cpu1 simulation.
#   The step name is "run_simulation"
#   The commandline being run for this step is "runsim cpu1"
#   The logpro file to validate the output from the run is "runsim.logpro"
​
$MT_MEGATEST -runstep run_simulation_cpu1 -logpro runsim.logpro "runsim cpu1" &amp;&amp; \
$MT_MEGATEST -runstep run_simulation_cpu2 -logpro runsim.logpro "runsim cpu2"
$MT_MEGATEST -test-status :state COMPLETED :status $?
</pre>
</div>

</div>
<h1 class="Section">
<a class="toc" name="toc-Section-5">5</a> Simple Test with Multiple Steps, Some in Parallel
</h1>
<h2 class="Subsection">
<a class="toc" name="toc-Subsection-5.1">5.1</a> The Makefile
</h2>
<div class="Standard">
A good way to run steps in parallel within a single test, especially when there are following steps, is to use the Unix Make utility. Writing Makefiles is beyond the scope of this document but here is a minimal example that will run “runsim cpu1” and “runsim cpu2” in parallel. For more information on make try “info make” at the Linux command prompt.
</div>
<div class="Standard">
<div class="listing">
<pre class="listing"># Example Makefile to run two steps in parallel
​
RTLDIR=/path/to/rtl
CPUS = cpu1 cpu2
​
run_simulation_$(CPUS).html : $(RTLDIR)/$(CPUS)
	$(MT_MEGATEST) -runstep run_simulation_$(CPUS) -logpro runsim.logpro "runsim $(CPUS)
</pre>
</div>

</div>
<h2 class="Subsection">
<a class="toc" name="toc-Subsection-5.2">5.2</a> The main.csh file
</h2>
<div class="Standard">
<div class="listing">
<pre class="listing">#!/bin/tcsh -x
​
# run the cpu1 and cpu2 simulations in parallel. 
# The -j parameter tells make how many jobs it may run in parallel
​
<div class="left">
make -j 2 
</div>

$MT_MEGATEST -test-status :state COMPLETED :status $?
</pre>
</div>

</div>
<h1 class="Section">
<a class="toc" name="toc-Section-6">6</a> Simple Test with Iteration
</h1>
<div class="Standard">
Since no jobs run after the cpu1 and cpu2 simulations in this test it is possible to use iterated mode.
</div>
<h2 class="Subsection">
<a class="toc" name="toc-Subsection-6.1">6.1</a> Update your testconfig file for iteration
</h2>
<div class="Standard">
<div class="listing">
<pre class="listing">[setup]
runscript main.csh
<div class="left">

</div>

[items]
CPU cpu1 cpu2
</pre>
</div>

</div>
<h2 class="Subsection">
<a class="toc" name="toc-Subsection-6.2">6.2</a> Rewrite your main.csh for iteration
</h2>
<div class="Standard">
<div class="listing">
<pre class="listing"><div class="left">
#!/bin/tcsh -x
</div>

<div class="left">

</div>

<div class="left">
# run the cpu simulation but now use the environment variable $CPU
</div>

<div class="left">
# to select what cpu to run the simulation against
</div>

<div class="left">

</div>

<div class="left">
$MT_MEGATEST -runstep run_simulation -logpro runsim.logpro "runsim $CPU"
</div>

<div class="left">
# As of version 1.07 Megatest automatically converts a status of "0"




</div>

<div class="left">
# to "PASS", any other number to "FAIL" and directly uses the value of
</div>



<div class="left">
# a string passed in.
</div>

<div class="left">
$MT_MEGATEST -test-status :state COMPLETED :status $?
</div>

</pre>
</div>

</div>
<h2 class="Subsection">
<a class="toc" name="toc-Subsection-6.3">6.3</a> Tests with Inter-test dependencies
</h2>
<div class="Standard">
Sometimes a test depends on the output from a previous test or it may not make sense to run a test is another test does not complete with status “PASS”. In either of these scenarios you can use the “waiton” keyword in your testconfig file to indicate that this test must wait on one or more tests to complete before being launched. In this example there is no point in running the “system” test if the “cpu” and “mem” tests either do not complete or complete but with status “FAIL”.
</div>
<div class="Standard">
<div class="listing">
<pre class="listing"><div class="left">
# testconfig for the "system" test
</div>

<div class="left">
[setup]
</div>

<div class="left">
runscript main.csh
</div>

<div class="left">
waiton cpu mem

</div>

</pre>
</div>

</div>
<h1 class="Section">
<a class="toc" name="toc-Section-7">7</a> Dashboard
</h1>
<div class="Standard">
<img class="embedded" src="dashboard.png" alt="figure dashboard.png" style="width: 730px; max-width: 913px; height: 626px; max-height: 783px; "/>

</div>
<div class="Standard">
<img class="embedded" src="dashboard-test.png" alt="figure dashboard-test.png" style="width: 643px; max-width: 804px; height: 571px; max-height: 714px; "/>

</div>
<h1 class="Section">
<a class="toc" name="toc-Section-8">8</a> Reference
</h1>
<h2 class="Subsection">
<a class="toc" name="toc-Subsection-8.1">8.1</a> Configuration file Syntax
</h2>
<h3 class="Subsubsection">
<a class="toc" name="toc-Subsubsection-8.1.1">8.1.1</a> Sections
</h3>
<div class="Standard">
<div class="listing">
<pre class="listing">[section name]
</pre>
</div>

</div>
<div class="Standard">
This creates a section named “section name”
</div>
<h3 class="Subsubsection">
<a class="toc" name="toc-Subsubsection-8.1.2">8.1.2</a> Variables
</h3>
<div class="Standard">
<div class="listing">
<pre class="listing">VARX has this value
</pre>
</div>

</div>
<div class="Standard">
The variable “VARX” will have the value “has this value”
</div>
<h3 class="Subsubsection">
<a class="toc" name="toc-Subsubsection-8.1.3">8.1.3</a> Includes
</h3>
<div class="Standard">
<div class="listing">
<pre class="listing">[include filename]
</pre>
</div>

</div>
<div class="Standard">
The file named “filename” will be included as if part of the calling file. NOTE: This means no section can be named “include “ (with the whitespace).
</div>
<h3 class="Subsubsection">
<a class="toc" name="toc-Subsubsection-8.1.4">8.1.4</a> Setting a variable by running a command
</h3>
<div class="Standard">
<div class="listing">
<pre class="listing">VARNAME [system ls /tmp]
</pre>
</div>

</div>
<div class="Standard">
The variable “VARNAME” will get a value created by the Unix command “ls /tmp”. All lines of output from the command will be joined with a space.
</div>
<h3 class="Subsubsection">
<a class="toc" name="toc-Subsubsection-8.1.5">8.1.5</a> Notes
</h3>
<ul>
<li>
Some variables are infered as lists. Each token on the line separated by whitespace will be member of the list.
</li>
<li>
Comments (lines starting with #) and blank lines are ignored.
</li>

</ul>
<h2 class="Subsection">
<a class="toc" name="toc-Subsection-8.2">8.2</a> Environment variables
</h2>
<div class="Standard">
<table>
<tr>
<td align="left" valign="top">
Variable
</td>
<td align="left" valign="top">
Purpose
</td>




</tr>
<tr>
<td align="left" valign="top">
MT_CMDINFO
</td>
<td align="left" valign="top">
Conveys test variables to the megatest test runner.
</td>




</tr>
<tr>
<td align="left" valign="top">
MT_TEST_RUN_DIR
</td>
<td align="left" valign="top">
Directory assigned by megatest for the test to run.
</td>




</tr>
<tr>
<td align="left" valign="top">
MT_TEST_NAME
</td>
<td align="left" valign="top">
Name of the test, corrosponds to the directory name under tests.



</td>

</tr>
<tr>
<td align="left" valign="top">
MT_ITEM_INFO
</td>
<td align="left" valign="top">
Iterated tests will set this to a sequence of key/values ((KEY val) ...) 



</td>

</tr>
<tr>
<td align="left" valign="top">
MT_RUN_AREA_HOME
</td>
<td align="left" valign="top">
Directory where megatest was launched from and where the tests code can be found



</td>

</tr>
<tr>
<td align="left" valign="top">
MT_RUNNAME
</td>
<td align="left" valign="top">
Name of this run as set by the :runname parameter



</td>

</tr>
<tr>
<td align="left" valign="top">
MT_MEGATEST
</td>
<td align="left" valign="top">
Path/Filename to megatest executable. Found either from called path or but using the “exectuable” keyword in the [setup] section.



</td>

</tr>
<tr>
<td align="left" valign="top">
&lt;field1&gt; ....
</td>
<td align="left" valign="top">
The field values as set on the megatest -runall command line (e.g. :field1 abc)
</td>




</tr>

</table>

</div>
<h2 class="Subsection">
<a class="toc" name="toc-Subsection-8.3">8.3</a> megatest.config
</h2>



<div class="Standard">
<table>
<tr>
<td align="left" valign="top">
section
</td>
<td align="left" valign="top">
variable
︙︙
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848

</td>

</tr>

</table>

</div>
<h3 class="Subsection">
<a class="toc" name="toc-Subsection-4.3.2">4.3.2</a> runconfigs.config file
</h3>
<div class="Unindented">
<table>
<tr>
<td align="left" valign="top">
section
</td>
<td align="left" valign="top">
variable








|
|
|
|








952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969

</td>

</tr>

</table>

</div>
<h2 class="Subsection">
<a class="toc" name="toc-Subsection-8.4">8.4</a> runconfigs.config file
</h2>
<div class="Standard">
<table>
<tr>
<td align="left" valign="top">
section
</td>
<td align="left" valign="top">
variable
︙︙
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911

</td>

</tr>

</table>

</div>
<div class="Indented">
Example: a test suite that checks that a piece of software works correctly for different customer configurations and locations each of which is done as a separate release regression run. The fields, CUSTOMER and LOCATION were chosen. The following runconfigs.config file would set some variables specific to runs for megacorp in India and femtocorp in the Cook Islands and New Zealand:
</div>
<div class="Indented">
<div class="listing">
<pre class="listing"># runconfigs.config
[default]
ENCRYTION true
​
[megacorp/india]
TESTPATH /nfs/testing/megacorp_runs








|


|








1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032

</td>

</tr>

</table>

</div>
<div class="Standard">
Example: a test suite that checks that a piece of software works correctly for different customer configurations and locations each of which is done as a separate release regression run. The fields, CUSTOMER and LOCATION were chosen. The following runconfigs.config file would set some variables specific to runs for megacorp in India and femtocorp in the Cook Islands and New Zealand:
</div>
<div class="Standard">
<div class="listing">
<pre class="listing"># runconfigs.config
[default]
ENCRYTION true
​
[megacorp/india]
TESTPATH /nfs/testing/megacorp_runs
︙︙
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954

</div>
<div class="Code">
ENCRYPTION true
</div>
<div class="Code">
TESTPATH /nfs/testing/megacorp_runs
</div>
<h2 class="Section">
<a class="toc" name="toc-Section-4.4">4.4</a> Writing tests
</h2>
<h3 class="Subsection">
<a class="toc" name="toc-Subsection-4.4.1">4.4.1</a> testconfig file
</h3>
<div class="Unindented">
<table>
<tr>
<td align="left" valign="top">
section
</td>
<td align="left" valign="top">
variable








|
|

|
|

|








1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075

</div>
<div class="Code">
ENCRYPTION true
</div>
<div class="Code">
TESTPATH /nfs/testing/megacorp_runs
</div>
<h2 class="Subsection">
<a class="toc" name="toc-Subsection-8.5">8.5</a> Writing tests
</h2>
<h3 class="Subsubsection">
<a class="toc" name="toc-Subsubsection-8.5.1">8.5.1</a> testconfig file
</h3>
<div class="Standard">
<table>
<tr>
<td align="left" valign="top">
section
</td>
<td align="left" valign="top">
variable
︙︙
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035

</td>

</tr>

</table>

</div>
<h3 class="Subsection">
<a class="toc" name="toc-Subsection-4.4.2">4.4.2</a> Command line
</h3>
<div class="Unindented">
<table>
<tr>
<td align="left" valign="top">
switch or param
</td>
<td align="left" valign="top">
parameter








|
|

|








1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156

</td>

</tr>

</table>

</div>
<h3 class="Subsubsection">
<a class="toc" name="toc-Subsubsection-8.5.2">8.5.2</a> Command line
</h3>
<div class="Standard">
<table>
<tr>
<td align="left" valign="top">
switch or param
</td>
<td align="left" valign="top">
parameter
︙︙
1357
1358
1359
1360
1361
1362
1363






1364
1365
1366
1367
1368
1369
1370
1371

</td>

</tr>

</table>

</div>







<hr class="footer"/>
<div class="footer" id="generated-by">
Document generated by <a href="http://elyxer.nongnu.org/">eLyXer 1.0.0 (2010-07-21)</a> on <span class="create-date">2011-08-03T20:43:07.693353</span>
</div>
</div>
</body>
</html>








>
>
>
>
>
>



|





1478
1479
1480
1481
1482
1483
1484
1485
1486
1487
1488
1489
1490
1491
1492
1493
1494
1495
1496
1497
1498

</td>

</tr>

</table>

</div>
<h1 class="Section">
<a class="toc" name="toc-Section-A">A</a> Data
</h1>
<h1 class="Section">
<a class="toc" name="toc-Section-B">B</a> References
</h1>

<hr class="footer"/>
<div class="footer" id="generated-by">
Document generated by <a href="http://elyxer.nongnu.org/">eLyXer 1.0.0 (2010-07-21)</a> on <span class="create-date">2011-09-20T23:11:19.692083</span>
</div>
</div>
</body>
</html>

Modified docs/megatest.lyx from [9238d242de] to [23c11249c1].

︙︙
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419

</cell>
</row>
<row>
<cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
\begin_inset Text

\begin_layout Plain Layout
0.1
\end_layout

\end_inset
</cell>
<cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
\begin_inset Text

\begin_layout Plain Layout

\end_layout

\end_inset
</cell>
<cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
\begin_inset Text

\begin_layout Plain Layout

\end_layout

\end_inset
</cell>
<cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text









|








|








|








387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419

</cell>
</row>
<row>
<cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
\begin_inset Text

\begin_layout Plain Layout
v1.25
\end_layout

\end_inset
</cell>
<cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
\begin_inset Text

\begin_layout Plain Layout
matt
\end_layout

\end_inset
</cell>
<cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
\begin_inset Text

\begin_layout Plain Layout
converted to new document format
\end_layout

\end_inset
</cell>
<cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text
︙︙
1322
1323
1324
1325
1326
1327
1328
























1329
1330
1331
1332
1333
1334
1335


\begin_layout Plain Layout
\noindent
\align left

waiton cpu mem
\end_layout

























\end_inset


\end_layout

\begin_layout Section








>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>








1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
1353
1354
1355
1356
1357
1358
1359


\begin_layout Plain Layout
\noindent
\align left

waiton cpu mem
\end_layout

\end_inset


\end_layout

\begin_layout Section
Dashboard
\end_layout

\begin_layout Standard
\begin_inset Graphics
	filename dashboard.png
	scale 80

\end_inset


\end_layout

\begin_layout Standard
\begin_inset Graphics
	filename dashboard-test.png
	scale 80

\end_inset


\end_layout

\begin_layout Section
︙︙
2376
2377
2378
2379
2380
2381
2382
2383
2384
2385
2386
2387
2388
2389
2390
2391
2392
2393
2394

ENCRYPTION true
\end_layout

\begin_layout Code
TESTPATH /nfs/testing/megacorp_runs
\end_layout

\begin_layout Section
Writing tests
\end_layout

\begin_layout Subsection
testconfig file
\end_layout

\begin_layout Standard
\begin_inset Tabular
<lyxtabular version="3" rows="4" columns="5">
<features tabularvalignment="middle" tabularwidth="80page%">








|



|








2400
2401
2402
2403
2404
2405
2406
2407
2408
2409
2410
2411
2412
2413
2414
2415
2416
2417
2418

ENCRYPTION true
\end_layout

\begin_layout Code
TESTPATH /nfs/testing/megacorp_runs
\end_layout

\begin_layout Subsection
Writing tests
\end_layout

\begin_layout Subsubsection
testconfig file
\end_layout

\begin_layout Standard
\begin_inset Tabular
<lyxtabular version="3" rows="4" columns="5">
<features tabularvalignment="middle" tabularwidth="80page%">
︙︙
2593
2594
2595
2596
2597
2598
2599
2600
2601
2602
2603
2604
2605
2606
2607

</lyxtabular>

\end_inset


\end_layout

\begin_layout Subsection
Command line
\end_layout

\begin_layout Standard
\begin_inset Tabular
<lyxtabular version="3" rows="22" columns="4">
<features tabularvalignment="middle">








|








2617
2618
2619
2620
2621
2622
2623
2624
2625
2626
2627
2628
2629
2630
2631

</lyxtabular>

\end_inset


\end_layout

\begin_layout Subsubsection
Command line
\end_layout

\begin_layout Standard
\begin_inset Tabular
<lyxtabular version="3" rows="22" columns="4">
<features tabularvalignment="middle">
︙︙

This page was generated in about 0.134s by Fossil 2.24 [452f8e1ffc] 2024-03-26 15:36:00