Megatest

Check-in [b6762fdadf]
Login
☰Home Timeline Files Branches Tags Tickets Wiki Login
Overview
Comment:Manually tweaked megatest.wiki, remove top 10 or so lines, added missing title text back in. Removed last two lines
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA1: b6762fdadf6b99092266bcf76121bcc24e3bd011
User & Date: matt on 2011-09-21 00:53:02
Other Links: manifest | tags
Context
2011-09-21
17:13
Bumped to v1.26 check-in: 6340e533f1 user: mrwellan tags: trunk
00:53
Manually tweaked megatest.wiki, remove top 10 or so lines, added missing title text back in. Removed last two lines check-in: b6762fdadf user: matt tags: trunk
00:45
Adding html file as .wiki check-in: 513285e117 user: matt tags: trunk
Changes
Hide Diffs Side-by-Side Diffs Ignore Whitespace Patch

Modified docs/megatest.wiki from [50f0ba7bf3] to [a8e8eae5b4]. 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

24
25
26
27
28
29
30












1
2
3
4
5
6
7
8
9
10
11

12
13
14
15
16
17
18
19

-
-
-
-
-
-
-
-
-
-
-











-
+








<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
<meta name="generator" content="http://www.nongnu.org/elyxer/"/>
<meta name="create-date" content="2011-09-20"/>
<link rel="stylesheet" href="http://www.nongnu.org/elyxer/lyx.css" type="text/css" media="screen"/>
<title>Converted document</title>
</head>
<body>
<div id="globalWrapper">
<div class="Standard">

</div>
<div class="Standard">

</div>
<div class="Standard">

</div>
<h1 class="title">

Megatest
</h1>
<h2 class="author">
Matthew Welland
</h2>
<h2 class="Date">
Sept. 20, 
</h2>
︙
59
60
61
62
63
64
65
66

67
68
69
70
71
72
73

48
49
50
51
52
53
54

55
56
57
58
59
60
61
62








-
+








</div>
<div class="Standard">
<p><br/>
</p>

</div>
<h1 class="Section-">
<a class="toc" name="toc-Section--1"></a>Document Revision History
<a class="toc" name="toc-Section--1"></a>Megatest/document Revision History
</h1>
<div class="Standard">
<i>Notable revisions of the software are occasionally documented here</i>.
</div>
<div class="Standard">
<div class="center">
<table>
︙
90
91
92
93
94
95
96
97

98
99
100
101
102
103
104

79
80
81
82
83
84
85

86
87
88
89
90
91
92
93








-
+








<td align="center" valign="top">
v1.25
</td>
<td align="center" valign="top">
matt
</td>
<td align="justify" valign="top">
converted to new document format
converted to new document template
</td>
<td align="center" valign="top">
<div class="PlainVisible">

</div>

</td>
︙
112
113
114
115
116
117
118
119

120
121
122
123
124
125
126

101
102
103
104
105
106
107

108
109
110
111
112
113
114
115








-
+








</div>
<div class="fulltoc">
<div class="tocheader">
Table of Contents
</div>
<div class="tocindent">
<div class="toc">
<a class="Link" href="#toc-Section--1">Section: Document Revision History</a>
<a class="Link" href="#toc-Section--1">Section: Megatest/document Revision History</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Section-1">Section 1: Introduction</a>
</div>
<div class="tocindent">
<div class="toc">
<a class="Link" href="#toc-Subsection-1.1">Subsection 1.1: Megatest design philosophy</a>
︙
200
201
202
203
204
205
206






207
208
209
210
211
212

213



214
215
216

217
218
219
220

221
222
223

224
225
226

227
228
229

230
231
232

233
234
235
236

237
238
239

240
241
242

243
244
245

246
247
248
249

250
251
252

253
254
255
256
257
258
259

189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206

207
208
209
210
211
212
213

214
215
216
217

218
219
220

221
222
223

224
225
226

227
228
229

230
231
232
233

234
235
236

237
238
239

240
241
242

243
244
245
246

247
248
249

250
251
252
253
254
255
256
257








+
+
+
+
+
+





-
+

+
+
+


-
+



-
+


-
+


-
+


-
+


-
+



-
+


-
+


-
+


-
+



-
+


-
+








</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-6.2">Subsection 6.2: Rewrite your main.csh for iteration</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-6.3">Subsection 6.3: Tests with Inter-test dependencies</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-6.4">Subsection 6.4: Rolling up Miscellaneous Data</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-6.5">Subsection 6.5: Rolling up Runs</a>
</div>
</div>
<div class="toc">
<a class="Link" href="#toc-Section-7">Section 7: Dashboard</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Section-8">Section 8: Reference</a>
<a class="Link" href="#toc-Section-8">Section 8: Generating an OpenDocument Spreadsheet from the Database</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Section-9">Section 9: Reference</a>
</div>
<div class="tocindent">
<div class="toc">
<a class="Link" href="#toc-Subsection-8.1">Subsection 8.1: Configuration file Syntax</a>
<a class="Link" href="#toc-Subsection-9.1">Subsection 9.1: Configuration file Syntax</a>
</div>
<div class="tocindent">
<div class="toc">
<a class="Link" href="#toc-Subsubsection-8.1.1">Subsubsection 8.1.1: Sections</a>
<a class="Link" href="#toc-Subsubsection-9.1.1">Subsubsection 9.1.1: Sections</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsubsection-8.1.2">Subsubsection 8.1.2: Variables</a>
<a class="Link" href="#toc-Subsubsection-9.1.2">Subsubsection 9.1.2: Variables</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsubsection-8.1.3">Subsubsection 8.1.3: Includes</a>
<a class="Link" href="#toc-Subsubsection-9.1.3">Subsubsection 9.1.3: Includes</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsubsection-8.1.4">Subsubsection 8.1.4: Setting a variable by running a command</a>
<a class="Link" href="#toc-Subsubsection-9.1.4">Subsubsection 9.1.4: Setting a variable by running a command</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsubsection-8.1.5">Subsubsection 8.1.5: Notes</a>
<a class="Link" href="#toc-Subsubsection-9.1.5">Subsubsection 9.1.5: Notes</a>
</div>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-8.2">Subsection 8.2: Environment variables</a>
<a class="Link" href="#toc-Subsection-9.2">Subsection 9.2: Environment variables</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-8.3">Subsection 8.3: megatest.config</a>
<a class="Link" href="#toc-Subsection-9.3">Subsection 9.3: megatest.config</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-8.4">Subsection 8.4: runconfigs.config file</a>
<a class="Link" href="#toc-Subsection-9.4">Subsection 9.4: runconfigs.config file</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsection-8.5">Subsection 8.5: Writing tests</a>
<a class="Link" href="#toc-Subsection-9.5">Subsection 9.5: Writing tests</a>
</div>
<div class="tocindent">
<div class="toc">
<a class="Link" href="#toc-Subsubsection-8.5.1">Subsubsection 8.5.1: testconfig file</a>
<a class="Link" href="#toc-Subsubsection-9.5.1">Subsubsection 9.5.1: testconfig file</a>
</div>
<div class="toc">
<a class="Link" href="#toc-Subsubsection-8.5.2">Subsubsection 8.5.2: Command line</a>
<a class="Link" href="#toc-Subsubsection-9.5.2">Subsubsection 9.5.2: Command line</a>
</div>
</div>
</div>
<div class="toc">
<a class="Link" href="#toc-Section-A">Section A: Data</a>
</div>
<div class="toc">
︙
610
611
612
613
614
615
616



























































617
618
619







620
621
622
623



624
625
626
627
628
















629

630
631
632

633
634
635

636
637
638
639
640
641
642
643
644
645
646
647
648

649
650
651
652
653
654
655
656
657
658
659
660
661

662
663
664
665
666
667
668
669
670
671
672
673
674

675
676
677
678
679
680
681
682
683
684
685
686
687

688
689
690
691
692
693
694
695
696
697
698
699

700
701
702
703
704
705
706

608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711

712
713
714

715
716
717

718
719
720
721
722
723
724
725
726
727
728
729
730

731
732
733
734
735
736
737
738
739
740
741
742
743

744
745
746
747
748
749
750
751
752
753
754
755
756

757
758
759
760
761
762
763
764
765
766
767
768
769

770
771
772
773
774
775
776
777
778
779
780
781

782
783
784
785
786
787
788
789








+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+



+
+
+
+
+
+
+




+
+
+





+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
-
+


-
+


-
+












-
+












-
+












-
+












-
+











-
+








waiton cpu mem
</div>

</pre>
</div>

</div>
<h2 class="Subsection">
<a class="toc" name="toc-Subsection-6.4">6.4</a> Rolling up Miscellaneous Data
</h2>
<div class="Standard">
Use the -load-test-data switch to roll up arbitrary data from a test into the test_data table.
</div>
<div class="Standard">
<div class="listing">
<pre class="listing"># Fields are:
# category,variable,value,expected,tol,units,comment,status
​
$MT_MEGATEST -load-test-data &lt;&lt; EOF
foo,bar,1.2,1.9,&gt;
foo,rab,1.0e9,10e9,1e9
foo,bla,1.2,1.9,&lt;
foo,bal,1.2,1.2,&lt;,,Check for overload
foo,alb,1.2,1.2,&lt;=,Amps,This is the high power circuit test
foo,abl,1.2,1.3,0.1
foo,bra,1.2,pass,silly stuff
faz,bar,10,8mA,,,"this is a comment"
EOF
</pre>
</div>

</div>
<div class="Standard">
New entries are keyed on the category and variable. If a new record is inserted with a category and variable that have already been used the new record will replace the old record.
</div>
<div class="Standard">
Where value, expected and tol are specified the behavior is as follows.
</div>
<ul>
<li>
If value, expected and tol are numbers then status is calculated as PASS if (expected-tol) &lt;= value &lt;= (expected+tol)
</li>
<li>
If value and expected are numbers and tol is &gt;, &lt;, &gt;= or &lt;= then value is compared with expected using the operator given by tol
</li>
<li>
If status is specified its value overrides the above calculations.
</li>

</ul>
<h2 class="Subsection">
<a class="toc" name="toc-Subsection-6.5">6.5</a> Rolling up Runs
</h2>
<div class="Standard">
To roll up a number of tests in a sequence of runs to a single run use the -rollup command. 
</div>
<div class="Standard">
<div class="listing">
<pre class="listing">megatest -rollup :sysname ubuntu :fsname nfs :datapath none :runname rollup_ww38
</pre>
</div>

</div>
<div class="Standard">
All keys must be specified and the runname is the name of the run that will be created. All paths are kept original inside the database. When -remove-runs is used to delete runs the data is not deleted if there are rollups that refer to the data. 
</div>
<h1 class="Section">
<a class="toc" name="toc-Section-7">7</a> Dashboard
</h1>
<div class="Standard">
<div class="listing">
<pre class="listing">&gt; dashboard &amp;
</pre>
</div>

</div>
<div class="Standard">
<img class="embedded" src="dashboard.png" alt="figure dashboard.png" style="width: 730px; max-width: 913px; height: 626px; max-height: 783px; "/>

</div>
<div class="Standard">
Pushing one of the buttons on the main dashboard will bring up the test specific dashboard. Values are updated in semi-real time as the test runs.
</div>
<div class="Standard">
<img class="embedded" src="dashboard-test.png" alt="figure dashboard-test.png" style="width: 643px; max-width: 804px; height: 571px; max-height: 714px; "/>

</div>
<h1 class="Section">
<a class="toc" name="toc-Section-8">8</a> Generating an OpenDocument Spreadsheet from the Database
</h1>
<div class="Standard">
And OpenDocument multi-paned spreadsheet can be generated from the megatest.db file by running -extract-ods
</div>
<div class="Standard">
<div class="listing">
<pre class="listing">megatest -extract-ods results.ods :runname % 
</pre>
</div>

</div>
<div class="Standard">
You can optionally specify the keys for your database to limit further the runs to extract into the spreadsheet. The first sheet contains all the run data and subsequent sheets contain data rolled up for the individual tests.
</div>
<h1 class="Section">
<a class="toc" name="toc-Section-8">8</a> Reference
<a class="toc" name="toc-Section-9">9</a> Reference
</h1>
<h2 class="Subsection">
<a class="toc" name="toc-Subsection-8.1">8.1</a> Configuration file Syntax
<a class="toc" name="toc-Subsection-9.1">9.1</a> Configuration file Syntax
</h2>
<h3 class="Subsubsection">
<a class="toc" name="toc-Subsubsection-8.1.1">8.1.1</a> Sections
<a class="toc" name="toc-Subsubsection-9.1.1">9.1.1</a> Sections
</h3>
<div class="Standard">
<div class="listing">
<pre class="listing">[section name]
</pre>
</div>

</div>
<div class="Standard">
This creates a section named “section name”
</div>
<h3 class="Subsubsection">
<a class="toc" name="toc-Subsubsection-8.1.2">8.1.2</a> Variables
<a class="toc" name="toc-Subsubsection-9.1.2">9.1.2</a> Variables
</h3>
<div class="Standard">
<div class="listing">
<pre class="listing">VARX has this value
</pre>
</div>

</div>
<div class="Standard">
The variable “VARX” will have the value “has this value”
</div>
<h3 class="Subsubsection">
<a class="toc" name="toc-Subsubsection-8.1.3">8.1.3</a> Includes
<a class="toc" name="toc-Subsubsection-9.1.3">9.1.3</a> Includes
</h3>
<div class="Standard">
<div class="listing">
<pre class="listing">[include filename]
</pre>
</div>

</div>
<div class="Standard">
The file named “filename” will be included as if part of the calling file. NOTE: This means no section can be named “include “ (with the whitespace).
</div>
<h3 class="Subsubsection">
<a class="toc" name="toc-Subsubsection-8.1.4">8.1.4</a> Setting a variable by running a command
<a class="toc" name="toc-Subsubsection-9.1.4">9.1.4</a> Setting a variable by running a command
</h3>
<div class="Standard">
<div class="listing">
<pre class="listing">VARNAME [system ls /tmp]
</pre>
</div>

</div>
<div class="Standard">
The variable “VARNAME” will get a value created by the Unix command “ls /tmp”. All lines of output from the command will be joined with a space.
</div>
<h3 class="Subsubsection">
<a class="toc" name="toc-Subsubsection-8.1.5">8.1.5</a> Notes
<a class="toc" name="toc-Subsubsection-9.1.5">9.1.5</a> Notes
</h3>
<ul>
<li>
Some variables are infered as lists. Each token on the line separated by whitespace will be member of the list.
</li>
<li>
Comments (lines starting with #) and blank lines are ignored.
</li>

</ul>
<h2 class="Subsection">
<a class="toc" name="toc-Subsection-8.2">8.2</a> Environment variables
<a class="toc" name="toc-Subsection-9.2">9.2</a> Environment variables
</h2>
<div class="Standard">
<table>
<tr>
<td align="left" valign="top">
Variable
</td>
︙
782
783
784
785
786
787
788
789

790
791
792
793
794
795
796

865
866
867
868
869
870
871

872
873
874
875
876
877
878
879








-
+









</tr>

</table>

</div>
<h2 class="Subsection">
<a class="toc" name="toc-Subsection-8.3">8.3</a> megatest.config
<a class="toc" name="toc-Subsection-9.3">9.3</a> megatest.config
</h2>
<div class="Standard">
<table>
<tr>
<td align="left" valign="top">
section
</td>
︙
953
954
955
956
957
958
959
960

961
962
963
964
965
966
967

1036
1037
1038
1039
1040
1041
1042

1043
1044
1045
1046
1047
1048
1049
1050








-
+









</tr>

</table>

</div>
<h2 class="Subsection">
<a class="toc" name="toc-Subsection-8.4">8.4</a> runconfigs.config file
<a class="toc" name="toc-Subsection-9.4">9.4</a> runconfigs.config file
</h2>
<div class="Standard">
<table>
<tr>
<td align="left" valign="top">
section
</td>
︙
1056
1057
1058
1059
1060
1061
1062
1063

1064
1065
1066

1067
1068
1069
1070
1071
1072
1073

1139
1140
1141
1142
1143
1144
1145

1146
1147
1148

1149
1150
1151
1152
1153
1154
1155
1156








-
+


-
+








<div class="Code">
ENCRYPTION true
</div>
<div class="Code">
TESTPATH /nfs/testing/megacorp_runs
</div>
<h2 class="Subsection">
<a class="toc" name="toc-Subsection-8.5">8.5</a> Writing tests
<a class="toc" name="toc-Subsection-9.5">9.5</a> Writing tests
</h2>
<h3 class="Subsubsection">
<a class="toc" name="toc-Subsubsection-8.5.1">8.5.1</a> testconfig file
<a class="toc" name="toc-Subsubsection-9.5.1">9.5.1</a> testconfig file
</h3>
<div class="Standard">
<table>
<tr>
<td align="left" valign="top">
section
</td>
︙
1140
1141
1142
1143
1144
1145
1146
1147

1148
1149
1150
1151
1152
1153
1154

1223
1224
1225
1226
1227
1228
1229

1230
1231
1232
1233
1234
1235
1236
1237








-
+









</tr>

</table>

</div>
<h3 class="Subsubsection">
<a class="toc" name="toc-Subsubsection-8.5.2">8.5.2</a> Command line
<a class="toc" name="toc-Subsubsection-9.5.2">9.5.2</a> Command line
</h3>
<div class="Standard">
<table>
<tr>
<td align="left" valign="top">
switch or param
</td>
︙
1487
1488
1489
1490
1491
1492
1493
1494

1495
1496
1497
1498

1570
1571
1572
1573
1574
1575
1576

1577
1578
1579










-
+


-
-

</h1>
<h1 class="Section">
<a class="toc" name="toc-Section-B">B</a> References
</h1>

<hr class="footer"/>
<div class="footer" id="generated-by">
Document generated by <a href="http://elyxer.nongnu.org/">eLyXer 1.0.0 (2010-07-21)</a> on <span class="create-date">2011-09-20T23:11:19.692083</span>
Document generated by <a href="http://elyxer.nongnu.org/">eLyXer 1.0.0 (2010-07-21)</a> on <span class="create-date">2011-09-20T23:44:22.141636</span>
</div>
</div>
</body>
</html>

This page was generated in about 0.098s by Fossil 2.24 [452f8e1ffc] 2024-03-26 15:36:00