Megatest

Check-in [392f89b8e5]
Login
☰Home Timeline Files Branches Tags Tickets Wiki Login
Overview
Comment:stuff
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | v1.65-debugging-update-orig
Files: files | file ages | folders
SHA1: 392f89b8e5520d1a5917f527c0f66d1020e3360f
User & Date: mrwellan on 2021-09-10 10:20:38
Other Links: branch diff | manifest | tags
Context
2021-09-10
11:41
Updates to debugging section of manual check-in: 43b9e9cf05 user: mrwellan tags: v1.65-debugging-update-orig
10:20
stuff check-in: 392f89b8e5 user: mrwellan tags: v1.65-debugging-update-orig
05:54
Merged couple changes made accidentally on trunk check-in: ee69c12599 user: matt tags: v1.65-debugging-update-orig
Changes
Hide Diffs Unified Diffs Ignore Whitespace Patch

Modified docs/manual/bisecting.png from [6161dfa9e5] to [81573d185b].

cannot compute difference between binary files

Modified docs/manual/debugging.txt from [b4cb39220d] to [9d2c5b9df9].

︙︙
13
14
15
16
17
18
19


















20
21
22
23
24
25
26

//     GNU General Public License for more details.
// 
//     You should have received a copy of the GNU General Public License
//     along with Megatest.  If not, see <http://www.gnu.org/licenses/>.

Debugging
---------



















A word on Bisecting
~~~~~~~~~~~~~~~~~~~

Bisecting is a debug strategy intended to speed up finding the root
cause.









>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>








13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

//     GNU General Public License for more details.
// 
//     You should have received a copy of the GNU General Public License
//     along with Megatest.  If not, see <http://www.gnu.org/licenses/>.

Debugging
---------

Well Written Tests
~~~~~~~~~~~~~~~~~~

Test Design and Surfacing
^^^^^^^^^^^^^^^^^^^^^^^^^

Design your tests to surface errors. Ensure that all logs are processed by logpro and can be reached by a mouse click or two from the test control panel.



using the tool to make life easier

layers - less is often more


Clicks-to-error


A word on Bisecting
~~~~~~~~~~~~~~~~~~~

Bisecting is a debug strategy intended to speed up finding the root
cause.
︙︙
75
76
77
78
79
80
81
















82
83
84
85
86
87
88

step" to only rerun the step or click on "restart from here" to rerun
that step and downstream steps.

NOTE 1: visual feedback can take some time, give it a few seconds and
you will see the step change color to blue as it starts running.

NOTE 2: steping through only works if you are using ezsteps.

















Config File Processing
^^^^^^^^^^^^^^^^^^^^^^

As described above it is often helpful to know the content of
variables in various contexts as Megatest works through the actions
needed to run your tests. A handy technique is to force the startup of








>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>








93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122

step" to only rerun the step or click on "restart from here" to rerun
that step and downstream steps.

NOTE 1: visual feedback can take some time, give it a few seconds and
you will see the step change color to blue as it starts running.

NOTE 2: steping through only works if you are using ezsteps.

Bisecting megatest.csh/sh
^^^^^^^^^^^^^^^^^^^^^^^^^

Sometimes finding the environment variable that is causing the problem can be very difficult. Bisection can be applied.

Edit the megatest.csh or megatest.sh file and comment out 50% per round, source in fresh xterm and run the test.

This idea can also be applied to your .cshrc, .bashrc, .aliases and other similar files.

csh and -f
^^^^^^^^^^

A common issue when tcsh or csh shells are used for scripting is to forget or choose to not use -f in your #! line

.

Config File Processing
^^^^^^^^^^^^^^^^^^^^^^

As described above it is often helpful to know the content of
variables in various contexts as Megatest works through the actions
needed to run your tests. A handy technique is to force the startup of
︙︙

Modified docs/manual/megatest-test-stages.png from [6b9ff2543b] to [e53ee8abbe].

cannot compute difference between binary files

Modified docs/manual/megatest_manual.html from [3f040e4a9a] to [23e5f0702c]. 

1
2
3
4
5
6
7
8
9
10
11
12

<!DOCTYPE html>
<html lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name="generator" content="AsciiDoc 8.6.10">
<title>The Megatest Users Manual</title>
<style type="text/css">
/* Shared CSS for AsciiDoc xhtml11 and html5 backends */

/* Default font. */
body {
  font-family: Georgia,serif;





|








1
2
3
4
5
6
7
8
9
10
11
12

<!DOCTYPE html>
<html lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name="generator" content="AsciiDoc 8.6.7">
<title>The Megatest Users Manual</title>
<style type="text/css">
/* Shared CSS for AsciiDoc xhtml11 and html5 backends */

/* Default font. */
body {
  font-family: Georgia,serif;
︙︙
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105


ul, ol, li > p {
  margin-top: 0;
}
ul > li     { color: #aaa; }
ul > li > * { color: black; }

.monospaced, code, pre {
  font-family: "Courier New", Courier, monospace;
  font-size: inherit;
  color: navy;
  padding: 0;
  margin: 0;
}
pre {
  white-space: pre-wrap;
}

#author {
  color: #527bbd;
  font-weight: bold;
  font-size: 1.1em;
}
#email {








|
<
<
<



<
<
<








82
83
84
85
86
87
88
89



90
91
92



93
94
95
96
97
98
99


ul, ol, li > p {
  margin-top: 0;
}
ul > li     { color: #aaa; }
ul > li > * { color: black; }

pre {



  padding: 0;
  margin: 0;
}




#author {
  color: #527bbd;
  font-weight: bold;
  font-size: 1.1em;
}
#email {
︙︙
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234


div.exampleblock > div.content {
  border-left: 3px solid #dddddd;
  padding-left: 0.5em;
}

div.imageblock div.content { padding-left: 0; }
span.image img { border-style: none; vertical-align: text-bottom; }
a.image:visited { color: white; }

dl {
  margin-top: 0.8em;
  margin-bottom: 0.8em;
}
dt {








|








214
215
216
217
218
219
220
221
222
223
224
225
226
227
228


div.exampleblock > div.content {
  border-left: 3px solid #dddddd;
  padding-left: 0.5em;
}

div.imageblock div.content { padding-left: 0; }
span.image img { border-style: none; }
a.image:visited { color: white; }

dl {
  margin-top: 0.8em;
  margin-bottom: 0.8em;
}
dt {
︙︙
415
416
417
418
419
420
421






422
423
424
425
426
427
428

div.unbreakable { page-break-inside: avoid; }


/*
 * xhtml11 specific
 *
 * */







div.tableblock {
  margin-top: 1.0em;
  margin-bottom: 1.5em;
}
div.tableblock > table {
  border: 3px solid #527bbd;








>
>
>
>
>
>








409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428

div.unbreakable { page-break-inside: avoid; }


/*
 * xhtml11 specific
 *
 * */

tt {
  font-family: "Courier New", Courier, monospace;
  font-size: inherit;
  color: navy;
}

div.tableblock {
  margin-top: 1.0em;
  margin-bottom: 1.5em;
}
div.tableblock > table {
  border: 3px solid #527bbd;
︙︙
448
449
450
451
452
453
454






455
456
457
458
459
460
461

}


/*
 * html5 specific
 *
 * */







table.tableblock {
  margin-top: 1.0em;
  margin-bottom: 1.5em;
}
thead, p.tableblock.header {
  font-weight: bold;








>
>
>
>
>
>








448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467

}


/*
 * html5 specific
 *
 * */

.monospaced {
  font-family: "Courier New", Courier, monospace;
  font-size: inherit;
  color: navy;
}

table.tableblock {
  margin-top: 1.0em;
  margin-bottom: 1.5em;
}
thead, p.tableblock.header {
  font-weight: bold;
︙︙
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543

body.manpage div.sectionbody {
  margin-left: 3em;
}

@media print {
  body.manpage div#toc { display: none; }
}


@media screen {
  body {
    max-width: 50em; /* approximately 80 characters wide */
    margin-left: 16em;
  }

  #toc {








<
<








534
535
536
537
538
539
540


541
542
543
544
545
546
547

body.manpage div.sectionbody {
  margin-left: 3em;
}

@media print {
  body.manpage div#toc { display: none; }
}


@media screen {
  body {
    max-width: 50em; /* approximately 80 characters wide */
    margin-left: 16em;
  }

  #toc {
︙︙
1929
1930
1931
1932
1933
1934
1935










1936
1937
1938
1939
1940
1941
1942

needed for a test step into a script it is not necessary.</p></div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_debugging">Debugging</h2>
<div class="sectionbody">










<div class="sect2">
<h3 id="_a_word_on_bisecting">A word on Bisecting</h3>
<div class="paragraph"><p>Bisecting is a debug strategy intended to speed up finding the root
cause.</p></div>
<div class="imageblock graphviz">
<div class="content">
<img src="bisecting.png" alt="bisecting.png">








>
>
>
>
>
>
>
>
>
>








1933
1934
1935
1936
1937
1938
1939
1940
1941
1942
1943
1944
1945
1946
1947
1948
1949
1950
1951
1952
1953
1954
1955
1956

needed for a test step into a script it is not necessary.</p></div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_debugging">Debugging</h2>
<div class="sectionbody">
<div class="sect2">
<h3 id="_well_written_tests">Well Written Tests</h3>
<div class="sect3">
<h4 id="_test_design_and_surfacing">Test Design and Surfacing</h4>
<div class="paragraph"><p>Design your tests to surface errors. Ensure that all logs are processed by logpro and can be reached by a mouse click or two from the test control panel.</p></div>
<div class="paragraph"><p>using the tool to make life easier</p></div>
<div class="paragraph"><p>layers - less is often more</p></div>
<div class="paragraph"><p>Clicks-to-error</p></div>
</div>
</div>
<div class="sect2">
<h3 id="_a_word_on_bisecting">A word on Bisecting</h3>
<div class="paragraph"><p>Bisecting is a debug strategy intended to speed up finding the root
cause.</p></div>
<div class="imageblock graphviz">
<div class="content">
<img src="bisecting.png" alt="bisecting.png">
︙︙
2411
2412
2413
2414
2415
2416
2417
2418
2419
2420
2421
2422
2423
2424
2425

</div></div>
</div>
<div class="sect2">
<h3 id="_trim_trailing_spaces">Trim trailing spaces</h3>
<div class="admonitionblock">
<table><tr>
<td class="icon">
<img src="/usr/images/icons/note.png" alt="Note">
</td>
<td class="content">As of Megatest version v1.6548 trim-trailing-spaces defaults to yes.</td>
</tr></table>
</div>
<div class="listingblock">
<div class="content monospaced">
<pre>[configf:settings trim-trailing-spaces no]








|








2425
2426
2427
2428
2429
2430
2431
2432
2433
2434
2435
2436
2437
2438
2439

</div></div>
</div>
<div class="sect2">
<h3 id="_trim_trailing_spaces">Trim trailing spaces</h3>
<div class="admonitionblock">
<table><tr>
<td class="icon">
<img src="/nfs/pdx/disks/ice.disk.002/icfadm/pkgs/asciidoc/8.6.7/images/icons/note.png" alt="Note">
</td>
<td class="content">As of Megatest version v1.6548 trim-trailing-spaces defaults to yes.</td>
</tr></table>
</div>
<div class="listingblock">
<div class="content monospaced">
<pre>[configf:settings trim-trailing-spaces no]
︙︙
2728
2729
2730
2731
2732
2733
2734
2735
2736
2737
2738
2739
2740
2741
2742

<pre># A normal waiton waits for the prior tests to be COMPLETED
# and PASS, CHECK or WAIVED
waiton test1 test2</pre>
</div></div>
<div class="admonitionblock">
<table><tr>
<td class="icon">
<img src="/usr/images/icons/note.png" alt="Note">
</td>
<td class="content">Dynamic waiton lists must be capable of being calculated at the
beginning of a run. This is because Megatest walks the tree of waitons
to create the list of tests to execute.</td>
</tr></table>
</div>
<div class="listingblock">








|








2742
2743
2744
2745
2746
2747
2748
2749
2750
2751
2752
2753
2754
2755
2756

<pre># A normal waiton waits for the prior tests to be COMPLETED
# and PASS, CHECK or WAIVED
waiton test1 test2</pre>
</div></div>
<div class="admonitionblock">
<table><tr>
<td class="icon">
<img src="/nfs/pdx/disks/ice.disk.002/icfadm/pkgs/asciidoc/8.6.7/images/icons/note.png" alt="Note">
</td>
<td class="content">Dynamic waiton lists must be capable of being calculated at the
beginning of a run. This is because Megatest walks the tree of waitons
to create the list of tests to execute.</td>
</tr></table>
</div>
<div class="listingblock">
︙︙
3087
3088
3089
3090
3091
3092
3093
3094
3095
3096
3097
3098
3099
3100
3101

<div class="sect2">
<h3 id="_ezsteps">Ezsteps</h3>
<div class="paragraph"><p>Ezsteps is the recommended way to implement tests and automation in
Megatest.</p></div>
<div class="admonitionblock">
<table><tr>
<td class="icon">
<img src="/usr/images/icons/note.png" alt="Note">
</td>
<td class="content">Each ezstep must be a single line. Use the [scripts] mechanism
to create multiline scripts (see example below).</td>
</tr></table>
</div>
<div class="listingblock">
<div class="title">Example ezsteps with logpro rules</div>








|








3101
3102
3103
3104
3105
3106
3107
3108
3109
3110
3111
3112
3113
3114
3115

<div class="sect2">
<h3 id="_ezsteps">Ezsteps</h3>
<div class="paragraph"><p>Ezsteps is the recommended way to implement tests and automation in
Megatest.</p></div>
<div class="admonitionblock">
<table><tr>
<td class="icon">
<img src="/nfs/pdx/disks/ice.disk.002/icfadm/pkgs/asciidoc/8.6.7/images/icons/note.png" alt="Note">
</td>
<td class="content">Each ezstep must be a single line. Use the [scripts] mechanism
to create multiline scripts (see example below).</td>
</tr></table>
</div>
<div class="listingblock">
<div class="title">Example ezsteps with logpro rules</div>
︙︙
3112
3113
3114
3115
3116
3117
3118
3119
3120
3121
3122
3123
3124
3125
3126

<h4 id="_automatic_environment_propagation_with_ezsteps">Automatic environment propagation with Ezsteps</h4>
<div class="paragraph"><p>Turn on ezpropvars and environment variables will be propagated from
step to step. Use this to source script files that modify the
envionment where the modifications are needed in subsequent steps.</p></div>
<div class="admonitionblock">
<table><tr>
<td class="icon">
<img src="/usr/images/icons/note.png" alt="Note">
</td>
<td class="content">aliases and variables with strange whitespace or characters will
not propagate correctly. Put in a ticket on the
<a href="http://www.kiatoa.com/fossils/megatest">http://www.kiatoa.com/fossils/megatest</a> site if you need support for a
specific strange character combination.</td>
</tr></table>
</div>








|








3126
3127
3128
3129
3130
3131
3132
3133
3134
3135
3136
3137
3138
3139
3140

<h4 id="_automatic_environment_propagation_with_ezsteps">Automatic environment propagation with Ezsteps</h4>
<div class="paragraph"><p>Turn on ezpropvars and environment variables will be propagated from
step to step. Use this to source script files that modify the
envionment where the modifications are needed in subsequent steps.</p></div>
<div class="admonitionblock">
<table><tr>
<td class="icon">
<img src="/nfs/pdx/disks/ice.disk.002/icfadm/pkgs/asciidoc/8.6.7/images/icons/note.png" alt="Note">
</td>
<td class="content">aliases and variables with strange whitespace or characters will
not propagate correctly. Put in a ticket on the
<a href="http://www.kiatoa.com/fossils/megatest">http://www.kiatoa.com/fossils/megatest</a> site if you need support for a
specific strange character combination.</td>
</tr></table>
</div>
︙︙
3300
3301
3302
3303
3304
3305
3306
3307
3308
3309
3310
3311
3312
3313
3314

<div class="content monospaced">
<pre>[triggers]
COMPLETED/ xterm -e bash -s --</pre>
</div></div>
<div class="admonitionblock">
<table><tr>
<td class="icon">
<img src="/usr/images/icons/note.png" alt="Note">
</td>
<td class="content">There is a trailing space after the double-dash</td>
</tr></table>
</div>
<div class="paragraph"><p>There are a number of environment variables available to the trigger script
but since triggers can be called in various contexts not all variables are
available at all times. The trigger script should check for the variable and








|








3314
3315
3316
3317
3318
3319
3320
3321
3322
3323
3324
3325
3326
3327
3328

<div class="content monospaced">
<pre>[triggers]
COMPLETED/ xterm -e bash -s --</pre>
</div></div>
<div class="admonitionblock">
<table><tr>
<td class="icon">
<img src="/nfs/pdx/disks/ice.disk.002/icfadm/pkgs/asciidoc/8.6.7/images/icons/note.png" alt="Note">
</td>
<td class="content">There is a trailing space after the double-dash</td>
</tr></table>
</div>
<div class="paragraph"><p>There are a number of environment variables available to the trigger script
but since triggers can be called in various contexts not all variables are
available at all times. The trigger script should check for the variable and
︙︙
3637
3638
3639
3640
3641
3642
3643
3644
3645
3646
3647
3648
3649

</div>
</div>
</div>
<div id="footnotes"><hr></div>
<div id="footer">
<div id="footer-text">
Version 1.5<br>
Last updated
 2021-09-07 17:58:32 MST
</div>
</div>
</body>
</html>








|
<





3651
3652
3653
3654
3655
3656
3657
3658

3659
3660
3661
3662

</div>
</div>
</div>
<div id="footnotes"><hr></div>
<div id="footer">
<div id="footer-text">
Version 1.5<br>
Last updated 2021-09-10 06:09:41 PDT

</div>
</div>
</body>
</html>

Modified docs/manual/megatest_manual.pdf from [79c7bbbfd3] to [774cf56d82].

cannot compute difference between binary files

Modified docs/manual/reference.txt from [e5228eb513] to [624b8fd08d].

︙︙
761
762
763
764
765
766
767

768



769
770
771
772
773
774
775

[setup]
ezpropvars sh
---------------------------

.Write your ezsteps. The loadenv.csh step will use /bin/csh as its shell, other steps will use bash.
---------------------------
[ezsteps]

loadenv.csh source $REF/ourenviron.csh



compile make
install make install
---------------------------

Bash and csh are supported. You can override the shell binary location
from the default /bin/bash and /bin/csh if needed.









>

>
>
>








761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779

[setup]
ezpropvars sh
---------------------------

.Write your ezsteps. The loadenv.csh step will use /bin/csh as its shell, other steps will use bash.
---------------------------
[ezsteps]
# if your upstream file is csh you can force csh like this
loadenv.csh source $REF/ourenviron.csh
# if your upstream is bash 
loadenv     source $REF/ourenviron.sh

compile make
install make install
---------------------------

Bash and csh are supported. You can override the shell binary location
from the default /bin/bash and /bin/csh if needed.
︙︙

Modified docs/manual/server.png from [43882638fe] to [ae7d7ee58e].

cannot compute difference between binary files

This page was generated in about 0.132s by Fossil 2.24 [452f8e1ffc] 2024-03-26 15:36:00