Megatest

Diff
Login

Differences From Artifact [4a0af0abde]:

To Artifact [e5228eb513]:


119
120
121
122
123
124
125


126
127



128


129
130
131
132
133
134
135
# script that takes path as parameter and returns number of bytes available:
free-space-script check-space.sh
-------------------

Trim trailing spaces
~~~~~~~~~~~~~~~~~~~~



------------------
[configf:settings trim-trailing-spaces yes]



------------------



Job Submission Control
~~~~~~~~~~~~~~~~~~~~~~

Submit jobs to Host Types based on Test Name
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^








>
>

|
>
>
>

>
>







119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
# script that takes path as parameter and returns number of bytes available:
free-space-script check-space.sh
-------------------

Trim trailing spaces
~~~~~~~~~~~~~~~~~~~~

NOTE: As of Megatest version v1.6548 trim-trailing-spaces defaults to yes.

------------------
[configf:settings trim-trailing-spaces no]
#                 |<== next line padded with spaces to here
DEFAULT_INDENT
[configf:settings trim-trailing-spaces no]
------------------

The variable DEFAULT_INDENT would be a string of 3 spaces

Job Submission Control
~~~~~~~~~~~~~~~~~~~~~~

Submit jobs to Host Types based on Test Name
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

196
197
198
199
200
201
202

















203
204
205
206
207
208
209

-----------------
[setup]
# this will automatically kill the test if it runs for more than 1h 2m and 3s
runtimelim 1h 2m 3s
-----------------


















Tests browser view
~~~~~~~~~~~~~~~~~~

The tests browser (see the Run Control tab on the dashboard) has two views for displaying the tests. 

. Dot (graphviz) based tree
. No dot, plain listing







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233

-----------------
[setup]
# this will automatically kill the test if it runs for more than 1h 2m and 3s
runtimelim 1h 2m 3s
-----------------

Post Run Hook
+++++++++++++

This runs script to-run.sh after all tests have been completed. It is
not necessary to use -run-wait as each test will check for other
running tests on completion and if there are none it will call the
post run hook.

Note that the output from the script call will be placed in a log file
in the logs directory with a file name derived by replacing / with _
in post-hook-<target>-<runname>.log.

-------------------
[runs]
post-hook /path/to/script/to-run.sh
-------------------

Tests browser view
~~~~~~~~~~~~~~~~~~

The tests browser (see the Run Control tab on the dashboard) has two views for displaying the tests. 

. Dot (graphviz) based tree
. No dot, plain listing
353
354
355
356
357
358
359



















360
361
362
363
364
365
366
~~~~~~~~~~~~~~~~~~~

-------------------
# A normal waiton waits for the prior tests to be COMPLETED 
# and PASS, CHECK or WAIVED
waiton test1 test2
-------------------




















Mode
~~~~

The default (i.e. if mode is not specified) is normal. All pre-dependent tests
must be COMPLETED and PASS, CHECK or WAIVED before the test will start








>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
~~~~~~~~~~~~~~~~~~~

-------------------
# A normal waiton waits for the prior tests to be COMPLETED 
# and PASS, CHECK or WAIVED
waiton test1 test2
-------------------

NOTE: Dynamic waiton lists must be capable of being calculated at the
beginning of a run. This is because Megatest walks the tree of waitons
to create the list of tests to execute.

.This works
-------------------
waiton [system somescript.sh]
-------------------

.This does NOT work (the full context for the test is not available so #{shell ...} is NOT enabled to evaluate.
-------------------
waiton #{shell somescript.sh}
-------------------

.This does NOT work
-------------------
waiton [system somescript_that_depends_on_a_prior_test.sh]
-------------------

Mode
~~~~

The default (i.e. if mode is not specified) is normal. All pre-dependent tests
must be COMPLETED and PASS, CHECK or WAIVED before the test will start

620
621
622
623
624
625
626
627



628

629
630


631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649



























650
651






652
653
654
655
656
657
658
659
660
661
662
663
664


665



666




667

668



669

670






































671
672
673
674
675
676
677
A disks section in testconfig will override the disks section in
megatest.config. This can be used to allocate disks on a per-test or per item
basis.

Controlled waiver propagation
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If test is FAIL and previous test in run with same MT_TARGET is WAIVED then apply the following rules from the testconfig:



If a waiver check is specified in the testconfig apply the check and if it passes then set this FAIL to WAIVED


Waiver check has two parts, 1) a list of waiver, rulename, filepatterns and 2) the rulename script spec (note that "diff" and "logpro" are predefined)



-----------------
###### EXAMPLE FROM testconfig #########
# matching file(s) will be diff'd with previous run and logpro applied
# if PASS or WARN result from logpro then WAIVER state is set
#
[waivers]
# logpro_file    rulename      input_glob
waiver_1         logpro        lookittmp.log

[waiver_rules]

# This builtin rule is the default if there is no <waivername>.logpro file
# diff   diff %file1% %file2%

# This builtin rule is applied if a <waivername>.logpro file exists
# logpro diff %file1% %file2% | logpro %waivername%.logpro %waivername%.html
-----------------




























Ezsteps
~~~~~~~







.Example ezsteps with logpro rules
-----------------
[ezsteps]
lookittmp   ls /tmp

[logpro]
lookittmp ;; Note: config file format supports multi-line entries where leading whitespace is removed from each line
  ;;     a blank line indicates the end of the block of text 
  (expect:required in "LogFileBody" > 0 "A file name that should never exist!" #/This is a awfully stupid file name that should never be found in the temp dir/)

-----------------



To transfer the environment to the next step you can do the following:








.Propagate environment to next step

----------------------------



$MT_MEGATEST -env2file .ezsteps/${stepname}	  

----------------------------







































Scripts
~~~~~~~

.Specifying scripts inline (best used for only simple scripts)
----------------------------
[scripts]







|
>
>
>
|
>

|
>
>



















>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>


>
>
>
>
>
>













>
>
|
>
>
>

>
>
>
>
|
>
|
>
>
>
|
>
|
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
A disks section in testconfig will override the disks section in
megatest.config. This can be used to allocate disks on a per-test or per item
basis.

Controlled waiver propagation
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If test is FAIL and previous test in run with same MT_TARGET is WAIVED
or if the test/itempath is listed under the matching target in the
waivers roll forward file (see below for file spec) then apply the
following rules from the testconfig: If a waiver check is specified in
the testconfig apply the check and if it passes then set this FAIL to
WAIVED

Waiver check has two parts, 1) a list of waiver, rulename,
filepatterns and 2) the rulename script spec (note that "diff" and
"logpro" are predefined)

-----------------
###### EXAMPLE FROM testconfig #########
# matching file(s) will be diff'd with previous run and logpro applied
# if PASS or WARN result from logpro then WAIVER state is set
#
[waivers]
# logpro_file    rulename      input_glob
waiver_1         logpro        lookittmp.log

[waiver_rules]

# This builtin rule is the default if there is no <waivername>.logpro file
# diff   diff %file1% %file2%

# This builtin rule is applied if a <waivername>.logpro file exists
# logpro diff %file1% %file2% | logpro %waivername%.logpro %waivername%.html
-----------------

Waiver roll-forward files
^^^^^^^^^^^^^^^^^^^^^^^^^

To transfer waivers from one Megatest area to another it is possible
to dump waivers into a file and reference that file in another area.

.Dumping the waivers
---------------------------
megatest -list-waivers -runname %-a > mywaivers.dat
---------------------------

.Referencing the saved waivers
---------------------------
# In megatest.config, all files listed will be loaded - recomended to use
# variables to select directorys to minimize what gets loaded.
[setup]
waivers-dirs /path/to/waiver/files /another/path/to/waiver/files
---------------------------

.Waiver files format
---------------------------
[the/target/here]
# comments are fine
testname1/itempath A comment about why it was waived
testname2          A comment for a non-itemized test 
---------------------------

Ezsteps
~~~~~~~

Ezsteps is the recommended way to implement tests and automation in
Megatest.

NOTE: Each ezstep must be a single line. Use the [scripts] mechanism
to create multiline scripts (see example below).

.Example ezsteps with logpro rules
-----------------
[ezsteps]
lookittmp   ls /tmp

[logpro]
lookittmp ;; Note: config file format supports multi-line entries where leading whitespace is removed from each line
  ;;     a blank line indicates the end of the block of text 
  (expect:required in "LogFileBody" > 0 "A file name that should never exist!" #/This is a awfully stupid file name that should never be found in the temp dir/)

-----------------

Automatic environment propagation with Ezsteps
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Turn on ezpropvars and environment variables will be propagated from
step to step. Use this to source script files that modify the
envionment where the modifications are needed in subsequent steps.

NOTE: aliases and variables with strange whitespace or characters will
not propagate correctly. Put in a ticket on the
http://www.kiatoa.com/fossils/megatest site if you need support for a
specific strange character combination.

.Turn on auto propagate for bash
---------------------------
[setup]
ezpropvars sh
---------------------------

.Write your ezsteps. The loadenv.csh step will use /bin/csh as its shell, other steps will use bash.
---------------------------
[ezsteps]
loadenv.csh source $REF/ourenviron.csh
compile make
install make install
---------------------------

Bash and csh are supported. You can override the shell binary location
from the default /bin/bash and /bin/csh if needed.

.Turn on auto propagate for csh
---------------------------
[setup]
ezpropvars csh /bin/csh
---------------------------

.Example of auto propagation using extensions
---------------------------
[ezsteps]
step1.sh export SOMEVAR=$(ps -def | wc -l);ls /tmp
# The next step will get the value of $SOMEVAR from step1.sh
step2.sh echo $SOMEVAR
---------------------------

.Example of multi-line script
---------------------------
[scripts]
tarresults tar cfvz $DEST/srcdir1.tar.gz srcdir1
  tar cfvz $DEST/srcdir2.tar.gz srcdir2

[setup]
ezpropvars sh

[ezsteps]
step1 DEST=/tmp/targz;source tarresults
---------------------------

The above example will result in files; tarresults and ez_step1 being
created in the test dir.

Scripts
~~~~~~~

.Specifying scripts inline (best used for only simple scripts)
----------------------------
[scripts]