Megatest

Diff
Login

Differences From Artifact [45163346ae]:

To Artifact [eb7e0a0d9a]:


65
66
67
68
69
70
71

72

73
74
75
76
77
78
79
-------------------------
[a/2/b]
VAR1 VAL2
-------------------------

Can use either wildcard of "%" or a regular expression:


[/abc.*def/]


Disk Space Checks
^^^^^^^^^^^^^^^^^

Some parameters you can put in the [setup] section of megatest.config:

-------------------







>

>







65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
-------------------------
[a/2/b]
VAR1 VAL2
-------------------------

Can use either wildcard of "%" or a regular expression:

-------------------------
[/abc.*def/]
-------------------------

Disk Space Checks
^^^^^^^^^^^^^^^^^

Some parameters you can put in the [setup] section of megatest.config:

-------------------
141
142
143
144
145
146
147












148
149
150
151
152
153
154
Attempt to rerun tests in "STUCK/DEAD", "n/a", "ZERO_ITEMS" states.

.In megatest.config
------------------
[setup]
reruns 5
------------------













Run time limit
++++++++++++++

-----------------
[setup]
# this will automatically kill the test if it runs for more than 1h 2m and 3s







>
>
>
>
>
>
>
>
>
>
>
>







143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
Attempt to rerun tests in "STUCK/DEAD", "n/a", "ZERO_ITEMS" states.

.In megatest.config
------------------
[setup]
reruns 5
------------------

Replace the default blacklisted environment variables with user supplied
list.

Default list: USER HOME DISPLAY LS_COLORS XKEYSYMDB EDITOR MAKEFLAGS MAKEF MAKEOVERRIDES

.Add a "bad" variable "PROMPT" to the variables that will be commented out
in the megatest.sh and megatest.csh files:
-----------------
[setup]
blacklistvars USER HOME DISPLAY LS_COLORS XKEYSYMDB EDITOR MAKEFLAGS PROMPT
-----------------

Run time limit
++++++++++++++

-----------------
[setup]
# this will automatically kill the test if it runs for more than 1h 2m and 3s
284
285
286
287
288
289
290

291
292
293

294





295
296
297
298
299



300





301
302
303



304

















305
306
307
308
309

310
311
312
313

314
315

316










317



318
319
320
321
322
323
324
325
326
327
328

329
330
331
332
333
334
335
Itemmap Handling
~~~~~~~~~~~~~~~~

For cases were the dependent test has a similar but not identical
itempath to the downstream test an itemmap can allow for itemmatch
mode


-------------------
[requirements]
mode itemmatch

itemmap .*x/ y/






# ## pattern replacement notes
#
# ## Example
# ## Remove everything up to the last /



itemmap .*/





#
# ## Example
# ## Replace foo/ with bar/



itemmap foo/ bar/


















# multi-line; matches are applied in the listed order
# The following would map:
#   a123b321 to b321fooa123 then to 321fooa123p
#

itemmap (a\d+)(b\d+) \2foo\1
  b(.*) \1p
-------------------


Complex mappings
^^^^^^^^^^^^^^^^












Complex mappings can be handled with the [itemmap] section




// image::itemmap.png[]
image::complex-itemmap.png[]

Example:

. Request to run D/1/res
. Megatest uses rule "(\d+)/res" -> "\1/aa" to create item C/1/aa from D/1/res
. Full list to be run is now: D/1/res, C/1/aa
. Megatest uses rule "(\d+)/aa" -> "aa/\1" to create item A/aa/1
. Full list to be run is now: D/1/res, C/1/aa, A/aa/1


.Testconfig for Test C
----------------------
[requirements]
waiton A B

[itemmap]







>


|
>

>
>
>
>
>





>
>
>

>
>
>
>
>



>
>
>

>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>





>




>
|
|
>

>
>
>
>
>
>
>
>
>
>
|
>
>
>




<

<
<
<
<
<
>







298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386

387





388
389
390
391
392
393
394
395
Itemmap Handling
~~~~~~~~~~~~~~~~

For cases were the dependent test has a similar but not identical
itempath to the downstream test an itemmap can allow for itemmatch
mode

.example for removing part of itemmap for waiton test (eg: item +foo-x/bar+ depends on waiton's item +y/bar+)
-------------------
[requirements]
mode itemwait
# itemmap <item pattern for this test>  <item replacement pattern for waiton test>
itemmap .*x/ y/

-------------------

.example for removing part of itemmap for waiton test (eg: item +foo/bar/baz+ in this test depends on waiton's item +baz+)
-------------------

# ## pattern replacement notes
#
# ## Example
# ## Remove everything up to the last /
[requirements]
mode itemwait
# itemmap <item pattern for this test> <nothing here indicates removal>
itemmap .*/
-------------------

.example replacing part of itemmap for (eg: item +foo/1234+ will imply waiton's item +bar/1234+)
-------------------

#
# ## Example
# ## Replace foo/ with bar/
[requirements]
mode itemwait
# itemmap <item pattern for this test>  <item replacement pattern for waiton test>
itemmap foo/ bar/

-------------------

.example for backreference (eg: item +foo23/thud+ will imply waiton's item +num-23/bar/thud+
-------------------
#
# ## Example
# ## can use \{number} in replacement pattern to backreference a (capture) from matching pattern similar to sed or perl
[requirements]
mode itemwait
# itemmap <item pattern for this test>  <item replacement pattern for waiton test>
itemmap foo(\d+)/ num-\1/bar/

-------------------

.example multiple itemmaps
-------------------

# multi-line; matches are applied in the listed order
# The following would map:
#   a123b321 to b321fooa123 then to 321fooa123p
#
[requirements]
itemmap (a\d+)(b\d+) \2foo\1
  b(.*) \1p
-------------------


Complex mapping
^^^^^^^^^^^^^^^
Complex mappings can be handled with a separate [itemmap] section (instead if an itemmap line in the [requirements] section)

Each line in an itemmap section starts with a waiton test name followed by an itemmap expression

.eg: The following causes waiton test A item +bar/1234+ to run when our test's +foo/1234+ item is requested as well as causing waiton test B's +blah+ item to run when our test's +stuff/blah+ item is requested
--------------
[itemmap]
A foo/ bar/
B stuff/
--------------


Complex mapping example
^^^^^^^^^^^^^^^^^^^^^^^



// image::itemmap.png[]
image::complex-itemmap.png[]








We accomplish this by configuring the testconfigs of our tests C D and E as follows:

.Testconfig for Test C
----------------------
[requirements]
waiton A B

[itemmap]
346
347
348
349
350
351
352









353
354
355
356
357
358
359

.Testconfig for Test E
----------------------
[requirements]
waiton C
itemmap (\d+)/res \1/bb
----------------------










Dynamic Flow Dependency Tree
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.Autogeneration waiton list for dynamic flow dependency trees
-------------------
[requirements]







>
>
>
>
>
>
>
>
>







406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428

.Testconfig for Test E
----------------------
[requirements]
waiton C
itemmap (\d+)/res \1/bb
----------------------

Example from image just above, we want the following to occur:

. We want the above to execute when we request pattern +D/1/res+, eg from command line +megatest -run -testpatt D/1/res -target ... -runname ...+
. "(\d+)/res" -> "\1/aa" to require +C/1/aa+ be executed before +D/1/res+
. Full list to be run is now: D/1/res, C/1/aa
. "(\d+)/aa" -> "aa/\1" to create item +A/aa/1+
. Full list to be run is now: +D/1/res+, +C/1/aa+, +A/aa/1+


Dynamic Flow Dependency Tree
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.Autogeneration waiton list for dynamic flow dependency trees
-------------------
[requirements]
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499























500
501
502
503
504
505
506
----------------------------
$MT_MEGATEST -env2file .ezsteps/${stepname}	  
----------------------------

Triggers
~~~~~~~~

In your testconfig triggers can be specified 

-----------------
[triggers]

# Call script running.sh when test goes to state=RUNNING, status=PASS
RUNNING/PASS running.sh

# Call script running.sh any time state goes to RUNNING
RUNNING/ running.sh

# Call script onpass.sh any time status goes to PASS
PASS/ onpass.sh
-----------------

Scripts called will have; test-id test-rundir trigger, added to the commandline.

HINT

To start an xterm (useful for debugging), use a command line like the following:

-----------------
[triggers]
COMPLETED/ xterm -e bash -s -- 
-----------------

NOTE: There is a trailing space after the --

























Override the Toplevel HTML File
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Megatest generates a simple html file summary for top level tests of
iterated tests. The generation can be overridden. NOTE: the output of







|














|











>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
----------------------------
$MT_MEGATEST -env2file .ezsteps/${stepname}	  
----------------------------

Triggers
~~~~~~~~

In your testconfig or megatest.config triggers can be specified 

-----------------
[triggers]

# Call script running.sh when test goes to state=RUNNING, status=PASS
RUNNING/PASS running.sh

# Call script running.sh any time state goes to RUNNING
RUNNING/ running.sh

# Call script onpass.sh any time status goes to PASS
PASS/ onpass.sh
-----------------

Scripts called will have; test-id test-rundir trigger test-name item-path state status event-time, added to the commandline.

HINT

To start an xterm (useful for debugging), use a command line like the following:

-----------------
[triggers]
COMPLETED/ xterm -e bash -s -- 
-----------------

NOTE: There is a trailing space after the --

There are a number of environment variables available to the trigger script
but since triggers can be called in various contexts not all variables are
available at all times. The trigger script should check for the variable and
fail gracefully if it doesn't exist.

.Environment variables visible to the trigger script
[width="90%",cols="^,2m",frame="topbot",options="header"]
|======================
|Variable             | Purpose            
| MT_TEST_RUN_DIR     | The directory where Megatest ran this test                   
| MT_CMDINFO          | Encoded command data for the test                   
| MT_DEBUG_MODE       | Used to pass the debug mode to nested calls to Megatest                   
| MT_RUN_AREA_HOME    | Megatest home area 
| MT_TESTSUITENAME    | The name of this testsuite or area                   
| MT_TEST_NAME        | The name of this test
| MT_ITEM_INFO        | The variable and values for the test item
| MT_MEGATEST         | Which Megatest binary is being used by this area
| MT_TARGET           | The target variable values, separated by '/'
| MT_LINKTREE         | The base of the link tree where all run tests can be found
| MT_ITEMPATH         | The values of the item path variables, separated by '/'
| MT_RUNNAME          | The name of the run
|======================


Override the Toplevel HTML File
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Megatest generates a simple html file summary for top level tests of
iterated tests. The generation can be overridden. NOTE: the output of