bgcolor # mtime type status subsystem title _comments #f2dcdc e847db768e 2011-07-15 20:21:50 Feature_Request Open If no launcher go ahead and use process to put the tests in the background #f2dcdc acab36172b 2011-07-15 20:30:46 Feature_Request Open Add logging of memory consumption and space in /tmp These are often the root cause of test problems on shared machines. #f2dcdc 1565929e9c 2011-07-15 20:35:15 Feature_Request Open Add -result value for test or steps to record a result (beyond state and status values) #f2dcdc 44c3549dab 2011-11-10 22:25:10 Code_Defect Open dashboard refresh race condition. last update misses occasionally #f2dcdc 630ba8d5a6 2012-04-11 20:29:47 Code_Defect Open Running a test with pre-dependecies and -itempatt causes forever run #f2dcdc db6c35ee0c 2012-04-20 05:50:00 Code_Defect Open removal of last item dir not happening in -remove-runs, could be link is not created Remove runs on test_mt_vars and item 5 will not be deleted #f2dcdc bde7755167 2012-05-03 06:15:18 Feature_Request Open Need a test to ensure the blocking of completion by tests with failed prerequistes is caught #f2dcdc 5f826ec79c 2012-05-25 15:36:45 Feature_Request Open Set $HOME to test run area by default, allow override in megatest.config #f2dcdc 9a9a5b7339 2012-06-27 21:30:07 Feature_Request Open Ability to dump megatest db to generic format and re-import #f2dcdc 894f74c997 2012-10-11 15:56:18 Code_Defect Open Cannot set test status from commandline for parent of iterated test #f2dcdc f0a99444fd 2013-01-29 21:11:55 Feature_Request Open Add lock on -remove-runs so new run not started and remove not done on running Add locking (due to RUNNING etc.) on remove so that can safely remove symlinks #f2dcdc 290d262ed3 2013-01-29 21:13:47 Code_Defect Open megatest -generate-test crashes if script has spaces [ ] megatest -generate-test [ ] generate test issue - scripts vs. a command line. Look for |, space, quotes, / as signature of # Add additional steps here. Format is "stepname script" [ezsteps] getrepo fsl repo get `echo $TECH|sed 's/^p12/fdk/'` oalibs co_rel (cd oalibs;fsl co $RELEASE) convert convertall.sh Error: (open-output-file) cannot open file - No such file or directory: "../tests/csv_gen/fsl repo get `echo $TECH|sed 's/^p12/fdk/'` oalibs.sh" Call history: g210219 genexample.scm:240: print genexample.scm:241: print genexample.scm:242: get-environment-variable genexample.scm:242: print genexample.scm:243: get-environment-variable genexample.scm:243: print genexample.scm:244: print genexample.scm:245: print genexample.scm:246: print g245251 genexample.scm:251: conc genexample.scm:251: with-output-to-file genexample.scm:253: print genexample.scm:254: conc genexample.scm:254: with-output-to-file #f2dcdc f93339b7aa 2013-01-29 21:22:31 Feature_Request Open Launching via http or zmq transport #f2dcdc 7a8ae441af 2013-01-29 21:23:25 Feature_Request Open Per test max concurrent jobs limits #f2dcdc 96d8a782e9 2013-02-11 05:46:38 Code_Defect Open Tests; ezlog_fail, ezlog_warn, logpro_required result in PASS if logpro not avail on PATH #f2dcdc 4587f1787b 2013-03-06 20:00:25 Feature_Request Open convert all the messy environment variable blocks to use call-with-environment-variables #f2dcdc 9d45634313 2013-03-26 18:58:29 Feature_Request Open enhancements request to dashboard Qns on megatest flow / methodology: 1. Turn off option for "Press Any key to continue" while clean/re-running tests in the xterm 1. If I click execute again (accidentally), will it re-queue the tests again? 1. How to get dashboard with only the runs I want to see (only for 73 or 71 eg) 1. Collapse/uncollapse all tests - option on dashboard required 1. Checks are sorted order on website dashboard Vs unix dashboard unsorted * Option to clean and rerun in one step on dashboard #f2dcdc 613c71a508 2013-04-17 21:54:43 Feature_Request Open look at the eval of [system ] blocks and see if it can be done per test/item vs. per test. look at the eval of [system ] blocks and see if it can be done per test/item vs. per test. #f2dcdc d68357ce8d 2013-05-01 17:17:43 Code_Defect Open running -test-status to set CHECK in a linked test sets FAIL ../envqa/tests/dumpndiffcdf/main.csh:megatest -test-status : state COMPLETED : status CHECK -setlog cdfdiff_prim/index.html #f2dcdc 3d3046b2c2 2013-05-09 23:01:33 Code_Defect Open Missing escaping in dashboard test selector "SELECT id,run_id,testname,state,status,event_time,host,cpuload,diskfree,uname,rundir,item_path,run_duration,final_logf,comment FROM tests WHERE run_id=? AND ((testname LIKE '%if %'' AND item_path LIKE '%')) ;" Call history: db.scm:564: string-intersperse tests.scm:100: db:patt->like db.scm:560: ##sys#get-keyword db.scm:564: g779 substring-index db.scm:566: conc db.scm:564: string-intersperse tests.scm:101: conc tests.scm:104: reverse tests.scm:104: append tests.scm:104: string-intersperse db.scm:796: conc db.scm:792: conc db.scm:802: debug:print-info common_records.scm:58: debug:debug-mode db.scm:803: sqlite3#for-each-row <-- #f2dcdc b340dd6ecc 2013-07-19 23:27:33 Code_Defect Open runs and links directories are not found if relative. causes crash #f2dcdc 83eb0489cd 2013-07-21 22:20:35 Code_Defect Open Create area wizard fails to create directory - should detect calculated dirs ================== Now you need an initial place to store your runs. These are called "disks" and you can add more at any time. To get going provide a writeable directory name. Enter your test runs directory: #{getenv MT_TEST_RUN_AREA}/../runs WARNING: you have specified a path #{getenv MT_TEST_RUN_AREA}/../runs that does not exist. Attempting to create it... Error: (create-directory) cannot create directory - File exists: "#{getenv MT_TEST_RUN_AREA}/.." Call history: genexample.scm:49: read-line genexample.scm:51: directory? genexample.scm:57: conc genexample.scm:57: file-exists? genexample.scm:68: print genexample.scm:81: display genexample.scm:82: read-line genexample.scm:84: string-split genexample.scm:84: g25 genexample.scm:84: g25 genexample.scm:87: print genexample.scm:89: display genexample.scm:90: read-line genexample.scm:91: directory? genexample.scm:93: print genexample.scm:94: create-directory <-- #f2dcdc cea380846b 2013-07-22 01:56:53 Code_Defect Open Test steps are not being cleared (regression?) when re-running without CLEAN #f2dcdc 89c0faa86d 2013-08-05 18:24:21 Code_Defect Open scheme not evaluating properly in megatest.config [evn-override] # BEQASCRIPTS #{scheme (print *toppath*)}/scripts #f2dcdc 258ecdb207 2013-08-17 06:34:40 Code_Defect Open Crash in gui going from Run Summary to Run control Launching /tmp/megatest/tests/fdktestqa/simplelinks/a/bigrun2/a/bigrun/890 Error: (hash-table-ref) hash-table does not contain key tests-info # Call history: dashboard.scm:1053: iup-base#attribute-set! dashboard.scm:1055: iup-base#attribute-set! dashboard.scm:1342: current-milliseconds dashboard.scm:1389: mutex-lock! dashboard.scm:1391: mutex-unlock! dashboard.scm:1381: mutex-lock! dashboard.scm:1385: mutex-unlock! dashboard.scm:1388: dashboard:run-update dashboard.scm:1308: file-modification-time dashboard.scm:1309: file-modification-time dashboard.scm:1310: current-seconds dashboard.scm:1311: dashboard:recalc dashboard.scm:1296: current-milliseconds dashboard.scm:1389: mutex-lock! dashboard.scm:1391: mutex-unlock! dashboard.scm:813: hash-table-ref #f2dcdc a9acdc76d4 2013-10-20 17:17:29 Incident Open monitor job breakage when overloaded cat NBFAKE-2013WW42.7_08\:42\:51 Warning (#): in thread: bad argument type - not a pointer: #f Call history: db.scm:277: k537 db.scm:277: g541 tests.scm:729: sqlite3#execute <-- Error: uncaught exception: # Call history: launch.scm:358: thread-join! <-- #f2dcdc 58552973a3 2013-11-02 17:08:19 Code_Defect Open Crash on db access to testdat.db #f2dcdc 03dfedbb23 2013-11-04 21:55:15 Code_Defect Open output from launcher script into mt_launch.log appears to not retain any newlines. #f2dcdc e1c04aa7aa 2013-11-05 16:57:08 Code_Defect Open Add test for handling corrupted or non-readable testdat.db #f2dcdc 71cf4cf1bc 2014-02-13 00:04:26 Code_Defect Open single step not working Warning (#<thread: ezstep run single step gencurrgds&g...Warning (#): in thread: unbound variable: run-id Call history: fs-transport.scm:42: debug:print-info common_records.scm:60: debug:debug-mode fs-transport.scm:43: db:process-queue-item db.scm:1849: g2314 db.scm:1850: g2318 db.scm:1851: g2322 db.scm:1852: g2326 db.scm:1853: alist-ref db.scm:1855: debug:print-info common_records.scm:60: debug:debug-mode db.scm:1865: debug:print-info common_records.scm:60: debug:debug-mode db.scm:1866: server:reply server.scm:108: debug:print-info common_records.scm:60: debug:debug-mode ezsteps.scm:167: cdb:roll-up-pass-fail-counts <-- shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory 4:58 PM #f2dcdc f1ce13e29b 2014-02-13 17:59:25 Code_Defect Open setlog crashes if run in wrong directory. chlr12706> megatest -step showresults -setlog results.html :state end :status 0 shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory Error: (current-directory) cannot retrieve current directory Call history: megatest.scm:316: args:get-arg margs.scm:17: hash-table-ref/default megatest.scm:333: args:get-arg margs.scm:17: hash-table-ref/default megatest.scm:348: hash-table-keys megatest.scm:346: lset-intersection megatest.scm:352: setup-for-run launch.scm:422: hash-table? launch.scm:425: args:get-arg margs.scm:17: hash-table-ref/default launch.scm:427: get-environment-variable launch.scm:424: find-and-read-config configf.scm:253: ##sys#get-keyword configf.scm:253: ##sys#get-keyword configf.scm:253: ##sys#get-keyword configf.scm:254: current-directory <-- chlr12706> cd $PWD chlr12706> megatest -step showresults -setlog results.html :state end :status 0 #f2dcdc 48c7b2d605 2014-02-24 05:36:48 Code_Defect Open crash in single test [0] http-transport:client-connect -> ("192.168.0.35" 8080 # #) pid:1559 INFO: (1) Running -runstep, first change to directory /mfs/matt/data/megatest-inmem-with-per-run/tests/fullrun/tmp/mt_links/ubuntu/nfs/none/w08.7.21.45_aa/singletest Error: (string-split) bad argument type - not a string: #f Call history: margs.scm:17: hash-table-ref/default megatest.scm:931: args:get-arg margs.scm:17: hash-table-ref/default megatest.scm:934: args:get-arg margs.scm:17: hash-table-ref/default megatest.scm:937: args:get-arg margs.scm:17: hash-table-ref/default megatest.scm:940: args:get-arg margs.scm:17: hash-table-ref/default megatest.scm:946: args:get-arg margs.scm:17: hash-table-ref/default megatest.scm:947: args:get-arg margs.scm:17: hash-table-ref/default megatest.scm:948: conc megatest.scm:952: get-environment-variable megatest.scm:952: string-split <-- ; tracing tasks:server-lock-slot #f2dcdc 2661ce8c0f 2014-02-26 23:56:02 Code_Defect Open uncaught exception on pass/fail roll up for items Obtained lock for megatest-rollup-pycell_xor.html ... stuff ... Error: uncaught exception: # #f2dcdc 1704fc58bb 2014-03-14 19:27:32 Feature_Request Open move run directories to trash then remove, clean up trash dir as can #f2dcdc bda469557b 2014-03-14 19:45:03 Feature_Request Open Add -clean-run-waitons Need to be able to automatically clean and rerun waitons. Could also separate where -clean is a switch that modifies -runtests to clean before running. #f2dcdc 0b723d1367 2014-03-17 03:49:10 Code_Defect Open (print (rmt:test-get-records-for-index-file 1 "test1")) => unspecified INFO: (1) Test exited in state=RUNNING, setting state/status based on exit code of #t and rollup-status of 127 Error: (for-each) bad argument type - not a list: # Call history: http-transport.scm:274: thread-terminate! http-transport.scm:275: debug:print-info common_records.scm:68: debug:debug-mode http-transport.scm:225: k682 http-transport.scm:225: g686 rmt.scm:59: db:string->obj db.scm:1749: regex#regexp db.scm:1748: regex#string-substitute db.scm:1747: base64#base64-decode db.scm:1746: with-input-from-string db.scm:1750: s11n#deserialize tests.scm:331: with-output-to-port tests.scm:334: conc <-- Obtained lock for megatest-rollup-sqlitespeed.html #f2dcdc ca67102876 2014-03-21 21:19:02 Feature_Request Open Create disk0 if not exists Optional - make configurable #f2dcdc 869d68d8ae 2014-04-17 18:06:10 Feature_Request Open Add run state and status to display on dashboard - esp STALE #f2dcdc 274d1dd1e7 2014-04-25 22:08:12 Feature_Request Open Expanded tests go to bottom of list I don't know how the order of tests is decided in the megatest dashboard, but it looks like now expanding a test brings it to the bottom. It didn't used to do that, and I find it a little confusing. If it was not specifically requested by another user, I would recommend leaving the order the same, and just expanding in place. #f2dcdc 3599e71a11 2014-05-09 22:47:29 Code_Defect Open Runs with failed tests showing as running Runs with failed tests showing as running when a test launches a new megatest instance. #f2dcdc 051fdf14a9 2014-05-11 15:11:58 Feature_Request Open clean-but-keep-logs #f2dcdc 173a785ec7 2014-05-11 15:14:13 Feature_Request Open Add %PRE%, %POST% and config entries to support log file handling (current is xterm only) #f2dcdc d89ce5c251 2014-06-27 23:33:12 Feature_Request Open Improvements request #f2dcdc e90f784630 2014-09-10 21:41:25 Feature_Request Open need reporting for catching long running jobs #f2dcdc 410ccd8f36 2014-09-24 22:52:41 Code_Defect Open crash in dashboard when no tests in tests dir. #f2dcdc 41bbd050d0 2015-04-30 16:56:12 Code_Defect Open server stuck issue - user can't work until cleanup-db is run #f2dcdc caa371c882 2015-05-27 11:33:22 Feature_Request Open runwait should only wait on the tests specified by its testpatt #f2dcdc 06156b8706 2015-06-13 22:36:17 Feature_Request Open -run should put out (optionally?) PASS/FAIL status of tests as they complete. Esp if in -runwait mode #f2dcdc b68e393028 2015-06-14 03:48:26 Code_Defect Open test2 is not detecting that it can end. #f2dcdc eddf860353 2015-06-24 20:16:22 Code_Defect Open Dashboard collapse option with first-rime user I was helping a new user get set up today and I think we hit a first-use behavior in the dashboard. The behavior we noticed is that tests with test items start as expanded, which is fine. I had him select the collapse button and the test in progress collapsed. But as each successive test was populated with test items, that new test was presented as expanded. It appears that if there is nothing to collapse in a test and there is no previous regression using the test, the collapse is not applied. #f2dcdc 49481ee087 2015-08-05 00:13:44 Code_Defect Open Ensure MT_TEST_NAME and MT_ITEM_PATH are visible in testconfig processing #f2dcdc 3c3a1b98a2 2015-08-20 14:23:45 Code_Defect Open Incorrect behavior of section/-var in -show-runconfig #f2dcdc 3c517427a3 2015-08-28 15:53:05 Code_Defect Open Tests with items rolling up to UNKNOWN state Tests with items are rolling up to UNKNOWN state rather than PASS/FAIL/WARN as applicable. This is happening with any number of items - I've tried 1 and 15. #f2dcdc a706ae9e4f 2015-08-28 15:57:14 Code_Defect Open Itemmap handling of % When using itemmap and mode itemwait, if the mapped item contains a '%' the itemwait is behaving like toplevel - it is not starting the item when its dependencies have been met. Example, TESTA (waiton TESTB, mode itemwait) ITEM1/fluff ITEM2/fluff TESTB repo/ITEM1 repo/ITEM2 If I use itemmap to convert ITEM1/fluff to %/ITEM1, TESTA/ITEM1/fluff does not start until all items in TESTB are complete. #f2dcdc 5c59aae6cc 2015-09-03 04:09:07 Feature_Request Open Need test mutex #f2dcdc de737b7f90 2015-09-10 17:45:12 Code_Defect Open megatest not honoring hard coded launcher #f2dcdc 51f2557c24 2015-09-23 20:37:26 Feature_Request Open Provide switch to emit run status of a run #f2dcdc 2a890be4f1 2015-09-24 00:06:34 Code_Defect Open Have observed that hooks fire twice #f2dcdc 4b072de97c 2015-10-05 23:19:39 Feature_Request Open provide means to specify desire to rerun a test/step N times on fail #f2dcdc 7abe077381 2015-12-01 18:31:01 Code_Defect Open dashboard crashes after 5-10 mins Its the central run dir. Pls let me know if you need more info. Warning (#): in thread: (finalize!) database is locked: # Call history: db.scm:3181: thread-sleep! db.scm:3182: db:delay-if-busy db.scm:3165: ##sys#get-keyword db.scm:3166: configf:lookup configf.scm:288: hash-table? configf.scm:289: hash-table-ref/default db.scm:3169: db:dbdat-get-path db.scm:3170: db:dbdat-get-db db.scm:3171: conc db.scm:3172: call-with-current-continuation db.scm:3172: with-exception-handler db.scm:3172: ##sys#call-with-values db.scm:3178: file-exists? db.scm:3172: k3877 db.scm:3172: g3881 db.scm:2039: sqlite3#for-each-row <-- Error: uncaught exception: # #f2dcdc 4e019d7072 2015-12-01 18:37:04 Feature_Request Open Enhancements requested for megatest 1. capability to select entire row on dashboard 2. Select all visible runs on the dashboard and delete option 3. xterm sourcing setup.csh or some pre-command executed when launched from TCP. #f2dcdc 3c7ceb7107 2016-01-29 17:30:19 Code_Defect Open Megatest dashboard hides some tests when number of tests is large When I expand a test with many subtests (200), if I use the scrollbar to scroll to the bottom, some tests still do not show up. This is intermittent. Sometimes if I scroll to the top and then back down, it then shows all tests; other times I cannot see the additional tests at all. #f2dcdc 4a26d0d54d 2016-02-05 16:57:54 Code_Defect Open Run data disappears from dashboard Run data intermittently disappears from the megatest dashboard. It can be recovered by closing the dashboard and reopening it, but "Refresh" does nothing. #f2dcdc 676ea51d07 2016-02-25 14:43:21 Feature_Request Open Support items in a waiton: test1/%foo #f2dcdc c4b292ae45 2016-05-07 19:14:49 Feature_Request Open Run step from test xterm - or ability to dump testconfig in a copy/paste friendly way #f2dcdc ab7da7f5f0 2016-05-07 19:22:07 Feature_Request Open access testconfig from gui for editing - tiresome to dig through filesystem #f2dcdc d5640a8cc1 2016-05-09 04:53:20 Code_Defect Open Finish launch:recover-test or refactor launch:execute to support running with no master mtest process #f2dcdc 8ea9b6e119 2016-05-12 04:45:47 Code_Defect Open deadlock and mutex failure #f2dcdc 3699db2b31 2016-06-21 23:07:42 Code_Defect Open test vacuously "PASS"es if missing ezsteps / setup section When a testconfig is missing a specification of things to run, eg in [ezsetps] section, the test passes. II would expect the test to fail since its specification is incomplete. IIn my case, the error was a typo in spelling "ezsteps", but the test passed whereas it should have failed since the underlying step was actually failing but was masked by this issue. #f2dcdc aa2b3ffab0 2016-06-22 20:57:12 Code_Defect Open dashboard crashes when clicking on "Run Details" tab Warning (#): in thread: call of non-procedure: #f Call history: dashboard.scm:1853: file-modification-time dashboard.scm:1853: file-modification-time dashboard.scm:1853: file-modification-time dashboard.scm:1853: file-modification-time dashboard.scm:1853: file-modification-time dashboard.scm:1846: k3511 dashboard.scm:1846: g3515 dashboard.scm:1858: file-exists? dashboard.scm:1859: file-modification-time dashboard.scm:1861: current-seconds dashboard.scm:1862: d:alldat-please-update dashboard.scm:1862: d:alldat-last-db-update dashboard.scm:1862: dashboard:recalc dashboard.scm:1863: d:alldat-curr-tab-num dashboard.scm:1871: d:alldat-curr-tab-num dashboard.scm:1889: dashboard:update-new-view-tab <-- Error: uncaught exception: # [6] Exit 70 dashboard -rows 25 #f2dcdc a926703df5 2016-08-01 21:03:05 Code_Defect Open ezstep exiting 6 transitions status to "FAIL". Expectation is a transition to status "SKIP" in my testconfig, I have a step which runs a script having the code "exit 6" In the test control panel, the Steps table correctly reflects Status=6 for the step. However, the overall status is "FAIL". I expected "SKIP". #f2dcdc 503763b420 2016-08-02 21:55:34 Code_Defect Open #{get foo} => fails ungracefully (needs message for missing param) #f2dcdc 09f81a879a 2016-08-04 23:29:54 Code_Defect Open Run Summary tab: Need to toggle run selection 3 times to see main pane update to new run When having multiple runs, when I launch a new run, I click on the new run name that appears on the left pane of the Run Summary tab. The pain pane still shows results for the run that was previously selected. If I reselect the prior run, then select the new run again, it updates properly. #f2dcdc 5a2549bc82 2016-08-26 19:59:37 Code_Defect Open Remove support for db and monitor directory control and improve lt support. #f2dcdc 0954ef0fbc 2016-09-02 19:51:52 Code_Defect Open megatest -list-targets error when no write access to MT_RUN_AREAHOME In megatest 1.6104 -list-targets attempts to create a symlink, so cannot check targets in another user's area. NEEDS A TEST? #f2dcdc 61054a5780 2016-09-13 15:24:04 Code_Defect Open dashboard sort by test name is not sorting by test name The sort +testname and -testname in dashboard are not sorting testname alphabetically - this should take the itempath into account if it exists. #f2dcdc 78b26c37f3 2017-01-13 18:52:17 Code_Defect Open Dashboard hangs when core count is 32 or 64 For example, plxcm7003. Hang occurs when switching to dashboard "run" or "run summary" tab #f2dcdc bc11edad89 2017-02-02 20:34:21 Code_Defect Open add -state and -status to -find-files #f2dcdc fe82fa6b60 2017-02-03 00:57:11 Code_Defect Open dashboard font missing underscores (_) or widget too small and cuts it off... In Run tab, when viewing test names on leftmost column, the test names look like they have blank spaces where one would expect underscores. This was not the case in 1.60/29 (marking severe - reported by two users) #f2dcdc 7cb1419578 2017-02-03 00:57:53 Code_Defect Open run tab - cannot resize columns. runnames and targets of long length cannot be distinguished In the run tab, the width of matrix columns for runs is fixed. If a field value for the target or the run name spans more than ~14 characters, it is cut off. This could be worked around in v1.60/29 by stretching the entire window horizontally by dragging the right corner out. Since the matrix was bound to the right side, the column widths would scale out. In current version (1.63/05) the matrix is not bound to the left edge, so stretching has no effect. Marking Severe; reported by end user. #f2dcdc 0331ed63d5 2017-02-28 21:54:18 Code_Defect Open megatest -v returns stack dump after showing help. Expect no stack dump. #f2dcdc 7b698cdf7b 2017-03-10 22:14:44 Code_Defect Open mtutil run crashes mtutil run -area ext-tests -target a/b/c/d Error: (null-list?) bad argument type - not a list: # Call history: configf.scm:50: hash-table-ref/default configf.scm:49: config:assoc-safe-add configf.scm:41: ##sys#get-keyword configf.scm:42: filter configf.scm:43: append configf.scm:48: hash-table-set! mtut.scm:843: configf:lookup configf.scm:416: hash-table? configf.scm:417: hash-table-ref/default mtut.scm:844: string-split mtut.scm:845: hash-table-copy mtut.scm:841: ##sys#call-with-values mtut.scm:852: command-line->pkt mtut.scm:392: current-seconds mtut.scm:397: current-user-name mtut.scm:410: filter <-- #f2dcdc 9f9c407694 2017-04-20 22:42:18 Code_Defect Open Please enable section creation to be effective without any variables in config files #f2dcdc 514327decf 2017-04-21 00:00:45 Code_Defect Open "misc" "default" and "%: appear in hierarchy widget of Run Control panel. THey shoulod not Since default and misc and % expand to non-sensical values and are not valid targets for runs, we do not expect to see these as selectable targets in run control panel. #f2dcdc 67b54f0b1c 2017-04-24 20:05:06 Feature_Request Open Finer grained job limit control I would like to propose a new [setup] option. In addition to max_concurrent_jobs, I would like to see something like max_waiting_jobs that would allow users to limit how many jobs are in the waiting or launched state. This would allow users to run as many jobs as possible, while still holding back the scheduling load of a batch system. #f2dcdc 944da29a23 2017-04-24 22:42:09 Code_Defect Open in run tab: state filter + status filter result in wrong result SETUP: 0) setup megatest build from tag v1.6407) 1) under megatest folder, create ext-tests. Poplulate subdir from megatest_qa fossil using commit d3e9c96dc307d7c941ea973d524adb4b267cbd56 (currently tip at time of writing) 2) Using megatest_qa testsuite, perform two runs, say named "run1" and "run2". run1 completes first. All tests in both runs should resolve to COMPLETED/PASS. 3) In run1, manually set tconfdisks/tom/n to RUNNING/PASS 5) in run1, manually set tconfdisks/fred/d to RUNNING/FAIL 6) in run1, manually set tconfdisks/bob/p to COMPLETED/FAIL 7) Make sure "HIDE" mode is selected for filters. 8) select state filter "RUNNING" 9) select status filter "FAIL" OBSERVE: two columns a. run2 having tconfdisks topeleven and items tom/n and fred/d items as "RUNNING" b. run1 having tconfdisks toplevel and items tom/n and fred/d as "RUNNING" c. run1 having tconfdisks item bob/p "FAIL". ISSUE: item a above is unexpected, since they are in fact COMPLETED/PASS in the database. item b and c are expected and correct. #f2dcdc c21083ffba 2017-05-30 22:21:03 Code_Defect Open client abort when server version does not match current behavior when client version does not match server version is for client to launch another server. net result is a darwinian struggle of multiple version variants of megatest server. #f2dcdc 12dfb44595 2017-05-30 22:23:17 Code_Defect Open add timestamp to items in server logs motivation - it is hard to debug when a server has gone wrong. this is a feature indented for analysts and developers. #f2dcdc 188f74ee68 2017-05-30 22:24:30 Feature_Request Open When run is removed, update hierarchy widget to delete run This caused confusion in a customer who did not understand the run indeed was deleted despite persisting in left pane hier browser. #f2dcdc fa12c1f200 2017-05-30 22:26:47 Feature_Request Open in run summary tab, ability to remove text from cells matrix widget Desire here is to have a broader view of the run. Not many columns and rows can fit on a single screen. If text were removed, then a cell could be only a few pixels tall and wide, allowing view of much more tests and items. Color is sufficient to indicate status. A toggle or cyclic pulldown could select "normal" vs "condensed" view. #f2dcdc 855c60b695 2017-05-30 22:29:22 Code_Defect Open freeze column width of matrix widget in run summary tab In an active run, the run summary matrix view induces headaches in yours truly. The column widths vary wildly from one second to the next. Eg: on second, the column fits width of "REMOTEHOSTSTART" and the next fits "PASS". multiply this delta by # of tests. #f2dcdc 69fe6d4e23 2017-06-07 21:42:33 Feature_Request Open Remove tests from tree in Runs tab when they are removed I have a test suite with tons of tests. When I did a series of "Clean Tests", I was expecting the list of runs on the left hand side to update to reflect the removed tests. It doesn't refresh until you kill the dashboard and restart it. #f2dcdc e27c06a338 2017-06-22 02:16:11 Code_Defect Open Crash on migration of target to new target pattern #f2dcdc 6cae932a9d 2017-06-25 05:11:14 Code_Defect Open make NUMTESTS=30 bigrun3 RUNNAME=stress_test003 in tests/fdktestqa/testqa never finishes. #f2dcdc 8c972842e9 2017-06-28 03:41:39 Code_Defect Open a test which results in "zero items" state/status will cause runner process to stop, even though tests which may run and are not dependent on this zero items test are ready to run. they are left dangling. #f2dcdc b99251aaf8 2017-07-12 18:15:08 Feature_Request Open megatest.csh / megatest.sh can set invalid variables that cause shell to error In runconfig files, common mistakes like creating a dividing line with ******** or accidentally wrapping a line, eg yvar /blah/blah -ot foo.4.ot to myvar /blah/blah -ot foo .4.0t Result in errors in csh and warnings in bash like ./megatest.sh: line 75: export: `.4.ot=': not a valid identifier When creating megatest.csh/.sh, lets filter and warn when a env var name is invalid, that is does not begin with an alphabetic #f2dcdc 311d767396 2017-07-17 15:01:42 Code_Defect Open Running list runs in tests directory causes crash #f2dcdc 029dec28fe 2017-08-03 23:33:51 Feature_Request Open potential hazard in contextmenu->rerun; request clean-run-execute in context menu I think a quick fix would be to add another button to context menu called CleanRunExecute! that does exactly what the test control panel button does and leave ReRun alone; if you dont trust one, or want more speed; choose your own fortune :) Observations follow: when i select test->rerun from right click menu in run tab of dashboard, the test doesnt seem to get cleaned before rerun i just noticed that in my crf build.. rerunning didnt clear steps... just ran them again... which looks weird since steps were out of order in test control panel with some #unspecified states (iup:menu-item (conc "Rerun " item-test-path) #:action (lambda (obj) (common:run-a-command (conc "megatest -set-state-status NOT_STARTED,n/a -run -target " target " -runname " runname " -testpatt " item-test-path " -preclean -clean-cache")))) No Title (iup:menu-item "Run" (iup:menu (iup:menu-item (conc "Rerun " testpatt) #:action (lambda (obj) ;; (print " run-id: " run-id " test-id: " test-id " target: " target " runname: " runname " test-name: " test-name " testpatt: " testpatt "item-path : " item-path) (common:run-a-command (conc "megatest -run -target " target " -runname " runname " -testpatt " testpatt " -preclean -clean-cache") ))) (iup:menu-item "Rerun Complete Run" #:action (lambda (obj) (common:run-a-command (conc "megatest -set-state-status NOT_STARTED,n/a -run -target " target " -runname " runname " -testpatt % " " -preclean -clean-cache")))) all have -preclean when I do clean-run-execute, it is explicitly cleared before retstarting... that is what guided my expectation it definitely doesn't kill them... watching the dasboard, I see a running item go from running->launched immediately... with c-r-e, I see it blanking out first (clean-run-execute (lambda (x) (let ((cmd (conc "megatest -remove-runs -target " keystring " -runname " runname " -testpatt " (conc testname "/" (if (equal? item-path "") "%" item-path)) ";megatest -target " keystring " -runname " runname " -run -preclean -testpatt " (conc testname "/" (if (equal? item-path "") "%" item-path)) " -clean-cache" ))) cre does a remove-runs first... that is the difference i guess I think there might be some hazard. It ishard to prove... but I can get a test that passes consistently to fail if i rerun while running . In this case, the running test is in the process of a fossil open the respawned one dies with current directory is not within an open checkout I #f2dcdc c70c843f96 2017-08-15 21:20:58 Feature_Request Open One-file megatest setup similar to a Makefile in capability mt.conf [fields] a b c [setup] ... [tasks] job1 waitons=job2,job3;\ logrules=(expect:required ...);\ runtimlim=1m 5s; \ envvars=A=BCD,C=DEF,;\ steps=runtest.sh #-OR- [waitons] job1 job2:im1 [itemmaps] im1 .*/a \1 etc. #f2dcdc be73f72774 2017-08-24 17:41:16 Code_Defect Open lt link in MT_RUN_AREA_HOME needs to be recreated if linktree changes If linktree is changed in megatest.config, lt link in MT_RUN_AREA_HOME should be updated to match. #f2dcdc 2bf349ad96 2017-09-06 17:42:11 Code_Defect Open selecting edit testconfig from the context menu in run tab launches vi even though my EDITOR env var is "emacs" #f2dcdc 73204f444b 2017-10-17 11:39:42 Feature_Request Open Need option to auto exit a runner if exact same target/runname/testpatt (or superset) started #f2dcdc 24c8995e89 2017-11-29 23:50:39 Code_Defect Open Run control panel has undefined targets for version 1.65/04 All the target definitions have "runs" in the run tree browser. #f2dcdc 4a2ba010ba 2017-12-06 21:42:25 Feature_Request Open On querying homehost load use exponential backoff if load is high #f2dcdc 8b11992f91 2017-12-06 21:43:21 Code_Defect Open Need protection against bad hosts in lowestload list. Exponential backoff if there is a timeout or very high load. #f2dcdc 5f5ca1c39a 2017-12-08 06:25:05 Build_Problem Open unable to build megatest from vanilla ubuntu system -- some packages fetched during installall.sh are not present on kiatoa site #f2dcdc d9c4f6e711 2018-05-22 23:30:56 Code_Defect Open broken wildcard target value binding when value uses [system #{getenv ... Example problem: from runconfigs.config: [full] ... [/.*/] THINGS_LIST [system #{getenv THING_ORACLE_EXE} listem] OKSTUFF hello % megatest -showrunconfig THINGS_LIST expected_value OKSTUFF hello % megatest -showrunconfig -target full OKSTUFF hello **THINGS_LIST does not appear** #f2dcdc 13680e7907 2018-08-10 17:39:44 Code_Defect Open Goodtests not starting due to no propagation of PREQ_FAIL #f2dcdc e2b83cfeaf 2018-09-04 19:54:06 Code_Defect Open toplevel test does not fire when dependent on a failed itemized test and not using -run-wait ../ #f2dcdc 5386ffc2b2 2018-11-07 23:11:21 Feature_Request Open Need mtutil to do "run" with no testpatt spec If no testpatt spec, then use the TESTPATT from runconfigs. #f2dcdc fabb7405f5 2018-11-08 17:35:40 Code_Defect Open Variables starting with ! leaking into environment when variable is set for a totally open regex When a user puts a !FOO variable into an include in the runconfig that matches everything, like [/.*/], it appears to match default as well, and when that happens, those variables leak into the environment. #f2dcdc 1144efd3b6 2019-07-15 04:48:15 Feature_Request Open Directive and support for localifying vars in a config #f2dcdc ca896bc63d 2019-07-30 00:12:46 Feature_Request Open request new state/status #f2dcdc 276ad8c9cf 2019-09-05 22:48:36 Code_Defect Open Need to add ext-tests/mt_vars to the testsuite and fix the MT_TARGET not set issue. #f2dcdc 07469a3053 2019-09-12 22:49:20 Code_Defect Open subrun requires runwait -> give an error is run-wait specified #f2dcdc c4f251d133 2019-10-22 16:12:29 Code_Defect Open megatest -list-runs % -fields run:target,runname+tests:testname,item_path incorrectly lists steps #f2dcdc 5d87c510e8 2019-10-22 17:48:37 Code_Defect Open Area to area run inheritance #f2dcdc 677155e2a6 2020-04-24 22:02:10 Code_Defect Open With no runconfig Megatest can crash - on no file need to return empty hash, not #f #f2dcdc d68d38f75f 2020-04-25 15:40:38 Feature_Request Open add -stale, use with run to propagate back STALE so that tests will run again #f2dcdc f7e3277660 2020-05-29 13:14:05 Feature_Request Open REFDB: add init as an option and add import from sqlite3 #f2dcdc cb70b19fa1 2020-05-29 14:17:59 Feature_Request Open REFDB: add view as option - read a refdb for view only. #f2dcdc 7ca381b8ad 2020-05-29 14:18:57 Feature_Request Open REFDB: for set use append mode rather than read, add, write #f2dcdc 57fd79cf36 2020-06-04 16:43:41 Code_Defect Open fix run removal if targets have been removed If the target has been removed from the runconfig, the run does not show up in dashboard, and cannot be removed using the dashboard. #f2dcdc adf67e869a 2020-09-23 19:14:09 Code_Defect Open Crash #f2dcdc e7596943a5 2020-09-23 22:04:47 Code_Defect Open spurious readonly database crash. #f2dcdc 04e5164c68 2020-09-23 22:40:45 Code_Defect Open Crash #f2dcdc 96a9e539f8 2020-10-07 04:53:24 Code_Defect Open Crash #f2dcdc 86662a0273 2020-10-08 20:21:59 Code_Defect Open Crash #f2dcdc 9644eb4ca3 2020-10-30 14:30:32 Code_Defect Open crash in server.log #f2dcdc ee2c833eaf 2021-02-19 19:16:39 Code_Defect Open crash on start dashboard #f2dcdc 64562cb2cd 2021-02-23 23:49:55 Feature_Request Open On-startup checks for viable setup. #f2dcdc fbfdb653fd 2021-03-19 18:03:44 Code_Defect Open v1.65/83 with only one disk0 entry does horrible things ... #f2dcdc bf11e5ecea 2022-03-21 17:01:01 Feature_Request Open split the pkts in pkts.db into columns For easier querying and editing, split the content of pkts in the db into columns. #f2dcdc 6fdbc6a042 2022-03-21 17:35:37 Feature_Request Open need a packet cleanup function to prevent pkts.db from getting too big We need a way to prevent the pkts.db from getting too large. We need a way to remove all pkts that are no longer needed. #f2dcdc 99f0cb983c 2022-03-22 13:34:04 Feature_Request Open mode exclusive needs better user notification #f2dcdc 7a2177ff76 2022-03-25 16:54:25 Feature_Request Open Remove keep feature enhancement: -keep-last-good For a given target: If the youngest run consists of state=completed, status=, then all older runs in same target can be eliminated. #f2dcdc d3dc03e1c6 2022-03-25 17:14:40 Feature_Request Open megatest list-test-time update I am using the megatest -list-test-time command to extract the run time for some regressions under my workarea. When using the command the run times are provided but it does not seem to filter them out based on run name which makes the data difficult to understand. In the log file additional test-patts are listed although only one has been specified. Lastly, the documentation for this command states to use "-run-patt" and -test-patt" as options whic may be outdated. Command: megatest -list-test-time -target $TARGET -runname my-run -testpatt test1 #f2dcdc 5442740dfa 2022-03-25 17:19:20 Feature_Request Open megatest skip: requesting to be able to have multiple rules in a [skip] section or'd together I think the current behavior in [skip] is to AND together multiple rules. Requesting capability to OR together multiple rules. I am OK if the default remains AND but have a megatest.config setting or a different section header like [skipany] to implement. #f2dcdc 354935ae2b 2022-03-25 18:47:51 Code_Defect Open runtime limit assigned in runconfigs not being honored Seeing instances of batch jobs running megatest far exceeding the runtime limit that is in the config. A couple of possible causes: megatest kills the job but the batch system doesn't detect that it's dead megatest does not honor the limit the limit could be getting overriden somewhere else that we're not seeing #f2dcdc 65bfe02932 2022-03-25 20:55:39 Code_Defect Open Megatest not executing some parts of a test For tests that have a step which is 2 commands joined with an && it's causing problems. The workaround for now is to split the command into two steps. #f2dcdc 23bdcb1c3e 2022-04-20 22:28:27 Code_Defect Open remove-runs removed non-matching extra runs #f2dcdc a2e4022d93 2022-04-26 16:51:32 Feature_Request Open Enhancement request to configure a test as pre-requisite of all test in a flow This is a request for an enhancement in megatest to be able to configure a test as a pre-requisite for all the tests within a flow without the need to manually configure the test dependency. #f2dcdc 181ee0a581 2022-04-28 17:23:49 Feature_Request Open megatest -list-runs should allow listing tests without steps if specifying fields If i run: megatest -list-runs % -target % -fields tests:test_name,item_path I expect to see only run and test info, not step info When specifying -fields, step info should only be displayed when steps or +steps is specified. The desired behavior is consistent with specifying run info in the fields - test info is suppressed then unless tests are specified # shows runs, tests, and steps -fields runs:runname+tests:test_name+steps:step_name # shows runs and tests, no steps: -fields runs:runname+tests:test_name+item_path For script automation around -list-runs, it would also be nice if the test output looked like: Test: cv item_path: c/lpp Rather than: Test: cv(c/lpp) If it needs to be handled with a different dumpmode, that is ok too #f2dcdc 5e66375eda 2022-05-02 19:50:09 Feature_Request Open skip status behavior skip is intended to short circuit a test expected behavior: * test: stop execution, mark test COMPLETED,SKIP, this is a "passing" status * step: stop test execution, mark test COMPLTED,SKIP, this is a "passing" status need to document use cases, eg how to "skip/short-circuit" a step but keep going #f2dcdc 000b351628 2022-05-02 20:01:04 Feature_Request Open requesting new "stop" behavior Requesting a new status: COMPLETED/STOP The expected behavior is that the test stops, and subsequent items do not run. This is a "soft pass" status color-wise (should not be red/purple) Suggestion to implement via a [stop] testconfig section similar to [skip] #f2dcdc fe055ad196 2022-05-04 20:49:30 Feature_Request Open Implement #{getenv VAR alt_result} #f2dcdc 590e7b7930 2022-05-04 20:50:29 Code_Defect Open when step crashes logpro rules do not get applied. #f2dcdc 9446af8b68 2022-05-11 15:12:36 Feature_Request Open mv and rm test run dirs performance #f2dcdc 577322c526 2022-05-11 15:17:38 Feature_Request Open regenerate logpro and scripts if testconfig changes Need to have a way to regenerate the test logpro and scripts if the testconfig changes in MT_RUN_AREA_HOME/tests/ area. Not sure if it is necessary to be able to regenerate the files individually for each step. #f2dcdc c58f15f80d 2022-05-20 21:26:06 Code_Defect Open The message "looking for zombie" is too frequent and or needs context. #f2dcdc 73e3742297 2022-05-21 01:35:20 Code_Defect Open current defaults for free disk space are based on some crazy large corporate needs. #f2dcdc 4e12dfbc1b 2022-05-22 02:06:10 Code_Defect Open Switch default to no local db access #f2dcdc 9243f53c6d 2022-06-02 02:24:24 Code_Defect Open access to remote-conndat needs to be protected against not-yet-set remote. #f2dcdc a5ae5422ab 2022-06-16 15:45:29 Code_Defect Open trailing space in config section prevents recognition and is hard to detect #f2dcdc 5b2cffd17c 2022-06-22 21:35:36 Code_Defect Open Variables in one target with no value produce "=" in another target's test's env When a variable is defined in a target earlier in the runconfig file, with no value, like MYENVVAR it appears with the name "MYENVVAR=" in the environment of a test run from a later target in the runconfig. It does not appear when doing megatest -show-runconfig -target The workaround is to put "" for the value of the var. #f2dcdc 0ff1cbfa82 2022-06-24 21:52:19 Feature_Request Open "dead mans switch" enhancement Need a megatest "dead man's switch" feature. When applied, this would mark all pending operations in the database as "canceled" or something along those lines. Any runners calling home would not see any action left to take and exit. Potentially should take target/runname/testpatt as arguments to filter which items to "cancel" #f2dcdc 2c61853004 2022-07-07 20:55:41 Code_Defect Open incorrect behavior with rerun-all #f2dcdc 940b26d681 2022-07-07 22:21:00 Feature_Request Open override logpro output dir in testconfig Looking to be able to override the logpro output directory for ezsteps. Assuming the general functionality is: step | logpro step.logpro step.html The ask is to be able to add a testconfig (maybe also runconfig?) setting to redirect the html to a different location: step | logpro step.logpro /step.html #f2dcdc cb58710522 2022-07-24 01:17:50 Feature_Request Open Add filter to dashboard command line #f2dcdc 21ff1c38b2 2022-08-02 15:19:11 Code_Defect Open Starting from clean (no cache in /tmp) the main.db is not always copied #f2dcdc 9004c56d24 2022-10-13 21:52:34 Code_Defect Open runconfigs should be read only once From Robert: There is a real issue with the time it takes megatest to process runconfigs. Every #{shell ...} command is processed in every target from every included file even if the target is not the one being run, or a simple -list-targets is given on the command line. To make matters worse, when running a regression, not only does the main megatest "mtest" call processes runconfigs, so does any server started by the regression run. And if it takes a LONG time to process the runconfigs, the regression run will spend a lot of time waiting for the server to start. Every single call to megatest, whether its to start the regression, or start a server, or whatever, is subject to the time it takes to process the runconfig files. The parsing of runconfigs should happen 1 time per session, and the evaluation of value should only happen 1 time and when its needed/used (there should not be a global evaluation of all parameters across all targets unless there is some kind of dump operation occurring). While I appreciate that there was some original reason/intent for re-parsing the runconfigs the entire process is leading to a substantial performance issues in regression runs. Even if there is a need to re-evaluate a variable value, there should be a means of indicating so, like: VAR #{eval ...} Something needs to be done to improve performance of reading the runconfigs. #f2dcdc cd7bd96b45 2022-12-08 00:06:59 Code_Defect Open variable leakage between tests when using pre-launch-env-vars and include #f2dcdc 2223f4fd04 2022-12-13 18:22:58 Code_Defect Open line ending in \ not working in scripts and # breaks scripts in testconfig #f2dcdc 71bc1fd416 2023-01-17 19:41:06 Code_Defect Open Shell and scheme are evaluated in comments. This is not proving useful. Please disable. #f2dcdc ddecbfdf1d 2023-02-22 19:50:57 Code_Defect Open Test recovery doesn't seem to be working right #f2dcdc ae9b88b9ac 2023-12-15 16:52:47 Code_Defect Open crash in remove-runs on highly loaded machine, need exception handler? #f2dcdc d18665ecd9 2024-01-23 21:46:51 Code_Defect Open testconfig evaluation leaking to other tests. #f2dcdc 9506c87ca5 2024-02-04 17:00:36 Code_Defect Open dashboard crashes in chained-waiton. #f2dcdc aeba047102 2011-07-15 20:03:40 Feature_Request Open Store vmstat info (extract signature for kernel being forced to swap out) Store vmstat data to see when the host is running low on memory vmstat 5 10 procs -----------memory---------- ---swap-- -----io---- -system-- -----cpu------ r b swpd free buff cache si so bi bo in cs us sy id wa st 2 0 107184 13671396 1967828 73863552 0 1 0 33 0 0 18 1 80 1 0 2 1 107184 13671476 1967828 73863552 0 0 0 22 4346 5371 11 1 88 1 0 1 0 107184 13669244 1967828 73863552 0 0 0 45 4991 5907 13 1 86 1 0 2 0 107184 13670680 1967828 73863552 0 0 0 198 4341 5371 11 1 87 1 0 1 0 107184 13669668 1967828 73863552 0 0 0 32 4910 5509 11 1 87 1 0 1 0 107184 13670744 1967828 73863552 0 0 0 68 4513 6001 11 1 87 1 0 4 0 107184 13669328 1967828 73863552 0 0 0 33 5002 5906 11 1 87 1 0 2 0 107184 13670512 1967828 73863552 0 0 0 40 4495 5793 11 1 87 1 0 2 0 107184 13671172 1967828 73863552 0 0 0 53 4665 5230 11 1 87 2 0 2 1 107184 13671704 1967828 73863552 0 0 0 15 4337 4935 11 1 88 1 0 The "so" column should be zero most of the time. #f2dcdc 47f1907c8d 2012-10-11 15:47:59 Feature_Request Open Ability to clean and re-run from a specified test megatest -run-from test1/item1 cleans all downstream tests and starts run from specified test #f2dcdc 5a591737fb 2013-03-07 00:07:35 Code_Defect Open getenv USER broken, -waive, -rerun o Error loading runconfigs that uses #{getenv USER} definition Failing to locate targets defined in unloaded runconfigs o –show-config => dumps (STDOUT) contents of megatest.config post-processing o –show-runconfig [–target ] => same but for runconfigs o –load