Check-in [3de9db9a0f]
Not logged in
Overview
SHA1 Hash:3de9db9a0f12fae5bc0e14215c3375f138d6df51
Date: 2012-04-23 02:36:33
User: matt
Comment:Added simple helper for creating new tests and megatest areas
Timelines: family | ancestors | descendants | both | extend-test-get-path
Diffs: root of this branch
Downloads: Tarball | ZIP archive
Other Links: files | file ages | manifest
Tags And Properties
Changes

Modified Makefile from [65935ed0bad2bd4f] to [baa18dd99f8c51c1].

1 1 2 PREFIX=. 2 PREFIX=. 3 CSCOPTS= 3 CSCOPTS= 4 INSTALL=install 4 INSTALL=install 5 SRCFILES = common.scm items.scm launch.scm \ 5 SRCFILES = common.scm items.scm launch.scm \ 6 ods.scm runconfig.scm server.scm configf.scm \ 6 ods.scm runconfig.scm server.scm configf.scm \ 7 db.scm keys.scm margs.scm megatest-version.scm \ 7 db.scm keys.scm margs.scm megatest-version.scm \ 8 process.scm runs.scm tasks.scm tests.scm | 8 process.scm runs.scm tasks.scm tests.scm genexample.scm 9 9 10 GUISRCF = dashboard.scm dashboard-tests.scm dashboard-guimonitor.scm dashboard- 10 GUISRCF = dashboard.scm dashboard-tests.scm dashboard-guimonitor.scm dashboard- 11 11 12 OFILES = $(SRCFILES:%.scm=%.o) 12 OFILES = $(SRCFILES:%.scm=%.o) 13 GOFILES = $(GUISRCF:%.scm=%.o) 13 GOFILES = $(GUISRCF:%.scm=%.o) 14 14 15 HELPERS=$(addprefix $(PREFIX)/bin/,mt_laststep mt_runstep mt_ezstep) 15 HELPERS=$(addprefix $(PREFIX)/bin/,mt_laststep mt_runstep mt_ezstep)

Modified genexample.scm from [7449a3004cf98335] to [35a078568515ba67].

6 ;; 6 ;; 7 ;; This program is distributed WITHOUT ANY WARRANTY; without even the 7 ;; This program is distributed WITHOUT ANY WARRANTY; without even the 8 ;; implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR 8 ;; implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR 9 ;; PURPOSE. 9 ;; PURPOSE. 10 ;;====================================================================== 10 ;;====================================================================== 11 11 12 (declare (unit genexample)) 12 (declare (unit genexample)) > 13 (use posix) 13 14 > 15 (define genexample:example-logpro > 16 #<<EOF > 17 ;; You should have at least one expect:required. This ensures that your process > 18 (expect:required in "LogFileBody" > 0 "Put description here" #/put pattern here/ 14 19 > 20 ;; You may need ignores to suppress false error or warning hits from the later e > 21 ;; NOTE: Order is important here! > 22 (expect:ignore in "LogFileBody" < 99 "Ignore the word error in comments" #/^\ > 23 (expect:warning in "LogFileBody" = 0 "Any warning" #/warn/) > 24 (expect:error in "LogFileBody" = 0 "Any error" (list #/ERROR/ #/error/)) ;; > 25 EOF > 26 ) > 27 > 28 (define genexample:example-script > 29 #<<EOF > 30 #!/usr/bin/env bash > 31 > 32 # Run your step here > 33 EOF > 34 ) > 35 15 (define (genexample:mk-megatest.config path) | 36 (define (genexample:mk-megatest.config) 16 (let ((keystr #f) 37 (let ((keystr #f) 17 (keys #f) 38 (keys #f) 18 (lntree #f) 39 (lntree #f) > 40 (path #f) 19 (firstd #f)) 41 (firstd #f)) 20 (debug:print 0 "Note: don't worry too much about typos in this process, you | 42 (print "Note: don't worry too much about typos in this process, you will be 21 the generated files before starting your first runs") 43 the generated files before starting your first runs") 22 44 > 45 ;; create the needed area > 46 (print "==================\nWhere can I create your Megatest regresssion/con > 47 "tests will not necessarily be run in this area, disk space needs are > 48 (display "Enter your megatest directory: ") > 49 (set! path (read-line)) > 50 > 51 (if (not (directory? path)) > 52 (begin > 53 (print "The path " path " does not exist or is not a directory. Attemp > 54 (create-directory path #t))) > 55 23 ;; first prompt user for fields 56 ;; first prompt user for fields 24 ;; 57 ;; 25 (debug:print 0 "First you must specify fields or keys for your megatest area | 58 (print "==================\nNext you must specify fields or keys for your me 26 be used to organise your runs. One field should probably be \"RELEASE\". 59 be used to organise your runs. One field should probably be \"RELEASE\". 27 Other examples of useful fields might be \"PLATFORM\", \"TARGET_OS\" or if you a 60 Other examples of useful fields might be \"PLATFORM\", \"TARGET_OS\" or if you a 28 semiconductor business perhaps things like \"TECHNOLOGY_NODE\", \"DESIGN_KIT\" o 61 semiconductor business perhaps things like \"TECHNOLOGY_NODE\", \"DESIGN_KIT\" o 29 62 30 The all caps is a convention because the variables you choose will be available 63 The all caps is a convention because the variables you choose will be available 31 as environment variables. You can edit these values later but it is generally a 64 as environment variables. You can edit these values later but it is generally a 32 settle on them and get them right early on. Your runs will be stored in director 65 settle on them and get them right early on. Your runs will be stored in director 33 your keys. Example, if you have keys OSFAMILY/VARIANT/OSVER/RELEASE you may get 66 your keys. Example, if you have keys OSFAMILY/VARIANT/OSVER/RELEASE you may get 34 in a directory like this: linux/ubuntu/11.04/rev_1.2/build 67 in a directory like this: linux/ubuntu/11.04/rev_1.2/build 35 68 36 Please enter your keys now, separated by spaces or slashes. Only alpha-numeric c 69 Please enter your keys now, separated by spaces or slashes. Only alpha-numeric c 37 Upper case recommended.") 70 Upper case recommended.") > 71 (display "Enter keys separated by spaces or slashes: ") 38 (set! keystr (read-line)) 72 (set! keystr (read-line)) 39 (set! keys (apply append 73 (set! keys (apply append 40 (map string-split (string-split keystr "/")))) 74 (map string-split (string-split keystr "/")))) 41 75 42 ;; Now get the link tree location and a first disk 76 ;; Now get the link tree location and a first disk 43 (debug:print 0 "Now you need an initial place to store your runs. These are | 77 (print "==================\nNow you need an initial place to store your runs 44 can add more at any time. To get going provide a writeable directory name. ") 78 can add more at any time. To get going provide a writeable directory name. ") > 79 (display "Enter your test runs directory: ") 45 (set! firstd (read-line)) 80 (set! firstd (read-line)) > 81 (if (not (directory? firstd)) > 82 (begin > 83 (print "WARNING: you have specified a path " firstd " that does not ex > 84 (create-directory firstd #t))) 46 85 47 (debug:print 0 "Megatest uses a tree of symlinks to provide a uniform struct | 86 (print "==================\nMegatest uses a tree of symlinks to provide a un 48 you run over time. Please provide a path where we can create this link tree.") 87 you run over time. Please provide a path where we can create this link tree.") > 88 (display "Enter link tree directory: ") 49 (set! lntree (read-line)) 89 (set! lntree (read-line)) > 90 (if (not (directory? lntree)) > 91 (begin > 92 (print "WARNING: you have specified a path " lntree "that does not exi > 93 (create-directory lntree #t))) > 94 > 95 (with-output-to-file (conc path "/megatest.config") > 96 (lambda () > 97 (print "[fields]") > 98 (map (lambda (k)(print k " TEXT")) keys) > 99 (print "") > 100 (print "[setup]") > 101 (print "# Adjust max_concurrent_jobs to limit how much you load your mac > 102 (print "max_concurrent_jobs 50\n") > 103 (print "# This is your link path, you can move it but it is generally be > 104 (print "linktree " lntree) > 105 (print "\n# Job tools are more advanced ways to control how your jobs ar > 106 (print "[jobtools]\nuseshell yes\nlauncher nbfind\n") > 107 (print "# You can override environment variables for all your tests here > 108 (print "[env-override]\nEXAMPLE_VAR example value\n") > 109 (print "# As you run more tests you may need to add additional disks, th > 110 (print "[disks]\ndisk0 " firstd))) > 111 > 112 (print "==================\nI'm now creating a runconfigs.config file for yo > 113 You can use this file to set variables for your tests based on the \"target\" (t > 114 of keys).") > 115 (with-output-to-file (conc path "/runconfigs.config") > 116 (lambda () > 117 (print "# The variables in the default category will be seen in all runs > 118 > 119 (print "# Your variables here are grouped by targets [" (string-interspe > 120 (let ((example-target (string-intersperse (map (lambda (k)(conc k "_val" > 121 (print "[" example-target "]") > 122 (print "ANOTHERVAR only defined if target is " example-target)))) > 123 > 124 ;; Now create a test and logpro file > 125 (print "You now have the basic common files for your megatest setup. Next ru > 126 "to create a test.\n\nThanks for using Megatest. You can edit your co > 127 "in the " path " directory"))) > 128 > 129 > 130 ;;====================================================================== > 131 ;; create skeleton files for a test > 132 ;;====================================================================== > 133 > 134 (define (genexample:mk-megatest-test testname) > 135 ;; Gather needed data > 136 (let ((waiton #f) > 137 (priority #f) > 138 (description #f) > 139 (steps '()) > 140 (scripts '()) > 141 (items '()) > 142 (rel-path #f)) > 143 (print "We are going to generate a skeleton set of files for your test " tes > 144 "too much about typos, you can edit the files when you are done") > 145 > 146 (print "==================\nPlease describe this test. The description will > 147 (display "Enter one line description for this test: ") > 148 (set! description (read-line)) > 149 > 150 (print "==================\nDoes this test, " testname ", require any other > 151 (display (conc "Enter space delimited list of tests which " testname " must > 152 (set! waiton (read-line)) > 153 > 154 (print "==================\nDo you wish to prioritize the running of this te > 155 (print "enter a number greater than zero here") > 156 (display "Enter a priority of 0 (default) or higher: ") > 157 (set! priority (read-line)) > 158 > 159 ;; Get the steps > 160 (print "==================\nNow to enter the one or more steps that make up > 161 (print "Hint; use .sh extension on the script names and we'll create placeho > 162 > 163 (let ((stepname #f) > 164 (scriptname #f)) > 165 (let loop ((done #f)) > 166 (display "Enter the name for this step (blank to stop): ") > 167 (set! stepname (read-line)) > 168 (if (not (equal? stepname "")) > 169 (begin > 170 (display "Enter the script or progam to run: ") > 171 (set! scriptname (read-line)) > 172 (set! steps (append steps (list (list stepname scriptname)))))) > 173 (if (not (equal? stepname "")) > 174 (loop #f)))) > 175 > 176 ;; Get the items > 177 (print "==================\nNext we need to get the variables and values you > 178 (let ((varname #f) > 179 (values #f)) > 180 (let loop ((done #f)) > 181 (display "Enter the variable name: ") > 182 (set! varname (read-line)) > 183 (if (not (equal? varname "")) > 184 (begin > 185 (display (conc "Enter the space separated list of values for " var > 186 (set! values (read-line)) > 187 (set! items (append items (list (conc varname " " values)))))) > 188 (if (not (equal? varname "")) > 189 (loop #f)))) 50 190 > 191 ;; Now create the test > 192 (cond > 193 ((file-exists? "megatest.config") (set! rel-path "./")) > 194 ((file-exists? "../megatest.config") (set! rel-path "../")) > 195 ((file-exists? "../../megatest.config") (set! rel-path "../../")) > 196 ((file-exists? "../../../megatest.config")(set! rel-path "../../../"))) ;; > 197 (if (not rel-path) > 198 (begin > 199 (print "ERROR: You must run this command in a megatest area under wher > 200 (exit 1)) > 201 (let ((testdir (conc rel-path "tests/" testname))) > 202 (create-directory testdir #t) 51 (with-output-to-file | 203 (with-output-to-file (conc testdir "/testconfig") 52 (lambda () | 204 (lambda () > 205 (print "# Add additional steps here. Format is \"stepname script\" > 206 (map (lambda (stp)(print (string-intersperse stp " "))) steps) 53 (print "[fields]") | 207 (print "") 54 (map (lambda (k)(print k " TEXT")) keys) < > 208 (print "# Test requirements are specified here\n[requirements]") > 209 (print "waiton " waiton) > 210 (print "priority " priority "\n") > 211 (print "# Iteration for your tests are controlled by the items sec > 212 (map print items) 55 (print "") | 213 (print "") > 214 (print "# test_meta is a section for storing additional data on yo > 215 (print "author " (get-environment-variable "USER")) > 216 (print "owner " (get-environment-variable "USER")) > 217 (print "description " description) > 218 (print "tags tagone,tagtwo") 56 (print "[setup]") | 219 (print "reviewed never"))) 57 (print "# Adjust max_concurrent_jobs to limit how much you load your m < 58 (print "max_concurrent_jobs 50\n") < 59 (print "# This is your link path, you can move it but it is generally < 60 (print "linktree " lntree) < > 220 ;; Now create shell scripts (if extension is .sh) and logpro files > 221 (for-each (lambda (stp) > 222 (let ((stepname (car stp)) > 223 (script (cadr stp))) > 224 (with-output-to-file (conc testdir "/" stepname ".logpro > 225 (lambda () > 226 (print genexample:example-logpro))) > 227 (with-output-to-file (conc testdir "/" script ".sh") > 228 (lambda () > 229 (print genexample:example-script))) > 230 (system (conc "chmod ug+r " (conc testdir "/" script ".s 61 (print ""))))) | 231 steps)))))) > 232 ;;

Modified megatest.scm from [a58a41b84eb4adbf] to [4e1718db7636140d].

17 (declare (uses common)) 17 (declare (uses common)) 18 (declare (uses megatest-version)) 18 (declare (uses megatest-version)) 19 (declare (uses margs)) 19 (declare (uses margs)) 20 (declare (uses runs)) 20 (declare (uses runs)) 21 (declare (uses launch)) 21 (declare (uses launch)) 22 (declare (uses server)) 22 (declare (uses server)) 23 (declare (uses tests)) 23 (declare (uses tests)) > 24 (declare (uses genexample)) 24 25 25 (define *db* #f) ;; this is only for the repl, do not use in general!!!! 26 (define *db* #f) ;; this is only for the repl, do not use in general!!!! 26 27 27 (include "common_records.scm") 28 (include "common_records.scm") 28 (include "key_records.scm") 29 (include "key_records.scm") 29 (include "db_records.scm") 30 (include "db_records.scm") 30 31 ................................................................................................................................................................................ 97 98 98 Spreadsheet generation 99 Spreadsheet generation 99 -extract-ods fname.ods : extract an open document spreadsheet from the databa 100 -extract-ods fname.ods : extract an open document spreadsheet from the databa 100 -pathmod path : insert path, i.e. path/runame/itempath/logfile.html 101 -pathmod path : insert path, i.e. path/runame/itempath/logfile.html 101 will clear the field if no rundir/testname/itempath/ 102 will clear the field if no rundir/testname/itempath/ 102 if it contains forward slashes the path will be conv 103 if it contains forward slashes the path will be conv 103 to windows style 104 to windows style > 105 Getting started > 106 -gen-megatest-area : create a skeleton megatest area. You will be prompte > 107 -gen-megatest-test : create a skeleton megatest test. You will be prompte 104 108 105 Examples 109 Examples 106 110 107 # Get test path, the '.' is required, could use '*' or a specific path/file 111 # Get test path, the '.' is required, could use '*' or a specific path/file 108 megatest -test-path . -target ubuntu/n%/no% :runname w49% -testpatt test_mt% 112 megatest -test-path . -target ubuntu/n%/no% :runname w49% -testpatt test_mt% 109 113 110 Called as " (string-intersperse (argv) " "))) 114 Called as " (string-intersperse (argv) " "))) ................................................................................................................................................................................ 149 "-server" 153 "-server" 150 "-extract-ods" 154 "-extract-ods" 151 "-pathmod" 155 "-pathmod" 152 "-env2file" 156 "-env2file" 153 "-setvars" 157 "-setvars" 154 "-set-state-status" 158 "-set-state-status" 155 "-debug" ;; for *verbosity* > 2 159 "-debug" ;; for *verbosity* > 2 > 160 "-gen-megatest-test" 156 "-override-timeout" 161 "-override-timeout" 157 ) 162 ) 158 (list "-h" 163 (list "-h" 159 "-force" 164 "-force" 160 "-xterm" 165 "-xterm" 161 "-showkeys" 166 "-showkeys" 162 "-test-status" 167 "-test-status" ................................................................................................................................................................................ 175 180 176 "-runall" ;; run all tests 181 "-runall" ;; run all tests 177 "-remove-runs" 182 "-remove-runs" 178 "-usequeue" 183 "-usequeue" 179 "-rebuild-db" 184 "-rebuild-db" 180 "-rollup" 185 "-rollup" 181 "-update-meta" 186 "-update-meta" > 187 "-gen-megatest-area" 182 188 183 "-v" ;; verbose 2, more than normal (normal is 1) 189 "-v" ;; verbose 2, more than normal (normal is 1) 184 "-q" ;; quiet 0, errors/warnings only 190 "-q" ;; quiet 0, errors/warnings only 185 ) 191 ) 186 args:arg-hash 192 args:arg-hash 187 0)) 193 0)) 188 194 ................................................................................................................................................................................ 751 (set! *didsomething* #t))) 757 (set! *didsomething* #t))) 752 758 753 (if (args:get-arg "-gui") 759 (if (args:get-arg "-gui") 754 (begin 760 (begin 755 (debug:print 0 "Look at the dashboard for now") 761 (debug:print 0 "Look at the dashboard for now") 756 ;; (megatest-gui) 762 ;; (megatest-gui) 757 (set! *didsomething* #t))) 763 (set! *didsomething* #t))) > 764 > 765 (if (args:get-arg "-gen-megatest-area") > 766 (begin > 767 (genexample:mk-megatest.config) > 768 (set! *didsomething* #t))) > 769 > 770 (if (args:get-arg "-gen-megatest-test") > 771 (let ((testname (args:get-arg "-gen-megatest-test"))) > 772 (genexample:mk-megatest-test testname) > 773 (set! *didsomething* #t))) 758 774 759 ;;====================================================================== 775 ;;====================================================================== 760 ;; Update the database schema on request 776 ;; Update the database schema on request 761 ;;====================================================================== 777 ;;====================================================================== 762 778 763 (if (args:get-arg "-rebuild-db") 779 (if (args:get-arg "-rebuild-db") 764 (begin 780 (begin

Modified tests.scm from [e315223b6123a087] to [ce4320f87b4210c8].

74 (not (null? tal))) 74 (not (null? tal))) 75 (loop (car tal)(cdr tal)) 75 (loop (car tal)(cdr tal)) 76 (if (null? results) #f 76 (if (null? results) #f 77 (car results)))))))))) 77 (car results)))))))))) 78 78 79 ;; get the previous records for when these tests were run where all keys match b 79 ;; get the previous records for when these tests were run where all keys match b 80 ;; NB// Merge this with test:get-previous-test-run-records? This one looks for a 80 ;; NB// Merge this with test:get-previous-test-run-records? This one looks for a 81 ;; can use wildcards. | 81 ;; can use wildcards. Also can likely be factored in with get test paths? 82 (define (test:get-matching-previous-test-run-records db run-id test-name item-pa 82 (define (test:get-matching-previous-test-run-records db run-id test-name item-pa 83 (let* ((keys (db:get-keys db)) 83 (let* ((keys (db:get-keys db)) 84 (selstr (string-intersperse (map (lambda (x)(vector-ref x 0)) keys) ", 84 (selstr (string-intersperse (map (lambda (x)(vector-ref x 0)) keys) ", 85 (qrystr (string-intersperse (map (lambda (x)(conc (vector-ref x 0) "=? 85 (qrystr (string-intersperse (map (lambda (x)(conc (vector-ref x 0) "=? 86 (keyvals #f) 86 (keyvals #f) 87 (tests-hash (make-hash-table))) 87 (tests-hash (make-hash-table))) 88 ;; first look up the key values from the run selected by run-id 88 ;; first look up the key values from the run selected by run-id

Added tests/simplerun/megatest.config version [12e33187cae74b02]

> 1 [fields] > 2 SYSTEM TEXT > 3 RELEASE TEXT > 4 > 5 [setup] > 6 # Adjust max_concurrent_jobs to limit how much you load your machines > 7 max_concurrent_jobs 50 > 8 > 9 # This is your link path, you can move it but it is generally better to keep it > 10 linktree ../simplelinks > 11 > 12 # Job tools are more advanced ways to control how your jobs are launched > 13 [jobtools] > 14 useshell yes > 15 launcher nbfind > 16 > 17 # You can override environment variables for all your tests here > 18 [env-override] > 19 EXAMPLE_VAR example value > 20 > 21 # As you run more tests you may need to add additional disks, the names are arbi > 22 [disks] > 23 disk0 ../simpleruns

Added tests/simplerun/runconfigs.config version [0cc0a45438b81030]

> 1 [default] > 2 ALLTESTS see this variable > 3 # Your variables here are grouped by targets [SYSTEM/RELEASE] > 4 [SYSTEM_val/RELEASE_val] > 5 ANOTHERVAR only defined if target is SYSTEM_val/RELEASE_val

Added tests/simplerun/tests/test1/testconfig version [9ddcfab86d548c4a]

> 1 # Add additional steps here. Format is "stepname script" > 2 [ezsteps] > 3 step1 step1.sh > 4 step2 step2.sh > 5 > 6 # Test requirements are specified here > 7 [requirements] > 8 waiton setup > 9 priority 0 > 10 > 11 # Iteration for your tests are controlled by the items section > 12 [items] > 13 PARTOFDAY morning noon afternoon evening night > 14 > 15 # test_meta is a section for storing additional data on your test > 16 [test_meta] > 17 author matt > 18 owner matt > 19 description An example test > 20 tags tagone,tagtwo > 21 reviewed never

Added tests/simplerun/tests/test2/step1.logpro version [22f12ee837d77ff8]

> 1 ;; You should have at least one expect:required. This ensures that your process > 2 (expect:required in "LogFileBody" > 0 "Put description here" #/put pattern here/ > 3 > 4 ;; You may need ignores to suppress false error or warning hits from the later e > 5 ;; NOTE: Order is important here! > 6 (expect:ignore in "LogFileBody" < 99 "Ignore the word error in comments" #/^\ > 7 (expect:warning in "LogFileBody" = 0 "Any warning" #/warn/) > 8 (expect:error in "LogFileBody" = 0 "Any error" (list #/ERROR/ #/error/)) ;;

Added tests/simplerun/tests/test2/step1.sh.sh version [67f9a133dc133b12]

> 1 #!/usr/bin/env bash > 2 > 3 # Run your step here

Added tests/simplerun/tests/test2/step2.logpro version [22f12ee837d77ff8]

> 1 ;; You should have at least one expect:required. This ensures that your process > 2 (expect:required in "LogFileBody" > 0 "Put description here" #/put pattern here/ > 3 > 4 ;; You may need ignores to suppress false error or warning hits from the later e > 5 ;; NOTE: Order is important here! > 6 (expect:ignore in "LogFileBody" < 99 "Ignore the word error in comments" #/^\ > 7 (expect:warning in "LogFileBody" = 0 "Any warning" #/warn/) > 8 (expect:error in "LogFileBody" = 0 "Any error" (list #/ERROR/ #/error/)) ;;

Added tests/simplerun/tests/test2/step2.sh.sh version [67f9a133dc133b12]

> 1 #!/usr/bin/env bash > 2 > 3 # Run your step here

Added tests/simplerun/tests/test2/testconfig version [704f9b3ec459865e]

> 1 # Add additional steps here. Format is "stepname script" > 2 [ezsteps] > 3 step1 step1.sh > 4 step2 step2.sh > 5 > 6 # Test requirements are specified here > 7 [requirements] > 8 waiton test1 > 9 priority 0 > 10 > 11 # Iteration for your tests are controlled by the items section > 12 [items] > 13 LANDTYPE desert plains forest jungle beach > 14 > 15 # test_meta is a section for storing additional data on your test > 16 [test_meta] > 17 author matt > 18 owner matt > 19 description Yet another example test > 20 tags tagone,tagtwo > 21 reviewed never