Megatest

Check-in [2b9e485b7f]
Login
Overview
Comment:prereq proc is supposed to return test records, not test names.
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | v1.64
Files: files | file ages | folders
SHA1: 2b9e485b7f099a339275b136045e9acfbad2cc01
User & Date: matt on 2017-03-27 12:41:02
Other Links: branch diff | manifest | tags
Context
2017-03-27
23:59
protected config file call to delete-file with exception handler. Fixed logic on connecting using CMDINFO. Fixed -list-servers and -kill-servers. Turned exception handler back on in portlogger. Removed the addition of a little noise from the server timeout handling in rmt.scm check-in: 7b318f91bd user: matt tags: v1.64
12:41
prereq proc is supposed to return test records, not test names. check-in: 2b9e485b7f user: matt tags: v1.64
12:05
Improvements to exclusive mode (but it still isn't working quite right). Factored clean-cache into a reusable function and called it also in -rerun*. Reduced number of server files to analyze. This may reduce the probability of a runaway server situation. check-in: 0f50a21b44 user: matt tags: v1.64
Changes

Modified db.scm from [2e4fa26f31] to [05478e9bc9].

3881
3882
3883
3884
3885
3886
3887


3888





3889
3890
3891
3892
3893
3894
3895
3881
3882
3883
3884
3885
3886
3887
3888
3889

3890
3891
3892
3893
3894
3895
3896
3897
3898
3899
3900
3901







+
+
-
+
+
+
+
+







						  #f           ;; not-in
						  #f           ;; sort by
						  #f           ;; sort order
						  'shortlist   ;; query type
						  0            ;; last update, beginning of time ....
						  #f           ;; mode
						  )))
	;;(map (lambda (testdat)
	;;	(if (equal? (db:test-get-item-path testdat) "")
	 (map db:test-get-testname running-tests))
	;;	    (db:test-get-testname testdat)
	;;	    (conc (db:test-get-testname testdat)
	;;		  "/"
	;;		  (db:test-get-item-path testdat))))
	 running-tests) ;; calling functions want the entire data
       '())
   (if (or (not waitons)
	   (null? waitons))
       '()
       (let* ((unmet-pre-reqs '())
	      (result         '()))
	 (for-each 
3940
3941
3942
3943
3944
3945
3946
3947

3948
3949
3950
3951
3952
3953
3954
3946
3947
3948
3949
3950
3951
3952

3953
3954
3955
3956
3957
3958
3959
3960







-
+







			      (member 'toplevel mode))              ;; toplevel does not block on FAIL
			  (and is-ok (member 'itemmatch mode))) ;; itemmatch blocks on not ok
		     (set! item-waiton-met #t)))))
	       tests)
	      ;; both requirements, parent and item-waiton must be met to NOT add item to
	      ;; prereq's not met list
	      (if (not (or parent-waiton-met item-waiton-met))
		  (set! result (append (if (null? tests) (list waitontest-name) tests) result)))
		  (set! result (append (if (null? tests) (list waitontest-name) tests) result))) ;; appends the string if the full record is not available
	      ;; if the test is not found then clearly the waiton is not met...
	      ;; (if (not ever-seen)(set! result (cons waitontest-name result)))))
	      (if (not ever-seen)
		  (set! result (append (if (null? tests)(list waitontest-name) tests) result)))))
	  waitons)
	 (delete-duplicates result)))))

Modified runs.scm from [3b13aedef2] to [ad868adae8].

668
669
670
671
672
673
674
675

676
677
678
679
680
681
682
668
669
670
671
672
673
674

675
676
677
678
679
680
681
682







-
+







		  (cons hed reruns)))
	  #f)) ;; #f flags do not loop

     ((and (not (null? fails))(member 'toplevel testmode))
      (if (or (not (null? reg))(not (null? tal)))
	   (list (car newtal)(append (cdr newtal) reg) '() reruns)
	  #f)) 
     ((null? runnables) #f) ;; if we get here and non-completed is null the it's all over.
     ((null? runnables) #f) ;; if we get here and non-completed is null then it is all over.
     (else
      (debug:print 0 *default-log-port* "WARNING: FAILS or incomplete tests maybe preventing completion of this run. Watch for issues with test " hed ", continuing for now")
      ;; (list (runs:queue-next-hed tal reg reglen regfull)
      ;;   	(runs:queue-next-tal tal reg reglen regfull)
      ;;   	(runs:queue-next-reg tal reg reglen regfull)
      ;;   	reruns)
      (list (car newtal)(cdr newtal) reg reruns)))))
843
844
845
846
847
848
849
850


851
852
853
854
855
856
857
843
844
845
846
847
848
849

850
851
852
853
854
855
856
857
858







-
+
+







      (list (car newtal)(cdr newtal) reg reruns))
     
     ;; This is the final stage, everything is in place so launch the test
     ;;
     ((and have-resources
	   (or (null? prereqs-not-met)
	       (and (member 'toplevel testmode) ;;  'toplevel)
		    (null? non-completed))))
		    (null? non-completed)
		    (not (member 'exclusive testmode)))))
      ;; (hash-table-delete! *max-tries-hash* (db:test-make-full-name test-name item-path))
      ;; we are going to reset all the counters for test retries by setting a new hash table
      ;; this means they will increment only when nothing can be run
      (set! *max-tries-hash* (make-hash-table))
      ;; well, first lets see if cpu load throttling is enabled. If so wait around until the
      ;; average cpu load is under the threshold before continuing
      (if (configf:lookup *configdat* "jobtools" "maxload") ;; only gate if maxload is specified
1403
1404
1405
1406
1407
1408
1409
1410


1411
1412
1413
1414
1415
1416
1417
1404
1405
1406
1407
1408
1409
1410

1411
1412
1413
1414
1415
1416
1417
1418
1419







-
+
+








(define (runs:calc-runnable prereqs-not-met)
  (filter 
   (lambda (t)
     (or (not (vector? t))
	 (and (equal? "NOT_STARTED" (db:test-get-state t))
	      (member (db:test-get-status t)
			      '("n/a" "KEEP_TRYING")))))
		      '("n/a" "KEEP_TRYING")))
	 (and (equal? "RUNNING" (db:test-get-state t))))) ;; account for a test that is running
   prereqs-not-met))

(define (runs:pretty-string lst)
  (map (lambda (t)
	 (if (not (vector? t))
	     (conc t)
	     (conc (db:test-get-testname t) ":" (db:test-get-state t) "/" (db:test-get-status t))))