Megatest

Check-in [7e63ac6bde]
Login
Overview
Comment:Removing some of the mutex enabling around db:with-db seems to improve performance
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | v1.80-revolution
Files: files | file ages | folders
SHA1: 7e63ac6bdee59776c776a23f60722742072a12ec
User & Date: matt on 2023-11-21 20:13:13
Other Links: branch diff | manifest | tags
Context
2023-11-21
20:50
removed another mutex but this seems to be worse than the last commit check-in: a1c90b4301 user: matt tags: v1.80-revolution
20:13
Removing some of the mutex enabling around db:with-db seems to improve performance check-in: 7e63ac6bde user: matt tags: v1.80-revolution
15:42
Lots of little changes check-in: fa88f0abd7 user: mrwellan tags: v1.80-revolution
Changes

Modified TODO from [e1f7aa3311] to [6085d32bc2].

20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

23WW46 - v1.80 branch
. Use file semaphore to kill tests, eliminate db load of the KILLREQ query
. Merge this change to revolution branch
23WW45 - the revolution branch
. Add "fast" db start option (no handshaking over NFS)
. Add server-ro to server types (just "server" is fine for read/write).
. Create pause-server and resume-server calls
. Create rsync or cp sync to MTRAH function
. Change rmt:send-receive to divert calls to read-only server when possible
. Change start server to call main.db server for 1..N.db servers, block until server is read for use.

23WW21
. Dashboard needs its own cache db in /tmp

23WW07
. Remove use of *dbstruct-dbs*








|


|







20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

23WW46 - v1.80 branch
. Use file semaphore to kill tests, eliminate db load of the KILLREQ query
. Merge this change to revolution branch
23WW45 - the revolution branch
. Add "fast" db start option (no handshaking over NFS)
. Add server-ro to server types (just "server" is fine for read/write).
. [DONE] Create pause-server and resume-server calls
. Create rsync or cp sync to MTRAH function
. Change rmt:send-receive to divert calls to read-only server when possible
. [DONE] Change start server to call main.db server for 1..N.db servers, block until server is read for use.

23WW21
. Dashboard needs its own cache db in /tmp

23WW07
. Remove use of *dbstruct-dbs*

Modified dbmod.scm from [39c3e50bc5] to [0a134beec4].

86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102

;;======================================================================
;; The cachedb one-db file per server method goes in here
;;======================================================================

;; NOTE: the r/w is now w/r, #t=db modified by query, #f=db NOT modified by query
(define (dbmod:with-db dbstruct run-id w/r proc params)
  (let* ((use-mutex (or (and w/r ;; use the mutex on queries that modify the db and for sync to disk
			     (> *api-process-request-count* 5)) ;; when writes are happening throttle more
			(> *api-process-request-count* 50)))
	 (dbdat     (dbmod:open-db dbstruct run-id (dbfile:db-init-proc)))
	 (dbh       (dbr:dbdat-dbh dbdat)) ;; this will be the cachedb handle
	 (dbfile    (dbr:dbdat-dbfile dbdat)))
    ;; if nfs mode do a sync if delta > 2
    (let* ((last-update (dbr:dbstruct-last-update dbstruct))
	   (sync-proc   (dbr:dbstruct-sync-proc dbstruct))
	   (curr-secs   (current-seconds)))







|
|
|







86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102

;;======================================================================
;; The cachedb one-db file per server method goes in here
;;======================================================================

;; NOTE: the r/w is now w/r, #t=db modified by query, #f=db NOT modified by query
(define (dbmod:with-db dbstruct run-id w/r proc params)
  (let* ((use-mutex w/r) ;; (or (and w/r ;; use the mutex on queries that modify the db and for sync to disk
			     ;; (> *api-process-request-count* 5)) ;; when writes are happening throttle more
			;; (> *api-process-request-count* 50)))
	 (dbdat     (dbmod:open-db dbstruct run-id (dbfile:db-init-proc)))
	 (dbh       (dbr:dbdat-dbh dbdat)) ;; this will be the cachedb handle
	 (dbfile    (dbr:dbdat-dbfile dbdat)))
    ;; if nfs mode do a sync if delta > 2
    (let* ((last-update (dbr:dbstruct-last-update dbstruct))
	   (sync-proc   (dbr:dbstruct-sync-proc dbstruct))
	   (curr-secs   (current-seconds)))