Index: common.scm ================================================================== --- common.scm +++ common.scm @@ -360,11 +360,13 @@ (let ((res (assoc key lst))) (if res (cadr res)(if (null? default) #f (car default))))) (define (common:get-testsuite-name) (or (configf:lookup *configdat* "setup" "testsuite" ) - (pathname-file *toppath*))) + (if *toppath* + (pathname-file *toppath*) + (pathname-file (current-directory))))) ;;====================================================================== ;; E X I T H A N D L I N G ;;====================================================================== Index: docs/manual/Makefile ================================================================== --- docs/manual/Makefile +++ docs/manual/Makefile @@ -13,11 +13,11 @@ # asciidoc -b html5 -a icons -a iconsdir=$(DISPATH)/images/icons -a toc2 design_spec.txt # all : server.ps megatest_manual.html client.ps complex-itemmap.png -megatest_manual.html : megatest_manual.txt getting_started.txt writing_tests.txt reference.txt ../plan.txt howto.txt *png +megatest_manual.html : megatest_manual.txt getting_started.txt writing_tests.txt reference.txt ../plan.txt howto.txt installation.txt *png asciidoc -b html5 -a icons -a iconsdir=$(DISPATH)/images/icons -a toc2 megatest_manual.txt # dos2unix megatest_manual.html megatest.pdf : megatest_manual.txt getting_started.txt writing_tests.txt reference.txt ../plan.txt howto.txt *png a2x -a toc -f pdf megatest_manual.txt Index: docs/manual/getting_started.txt ================================================================== --- docs/manual/getting_started.txt +++ docs/manual/getting_started.txt @@ -1,99 +1,99 @@ Getting Started -=============== +--------------- [partintro] -.Getting started with Megatest: creating a testsuite/flow and your first test with installation instructions following. +.Getting started with Megatest -- -How to create your first testsuite and add a test. +Creating a testsuite or flow and your first test or task. -- -After installing Megatest you can easily create a flow or testsuite and add some tests. +After installing Megatest you can create a flow or testsuite and add some +tests using the helpers. Here is a quickstart sequence to get you up and +running your first automated testsuite. Creating a Megatest Area ------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~ + +Choose Target Keys +^^^^^^^^^^^^^^^^^^ + +First choose your "target" keys. These are used to organise your runs in a +way that is meaningful to your project. If you are unsure about what to use +for keys just use a single generic key such as "RUNTYPE". These keys will be +used to hand values to your tests via environment variables so ensure they +are unique. Prefixing them with something such as PROJKEYS_ is a good +strategy. + +Examples of keys: + +.Example keys +[width="60%",options="header"] +|============================================== +| Option | Description +| RELEASE/ITERATION | This example is used by Megatest for its internal QA. +| ARCH/OS/RELEASE | For a software project targeting multiple platforms +| UCTRLR/NODETYPE | Microcontroller project with different controllers +running same software +|============================================== + +Create Area Config Files +^^^^^^^^^^^^^^^^^^^^^^^^ + +You will need to choose locations for your runs (the data generated every +time you run the testsuite) and link tree. For getting started answer the +prompts with "runs" and "links". We use the Unix editor "vi" in the examples +below but you can use any plain text editor. .Using the helper to create a Megatest area ------------------ megatest -create-megatest-area + +# optional: verify that the settings are ok +vi megatest.config +vi runconfigs.config ------------------ Creating a Test ---------------- +~~~~~~~~~~~~~~~ + +Choose the test name for your first test and run the helper. You can edit +the files after the initial creation. You will need to enter names and +scripts for the steps to be run and then edit the +tests//testconfig file and modify the logpro rules to properly +process the log output from your steps. For your first test just hit enter +for the "waiton", "priority" and iteration variable prompts. + +Hint: for geting started make your logpro rules very liberal. expect:error +patterns should match nothing and comment out expect:required rules. .Using the helper to create a Megatest test --------------- -megatest -create-test testname +megatest -create-test myfirsttest + +# then edit the generated config +vi tests/myfirsttest/testconfig +--------------- + +Running your test +~~~~~~~~~~~~~~~~~ + +First choose a target and runname. If you have a two-place target such as +RELEASE/ITERATION a target would look like v1.0/aff3 where v1.0 is the +RELEASE and aff3 is the ITERATION. For a run name just use something like +run1. + +.Running all tests (testpatt of "%" matches all tests) +--------------- +megatest -run -target v1.0/aff3 -runname run1 -testpatt % -log run1.log --------------- -Installation ------------- - -Dependencies -~~~~~~~~~~~~ - -Chicken scheme and a number of "eggs" are required for building -Megatest. See the script installall.sh in the utils directory of the -source distribution for an automated way to install everything -needed for building Megatest on Linux. - -footnote:[An example footnote.] -indexterm:[Example index entry] - -// -// -// And now for something completely different: ((monkeys)), lions and -// tigers (Bengal and Siberian) using the alternative syntax index -// entries. -// (((Big cats,Lions))) -// (((Big cats,Tigers,Bengal Tiger))) -// (((Big cats,Tigers,Siberian Tiger))) -// Note that multi-entry terms generate separate index entries. -// -// Here are a couple of image examples: an image:images/smallnew.png[] -// example inline image followed by an example block image: -// -// .Tiger block image -// image::images/tiger.png[Tiger image] -// -// Followed by an example table: -// -// .An example table -// [width="60%",options="header"] -// |============================================== -// | Option | Description -// | -a 'USER GROUP' | Add 'USER' to 'GROUP'. -// | -R 'GROUP' | Disables access to 'GROUP'. -// |============================================== -// -// .An example example -// =============================================== -// Lorum ipum... -// =============================================== -// -// [[X1]] -// Sub-section with Anchor -// ~~~~~~~~~~~~~~~~~~~~~~~ -// Sub-section at level 2. -// -// Chapter Sub-section -// ^^^^^^^^^^^^^^^^^^^ -// Sub-section at level 3. -// -// Chapter Sub-section -// +++++++++++++++++++ -// Sub-section at level 4. -// -// This is the maximum sub-section depth supported by the distributed -// AsciiDoc configuration. -// footnote:[A second example footnote.] -// -// -// The Second Chapter -// ------------------ -// An example link to anchor at start of the <>. -// indexterm:[Second example index entry] -// -// An example link to a bibliography entry <>. -// -// +Viewing the results +~~~~~~~~~~~~~~~~~~~ + +Start the dashboard and browse your run in the "Runs" tab. + +.Starting dashboard +---------------- +dashboard -rows 24 +---------------- Index: docs/manual/howto.txt ================================================================== --- docs/manual/howto.txt +++ docs/manual/howto.txt @@ -1,14 +1,14 @@ How To Do Things -================ +---------------- Process Runs ------------- +~~~~~~~~~~~~ Remove Runs -~~~~~~~~~~~ +^^^^^^^^^^^ From the dashboard click on the button (PASS/FAIL...) for one of the tests. From the test control panel that comes up push the clean test button. The command field will be prefilled with a template command for removing that test. You can edit the command, for example change the argument to -testpatt to "%" to remove all tests. @@ -21,30 +21,30 @@ ---------------- megatest -remove-runs -target %/%/% -runname % -testpatt % -v ---------------- Archive Runs -~~~~~~~~~~~~ +^^^^^^^^^^^^ Megatest supports using the bup backup tool (https://bup.github.io/) to archive your tests for efficient storage and retrieval. Archived data can be rapidly retrieved if needed. The metadata for the run (PASS/FAIL status, run durations, time stamps etc.) are all preserved in the megatest database. For setup information see the Archiving topic in the reference section of this manual. To Archive -^^^^^^^^^^ +++++++++++ Hint: use the test control panel to create a template command by pushing the "Archive Tests" button. .Archive a full run ---------------- megatest -target ubuntu/nfs/none -runname ww28.1a -archive save-remove -testpatt % ---------------- To Restore -^^^^^^^^^^ +++++++++++ .Retrieve a single test ---------------- megatest -target ubuntu/nfs/none -runname ww28.1a -archive restore -testpatt diskperf/% ---------------- @@ -78,17 +78,17 @@ # match. flexi-launcher yes ------------------------ Tricks -====== +------ This section is a compendium of a various useful tricks for debugging, configuring and generally getting the most out of Megatest. Limiting your running jobs --------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~ The following example will limit a test in the jobgroup "group1" to no more than 10 tests simultaneously. In your testconfig: @@ -102,13 +102,10 @@ --------------- [jobgroups] group1 10 custdes 4 --------------- - - - Debugging Tricks ---------------- Examining The Environment ADDED docs/manual/installation.txt Index: docs/manual/installation.txt ================================================================== --- /dev/null +++ docs/manual/installation.txt @@ -0,0 +1,10 @@ +Installation +------------ + +Dependencies +~~~~~~~~~~~~ + +Chicken scheme and a number of "eggs" are required for building +Megatest. See the script installall.sh in the utils directory of the +source distribution for an automated way to install everything +needed for building Megatest on Linux. Index: docs/manual/megatest_manual.html ================================================================== --- docs/manual/megatest_manual.html +++ docs/manual/megatest_manual.html @@ -1,10 +1,10 @@ - + The Megatest Users Manual