summaryrefslogtreecommitdiff
path: root/build-aux/test-driver.scm
AgeCommit message (Collapse)AuthorFilesLines
2026-03-21build-aux: Revert to :global-test-result: in test-driver.scm.Maxim Cournoyer1-2/+2
This follows clarification with upstream on the valid name to use: <https://lists.gnu.org/archive/html/automake/2026-03/msg00005.html>. The change was made in the recent commit ef4ba3191f01. * build-aux/test-driver.scm (test-runner-gnu): Revert global test result key to ':global-test-result:'. Change-Id: If0eca05b8b57f1fd5d848e82bc789c43640d3caf
2026-03-20build: test-driver.scm: Utilize test-runner-group-path.Tomas Volf1-8/+12
Test groups were not used in any meaningful way. The group path was not printed and it was not used in test selection mechanism. I think groups are useful, and it is nice to be able to, for example, run tests from a single group. This commit does two things. First, it changes the test reporting to include the value returned from test-runner-group-path, so you will know not only the test name, but the test group(s) as well. And second, it changes the test selection (and exclusion) process to match against the "full" test name, so group path + test name. Hence (test-begin "failing tests") (test-equal "this should fail" 1 2) (test-end) will, depending on the output location, produce following text. .trs: :test-result: FAIL failing tests: this should fail [0.000s] :test-global-result: FAIL :recheck: yes :copy-in-global-log: yes .log: test-name: failing tests: this should fail location: test.scm:140 source: + (test-equal "this should fail" 1 2) expected-value: 1 actual-value: 2 result: FAIL stdout: FAIL: test.scm - failing tests: this should fail [0.000s] * build-aux/test-driver.scm (current-test-full-name): New procedure. (test-runner-gnu): Use current-test-full-name instead of test-runner-test-name. (test-match-name*): Match against current-test-full-name. Use compose. (test-match-name*/negated): Rewrite in terms of test-match-name*. Change-Id: I3fb9a2a721165204f020b79e019533f799b790e4 Signed-off-by: Maxim Cournoyer <maxim@guixotic.coop> Modified-by: Maxim Cournoyer <maxim@guixotic.coop>
2026-03-20build: test-driver.scm: Refine the global test result.Tomas Volf1-4/+15
The :test-global-result: .trs metadata contained just either FAIL, SKIP or PASS with a comment that further refinements are required for XPASS. The description of :test-global-result: is in the manual is as follows: This is used to declare the "global result" of the script. Currently, the value of this field is needed only to be reported (more or less verbatim) in the generated global log file ‘$(TEST_SUITE_LOG)’, so it’s quite free-form. For example, a test script which runs 10 test cases, 6 of which pass and 4 of which are skipped, could reasonably have a ‘PASS/SKIP’ value for this field, while a test script which runs 19 successful tests and one failed test could have an ‘ALMOST PASSED’ value. As we can see, the examples as `PASS/SKIP' and `ALMOST PASSED', so there is no need to stick to strict model. Hence this commit changes the resulting value to be comma-separated list of PASS, FAIL, XPASS, XFAIL and SKIP. The respective elements are present only when the count of tests with such a result is positive. In practice, that should usually produce lines such as :test-global-result: PASS,FAIL or :test-global-result: PASS * build-aux/test-driver.scm (test-runner-gnu)[finalize]: Refine the output of :test-global-result:. Change-Id: I7178ac9703e1749adf6de2445f7ed0591983cef2 Signed-off-by: Maxim Cournoyer <maxim@guixotic.coop>
2026-03-19build: test-driver.scm: Output singleton metadata just once.Tomas Volf1-27/+33
Current implementation printed metadata supposed to be present just once per .trs file on the end of each test group. According to the automake's manual that is undefined behavior. This commit fixes it by printing that metadata just once, after all tests did run. Since there is no built-in hook that could be used for that (test-runner-on-final runs on *each* outermost test-end), I introduced new `finalize' procedure that need to be called by the user. Possibly not the most elegant solution, but since we are the only user, it works fine and produces actually valid .trs file. That also means there is no longer any use for test-runner-on-test-end!. * build-aux/test-driver.scm (test-runner-gnu): Define new procedure `finalize' and return it together with the runner. Do not call test-runner-on-group-end!. (main): Call the `finalize' after all tests are done. Signed-off-by: Maxim Cournoyer <maxim@guixotic.coop>
2026-03-19build-aux: Add workaround for broken --select in test-driver.scm.Maxim Cournoyer1-4/+13
* build-aux/test-driver.scm (test-result-kind*): New procedure. (test-runner-gnu): Use it. Change-Id: I9cfd3289b05c77a7ab3c3f9e449b178fe31499fc
2026-03-19build-aux: Fix typo in doc of test-driver.scm.Maxim Cournoyer1-1/+1
* build-aux/test-driver.scm (test-runner-gnu): Fix typo in doc. Change-Id: I07c39ec79ce13b7c0fd5971ba90bb5106f92318e
2026-03-19build-aux: Streamline test-driver.scm a bit.Maxim Cournoyer1-15/+1
Guile 3.0.11 ships with a new SRFI 64. * build-aux/test-driver.scm (%test-match-all): Delete procedure. (main): Replace %test-match-all with test-match-all. Change-Id: Ibcd7d98194694c1de679491be4f5a603a79f6e29
2026-01-25build: test-driver.scm: Fix test-reporting with guile 3.0.11.Dariqq1-2/+2
With srfi-64 from guile 3.0.11 the test name is no longer in the test-result-alist. * build-aux/test-driver.scm (test-runner-gnu): Use test-runner-test-name to access test-name. Change-Id: I53d0fdd8db0d1af5e636e3f2a68280cd2bddfe4c Signed-off-by: Ludovic Courtès <ludo@gnu.org> Merges: #5836 Signed-off-by: Rutherther <rutherther@ditigal.xyz>
2025-09-18Reapply "Update Maxim's email address."Maxim Cournoyer1-1/+1
This reverts commit a7db92d9b389cbe565395523882c34c6eff2c21a, this time with the more careful command, to avoid a world rebuild: grep -rl --exclude-dir=build --exclude-dir=patches --exclude=ld-wrapper.in \ --exclude=sitecustomize.py --exclude=guix-emacs.el maxim.cournoyer@gmail.com | xargs sed -i 's/maxim.cournoyer@gmail.com/maxim@guixotic.coop/g' Change-Id: I0871f5cf9ace79e34e78e24154d0da6ea95a3a91
2025-09-18Revert "Update Maxim's email address."Maxim Cournoyer1-1/+1
This reverts commit d0d87a744d724b7e7d5ac013c586039652087bb3. Oops! This caused a world rebuild. Change-Id: I25fff644b2b61d0ee93d69b457b04c72b5b74d15
2025-09-18Update Maxim's email address.Maxim Cournoyer1-1/+1
The change was automated via: git grep -l maxim.cournoyer@gmail.com | xargs sed -i 's/maxim.cournoyer@gmail.com/maxim@guixotic.coop/g' * .mailmap: New entry. Change-Id: Iaa88b1e51c94159c49138fb43910badb990eb169
2024-01-05build: test-driver.scm: Set mode.Tomas Volf1-1/+2
Emacs auto-detects the build-aux/test-driver.scm as sh-mode, due to the shebang. So override the auto-detection using a local variable forcing scheme-mode. * build-aux/test-driver.scm: Set mode to scheme. Change-Id: I07f12656dba5fb7b52447b94569307a2a605ffc1 Signed-off-by: Maxim Cournoyer <maxim.cournoyer@gmail.com>
2021-02-02build: Add a --show-duration option to the SCM test-driver.Maxim Cournoyer1-9/+34
* build-aux/test-driver.scm (script-version): Update. (show-help): Document it. (%options): Add the 'show-duration' option. (test-runner-gnu): Pass as a new argument. [test-cases-start-time]: New inner variable. [test-on-test-begin-gnu]: New hook, used to record the start time. [test-on-test-end-gnu]: Conditionally print elapsed time. Record it as the optional metadata in the test result file (.trs). * doc/guix.texi (Running the Test Suite): Document it.
2021-01-31build: test-driver.scm: Allow running as a standalone script.Maxim Cournoyer1-1/+4
* build-aux/test-driver.scm: Add an exec-based shebang and set the script executable bit. (main): Insert a newline after the version string is printed with --version.
2021-01-31build: test-driver.scm: Add a new '--errors-only' option.Maxim Cournoyer1-32/+43
* build-aux/test-driver.scm (show-help): Add the help text for the new '--errors-only' option. (%options): Add the errors-only option. (test-runner-gnu): Add the errors-only? parameter and update doc. Move the logging of the test data after the test has completed, so a choice can be made whether to keep it or discard it based on the value of the test result. (main): Pass the errors-only? option to the driver. * doc/guix.texi (Running the Test Suite): Document the new option.
2021-01-31build: test-driver.scm: Add test cases filtering options.Maxim Cournoyer1-11/+63
* build-aux/test-driver.scm (show-help): Add help text for the new --select and --exclude options. (%options): Add the new select and exclude options. (test-runner-gnu): Pass them to the test runner. Update doc. (test-match-name*, test-match-name*/negated, %test-match-all): New variables. (main): Compute the test specifier based on the values of the new options and apply it to the current test runner when running the test file. * doc/guix.texi (Running the Test Suite): Document the new options.
2021-01-31build: test-driver.scm: Enable colored test results by default.Maxim Cournoyer1-2/+5
The Automake parallel test harness does its own smart detection of the terminal color capability and always provides the --color-tests argument to the driver. This change defaults the --color-tests argument to true when the test driver is run on its own (not via Automake). * build-aux/test-driver.scm (main): Set the default value of the --color-tests argument to true when it's not explicitly provided.
2021-01-31build: test-driver.scm: Make output redirection optional.Maxim Cournoyer1-15/+21
This makes it easier (and less surprising) for users to experiment with the custom Scheme test driver directly. The behavior is unchanged from Automake's point of view. * build-aux/test-driver.scm (main): Make the --log-file and --trs-file arguments optional and update doc. Only open, redirect and close a port to a log file when the --log-file option is provided. Only open and close a port to a trs file when the --trs-file option is provided. (test-runner-gnu): Set OUT-PORT parameter default value to the current output port. Set the TRS-PORT parameter default value to a void port. Update doc.
2017-03-22build: 'test-driver.scm' prints Scheme values with 'write'.Ludovic Courtès1-28/+23
Previously it would print them with 'display', which was inconvenient. * build-aux/test-driver.scm (test-display): Use ~S for VALUE. (test-runner-gnu): Use 'format' instead of 'test-display' for the test name, location, and result kind.
2016-04-03build: Add a Guile custom test driver using SRFI-64.Mathieu Lirzin1-0/+189
Before that '.log' files for scheme tests were fragmented and not included in test-suite.log. This unifies the semantics of SRFI-64 API with Automake test suite. * build-aux/test-driver.scm: New file. * Makefile.am (SCM_LOG_DRIVER, AM_SCM_LOG_DRIVER_FLAGS): New variables. (SCM_LOG_COMPILER, AM_SCM_LOG_FLAGS): Delete variables. (AM_TESTS_ENVIRONMENT): Set GUILE_AUTO_COMPILE to 0. * test-env.in: Silence guix-daemon. * doc/guix.texi (Running the Test Suite): Describe how to display the detailed results. Bug reports require only 'test-suite.log' file. * tests/base32.scm, tests/build-utils.scm, tests/builders.scm, tests/challenge.scm, tests/cpan.scm, tests/cpio.scm, tests/cran.scm, tests/cve.scm, tests/derivations.scm, tests/elpa.scm, tests/file-systems.scm, tests/gem.scm, tests/gexp.scm, tests/gnu-maintenance.scm, tests/grafts.scm, tests/graph.scm, tests/gremlin.scm, tests/hackage.scm, tests/hash.scm, tests/import-utils.scm, tests/lint.scm, tests/monads.scm, tests/nar.scm, tests/packages.scm, tests/pk-crypto.scm, tests/pki.scm, tests/profiles.scm, tests/publish.scm, tests/pypi.scm, tests/records.scm, tests/scripts-build.scm, tests/scripts.scm, tests/services.scm, tests/sets.scm, tests/size.scm, tests/snix.scm, tests/store.scm, tests/substitute.scm, tests/syscalls.scm, tests/system.scm, tests/ui.scm, tests/union.scm, tests/upstream.scm, tests/utils.scm: Don't exit at the end of test groups. * tests/containers.scm: Likewise. Use 'test-skip' instead of exiting with error code 77.
generated by cgit v1.2.3 (git 2.25.1) at 2026-09-03 07:14:38 +0000