1 <html><head><title>The Allegro CL Test harness</title></head><body><table border="0" width="100%" cellpadding="1" cellspacing="0"><tr><td colspan="2" bgcolor="#00FFFF"><table border="0" cellpadding="5" cellspacing="3"><tr><td align="left" bgcolor="#00FFFF"><a href="contents.htm"><b>ToC</b></a></td><td align="left" bgcolor="#00FFFF"><a href="introduction.htm"><b>DocOverview</b></a></td><td align="left" bgcolor="#00FFFF"><a href="cgide.htm"><b>CGDoc</b></a></td><td align="left" bgcolor="#00FFFF"><a href="release-notes.htm"><b>RelNotes</b></a></td><td align="left" bgcolor="#00FFFF"><a href="index.htm"><b>Index</b></a></td><td align="left" bgcolor="#00FFFF"><a href="permuted-index.htm"><b>PermutedIndex</b></a></td></tr></table></td><td align="right"><b>Allegro CL version 6.1</b><br><small><a href="introduction.htm#updates-s">Unrevised</a></small></td></tr></table><h1>The Allegro CL Test harness</h1><p>This document contains the following sections:</p><a href="#tester-api-1">1.0 The tester module API</a><br> <a href="#tester-vars-2">1.1 Test Harness Variables</a><br> <a href="#tester-macros-2">1.2 Test Harness Macros</a><br> <a href="#tester-examples-2">1.3 Examples</a><br><p>
2 ANSI Common Lisp contains no functionality designed specifically for
3 testing applications. Because testing is an essential part of
4 application development, Franz Inc. is making public the test harness
5 used internally for testing Allegro CL itself. (A test harness is a
6 collection of macros and variables associated with testing, along with
7 templates for test forms.)
9 The test harness facility was added to Allegro CL in release 6.0. (It
10 was available as a patch for release 5.0.1).
12 To use the test harness, you must load the
13 <i>tester.fasl</i> module. Do this by evaluating
16 </pre><hr><hr><h2><a name="tester-api-1">1.0 The tester module API</a></h2>
19 All of the following symbols are exported from the
20 <code>util.test</code> package.
23 <hr><h2><a name="tester-vars-2">1.1 Test Harness Variables</a></h2>
26 The test harness API includes the following variables, each described
27 fully on its own page and briefly here.
33 <a href="pages/variables/util.test/s_break-on-test-failures_s.htm"><code>*break-on-test-failures*</code></a>:
34 If true, <a href="../ansicl/dictentr/break.htm"><b>break</b></a> is called when
39 <a href="pages/variables/util.test/s_error-protect-tests_s.htm"><code>*error-protect-tests*</code></a>: If true, errors
40 (other than in a test-error form) will be considered a failure and
45 <a href="pages/variables/util.test/s_test-errors_s.htm"><code>*test-errors*</code></a>:
46 The value is the number of test errors which have occurred.
50 <a href="pages/variables/util.test/s_test-successes_s.htm"><code>*test-successes*</code></a>:
51 The value is the number of test successes which have occurred.
55 <a href="pages/variables/util.test/s_test-unexpected-failures_s.htm"><code>*test-unexpected-failures*</code></a>:
56 The value is the number of unexpected test failures which have occurred.
61 <hr><h2><a name="tester-macros-2">1.2 Test Harness Macros</a></h2>
64 These macros wrap around a form to be tested and supply the expected
65 value (for the test macro) or the expected behavior, which is encoded
66 in the macro name (e.g. test-error). For example:
70 (test 1 (+ 0 1)) ;; (testing that the result of (+ 0 1) is
72 (test-error (+ 1 "2")) ;; (testing that an error is
73 ;; signaled when a string is
74 ;; passed as an argument to +)
78 Many more examples are given below.
82 <a href="pages/operators/util.test/with-tests.htm"><b>with-tests</b></a> wraps around
83 a collection of <strong>test</strong> or <strong>test-*</strong>
88 Note that many of the macros have <em>fail-info</em> and
89 <em>known-failure</em> keyword arguments.
93 <li><em>fail-info</em>, if non-nil, should be a string that will be printed if the test
94 fails. Typical strings provide information about what is being tested, such as "This
96 <li><em>known-failure</em>, if non-nil, affects what is printed when the test fails or
97 succeeds. Thus a failure is reported as "Test failed: known failure: ..." and a
98 success as "Expected test failure for [...] did not occur."</li>
102 Each macro is described briefly here and fully on its documentation page.
109 <a href="pages/operators/util.test/test.htm"><b>test</b></a>
111 <p><b>Arguments: </b><i>
112 expected-value test-form
113 </i>&key <i></i> (<i>test</i> #'eql)<i> multiple-values fail-info known-failure</i><i>
116 Perform a single test and compare <i>expected-value</i>
117 with the value actually returned by <i>test-form</i>.
123 <a href="pages/operators/util.test/test-error.htm"><b>test-error</b></a>
125 <p><b>Arguments: </b><i>
127 </i>&key <i>announce catch-breaks fail-info known-failure</i> (<i>condition-type</i> 'simple-error)<i> include-subtypes format-control format-arguments</i><i>
130 Perform a single test to see whether
131 <i>form</i> signals an error.
137 <a href="pages/operators/util.test/test-no-error.htm"><b>test-no-error</b></a>
139 <p><b>Arguments: </b><i>
141 </i>&key <i>announce catch-breaks fail-info known-failure</i><i>
144 Perform a single test to see that
145 <i>form</i> does not signal an error.
151 <a href="pages/operators/util.test/test-warning.htm"><b>test-warning</b></a>
153 <p><b>Arguments: </b><i>
155 </i>&key <i>fail-info known-failure</i><i>
158 Perform a single test to see that
159 <i>form</i> signals a warning.
165 <a href="pages/operators/util.test/test-no-warning.htm"><b>test-no-warning</b></a>
167 <p><b>Arguments: </b><i>
169 </i>&key <i>fail-info known-failure</i><i>
172 Perform a single test to see that
173 <i>form</i> does not signal a warning.
179 <a href="pages/operators/util.test/with-tests.htm"><b>with-tests</b></a>
181 <p><b>Arguments: </b><i>
182 (</i>&key <i></i> (<i>name</i> "unnamed")<i>)
183 </i> &body <i>body</i><i>
187 <i>body</i>, which should be a list of test forms,
188 and reports on the results.
194 <hr><h2><a name="tester-examples-2">1.3 Examples</a></h2>
197 The following are simple examples using the test harness. The test
198 forms themselves are trivial, and the purpose is to indicate the
199 behavior of the test harness macros.
204 user(1): (require :tester)
205 ; Fasl loading .../tester.fasl
207 user(2): (use-package :util.test)
212 * * * UNEXPECTED TEST FAILURE * * *
217 user(5): (defun foo (x) x)
219 user(6): (test 1 (foo 1))
221 user(7): (test 1 (foo 2))
222 * * * UNEXPECTED TEST FAILURE * * *
227 user(8): (setq *break-on-test-failures* t)
229 user(9): (test 1 (foo 2))
230 * * * UNEXPECTED TEST FAILURE * * *
234 Break: *break-on-test-failures* is non-nil.
236 Restart actions (select using :continue):
237 0: return from break.
238 1: Return to Top Level (an "abort" restart)
239 2: Abort #<process Initial Lisp Listener>
241 user(11): (setq *break-on-test-failures* nil)
243 user(12): (test 1 (error "foo"))
246 Restart actions (select using :continue):
247 0: Return to Top Level (an "abort" restart)
248 1: Abort #<process Initial Lisp Listener>
250 user(14): (setq *error-protect-tests* t)
252 user(15): (test 1 (error "foo"))
253 Condition type: simple-error
255 * * * UNEXPECTED TEST FAILURE * * *
256 Test failed: (error "foo")
257 Reason: an error (of type `simple-error') was detected.
259 user(16): (setq *error-protect-tests* nil)
261 user(17): *test-errors*
263 user(18): *test-successes*
265 user(19): (test 1 2 :known-failure t)
266 Test failed: known failure: 2
270 user(20): (test 1 (foo 1) :known-failure t)
271 Expected test failure for (foo 1) did not occur.
273 user(21): (test 1 (foo 1) :known-failure t :fail-info "This is bug666.")
274 Expected test failure for (foo 1) did not occur.
275 Additional info: This is bug666.
277 user(22): (test-error (error "foo"))
279 user(23): (test-no-error (error "foo"))
280 * * * UNEXPECTED TEST FAILURE * * *
281 Test failed: (error "foo")
282 Reason: detected an unexpected error of type `simple-error'.
284 user(24): (test-error (car '(10)))
285 * * * UNEXPECTED TEST FAILURE * * *
286 Test failed: (car '(10))
287 Reason: expected but did not detect an error of type `condition'.
289 user(25): (test-no-error (car '(10)))
291 user(26): (test-warning (warn "foo"))
293 user(27): (test-no-warning (warn "foo"))
294 * * * UNEXPECTED TEST FAILURE * * *
295 Test failed: (warn "foo")
299 user(28): (test-warning (car '(10)))
300 * * * UNEXPECTED TEST FAILURE * * *
301 Test failed: (car '(10))
305 user(29): (test-no-warning (car '(10)))
307 user(30): (test-error (error "foo: ~a" 10))
309 user(31): (test-error (error "foo: ~a" 10) :format-control "foo: ~a")
311 user(32): (test-error (error "foo: ~a" 10) :format-control "foo: ~a"
312 :format-arguments '(10))
314 user(33): (test-error (error "foo: ~a" 10) :format-control "foo: ~a")
315 * * * UNEXPECTED TEST FAILURE * * *
316 Test failed: (error "foo: ~a" 10)
317 Reason: the format-control was incorrect.
318 wanted: "~1@<foo: ~a~:@>"
319 got: "~1@<foo: ~a~:@>"
321 user(34): (test-error (error "foo: ~a" 10) :format-control "foo: ~a"
322 :format-arguments '(11))
323 * * * UNEXPECTED TEST FAILURE * * *
324 Test failed: (error "foo: ~a" 10)
325 Reason: the format-arguments were incorrect.
329 user(35): (test-error (error "foo: ~a" 10) :condition-type 'condition
332 user(36): (test-error (error "foo: ~a" 10) :condition-type 'simple-break
334 * * * UNEXPECTED TEST FAILURE * * *
335 Test failed: (error "foo: ~a" 10)
336 Reason: detected an incorrect condition type.
338 got: #<standard-class simple-error>
340 user(37): (test-error (break "foo: ~a" 10) :condition-type 'simple-break
343 [condition type: simple-break]
345 Restart actions (select using :continue):
346 0: return from break.
347 1: Return to Top Level (an "abort" restart)
348 2: Abort #<process Initial Lisp Listener>
350 user(39): (test-error (break "foo: ~a" 10) :catch-breaks t
351 :condition-type 'simple-break :include-subtypes t)
355 </body><hr><p><small>Copyright (c) 1998-2001, Franz Inc. Berkeley, CA., USA. All rights reserved.</small><br><small>Documentation for Allegro CL version 6.1 update # 1. This page was not revised.</small><br><small>Created 2001.12.15.</small></p><table border="0" width="100%" cellpadding="1" cellspacing="0"><tr><td colspan="2" bgcolor="#00FFFF"><table border="0" cellpadding="5" cellspacing="3"><tr><td align="left" bgcolor="#00FFFF"><a href="contents.htm"><b>ToC</b></a></td><td align="left" bgcolor="#00FFFF"><a href="introduction.htm"><b>DocOverview</b></a></td><td align="left" bgcolor="#00FFFF"><a href="cgide.htm"><b>CGDoc</b></a></td><td align="left" bgcolor="#00FFFF"><a href="release-notes.htm"><b>RelNotes</b></a></td><td align="left" bgcolor="#00FFFF"><a href="index.htm"><b>Index</b></a></td><td align="left" bgcolor="#00FFFF"><a href="permuted-index.htm"><b>PermutedIndex</b></a></td></tr></table></td><td align="right"><b>Allegro CL version 6.1</b><br><small><a href="introduction.htm#updates-s">Unrevised</a></small></td></tr></table></html>