-These tests require the setup of a configuration file with account
-information for MySQL and PostgreSQL SQL servers. Additionally,
-the Debian package acl-installer must be installed and a license downloaded
-to use the AODBC tests.
-
-This test suite looks for a configuration file named ".clsql-test.config"
-located in the users home directory.
-
-This file contains a single a-list that specifies the connection
- specs for each database type to be tested. For example, to test all
-platforms, a sample file might look like this:
-
-((:mysql ("localhost" "a-mysql-db" "user1" "secret"))
- (:aodbc ("my-dsn" "a-user" "pass"))
- (:postgresql ("localhost" "another-db" "user2" "dont-tell"))
- (:postgresql-socket ("pg-server" "a-db-name" "user" "secret-password")))
+* RUNNING THE REGRESSION SUITE
+
+Just load clsql.asd or put it somewhere where ASDF can find it
+and call:
+
+(asdf:oos 'asdf:load-op 'clsql)
+
+You'll then need to load a CLSQL backend before you can do anything.
+
+To run the regression tests load clsql-tests.asd or put it
+somewhere where ASDF can find it, edit the file tests/test-init.lisp
+and set the following variables to appropriate values:
+
+ *test-database-server*
+ *test-database-name*
+ *test-database-user*
+ *test-database-password*
+
+And then call:
+
+(asdf:oos 'asdf:load-op 'clsql-tests)
+(clsql-tests:test-usql BACKEND)
+
+where BACKEND is the CLSQL database interface to use (currently one of
+:postgresql, :postgresql-socket, :sqlite or :mysql).
+
+* REGRESSION TEST SUITE GOALS
+
+The intent of this test suite is to provide sufficient coverage for
+the system to support the following:
+
+** Refactoring and Redesign of particular subsystems
+
+Refactoring and redesign efforts are normally restricted to a single
+subsystem, or perhaps to interdependent subsystems. In such cases, a
+set of regression tests which excercise the existing interface of the
+rest of USQL to the changing subsystems should be in place and passing
+before the coding starts.
+
+** Ensuring portability and Supporting new ports.
+
+The more coverage the test suite provides the easier portability is to
+maintain, particularly if we have instances of the test suite running
+against the head on the supporting lisp environment/OS/hardware/DBMS
+combinations. Since no individual within the project has the ability
+to run all of those combinations themselves, we are dependent upon some
+informal coordination between the mintainers of the various ports.
+
+** Adding new RDBMS backends
+
+The entire USQL DBMS interface needs to be excercised by the test
+suite, such that a new RDBMS backend that passes all the tests can be
+reasonably assured of working with the USQL layers above that. These
+tests should also serve as impromptu documentation for the details of
+that interface and what it expects frothe RDBMS driver layers.
+
+** Bug identification and QA
+
+As new bugs are identified, they should have a regression test written
+which excercises them. This is to ensue that we donot start
+backtracking. These tests by theselves are also very valuable for
+developers, so even if you cannot fix a bug yourself, providing a
+testto excercise it greatly reduces the amount of timea developer must
+spend finding the bug prior to fixing it.
+
+
+* TEST DESIGN ISSUES
+
+** Multiple RDBMS Issues
+
+USQL supports several RDBMS backends, and it should be possible to run
+every test against all of them. However, there are some features
+which we want tests for but which are not implemented on several of
+the backends.
+
+** Test Hygiene
+
+Tests should be able to be run multiple times against the same
+database. It is also important that they clean up after themselves
+when they create tables, sequences or other pesistent entities in the
+RDBMS backends, because often there are limits to the number of those
+thatcan exist at one time, and it also makes debuging thru the SQL
+monitors difficult when there aretons of unused tables lying around.
+
+If test need to load large datasets, they should have a mechanism to
+ensure the dataset is loaded just once, and not with every test run.
+
+Lastly, because there are various idiosyncracies with RDBMSs, please
+ensure that you run the entire test suite once when you write your
+tests, to ensure that your test does not leave some state behind which
+causes other tests to fail.
+
+** Test Run Configuration
+
+The file test-init.lisp defines several variables which can be used to
+control the connection dictionary of the database against which tests
+will be run.
+
+
+* DATABASE CONNECTIONS/LIFECYCLE
+
+** CreateDB
+ *** Without existing DB
+ *** With existing DB and use old
+ *** With existing DB and use new
+ *** Error if existing DB
+
+** Data Definition
+ *** Create Tables/Sequences/Indexes -- Should cover creation of
+ tables with all supported types of fields.
+ *** Delete Tables/Sequences/Indexes
+ *** Inspection of Tables and attributes, including types
+
+** Data Manipulation
+ *** Update
+ *** Insert
+ *** Delete
+ *** Query
+
+** Functional Interface
+ *** Creation/Modification of SQL expressions
+ *** Querying
+
+** Embedded SQL syntax
+ *** Excercise all sql operators
+
+** Object Interface
+ *** View class definition
+ *** Object creation/manipulation/deletion
+ *** Inter-object Relations
+
+** Editing Contexts
+ *** Object Create/Modification/Deletion in a context -- partly covered already
+ *** Interaction of multiple contexts
+ *** Schema manipulation within a context
+ *** Rollback and error handling within a context
projects / clsql.git / blobdiff