Details
-
Bug
-
Status: In Progress
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
Description
The current dialect tests are quick and robust but not very thorough. In RelToSqlConverterTest there are about 400 tests (374 SQL statements and 30 relational expressions) that test translation to 538 SQL strings in target dialects (an average of 1.34 dialects per test). To run the 371 test methods takes about 12 seconds elapsed, 80 CPU seconds on my MacBook Pro. None of those SQL strings are executed, so we don't know whether they return the correct results or are even valid.
Improvements:
- Define a list of dialects that we wish to test (the "target dialects") and provide a means to connect to a database for each of these (typically a driver on the class path, a URL, user name and password);
- In each test, for each target dialect, generate a SQL string. The generation should not throw.
- In each test, the query is first executed against Calcite to create a sample. (For any test, this can be overridden to use a different reference dialect. Calcite will use a recording, if present.)
- In each test, for each target dialect that has an enabled connection, execute the generated SQL against that connection. If there is a reference result for the test, check that the result equals the reference result.
- In each test, for each target dialect that has a recording connection, execute the generated SQL against that connection. Record the outcome (error, column names and types, result) in the file. Proceed as for an enabled connection. (Check result against reference result, etc.)
- In each test, for each target dialect that has a replaying connection, look up the outcome of the generated SQL. If the outcome is present, proceed as for an enabled connection. (Check result against reference result, etc.)
- In each test, for each target dialect that has a replaying+recording connection, look up the outcome of the generated SQL. If the outcome is present, proceed as for an enabled connection. (Check result against reference result, etc.) If the outcome is not present, proceed as for a recording connection. (Execute the SQL, record the outcome, check against reference result, etc.)
With these improvements, the test suite:
- Remains fast and robust (because recordings are used most of the time);
- Can be run offline (using recordings);
- Can be run by contributors who do not necessarily have access to all of the supported databases;
- Has full coverage (runs all tests against all supported dialects);
- Can re-generate the 'golden' references. (Must be online, have the required driver and connection, and takes more time.)
Design:
- A new class DialectTestConfig describes the dialects to be tested, their names, connections, the connection state (enabled, recording, replaying, replaying+recording), and recording file name.
- A mock JDBC driver mediates recording/replay.
Attachments
Issue Links
- relates to
-
CALCITE-6458 RexPermuteInputsShuttle produces ArrayIndexOutOfBoundsException for join conditions with subselect
-
- Open
-
-
CALCITE-4723 Check whether JDBC adapter generates "GROUP BY ()" against Oracle, DB2, MSSQL
-
- Resolved
-
-
CALCITE-6833 JDBC adapter generates invalid table alias for semi-join in UNION
-
- Closed
-
-
CALCITE-6834 In query that applies COALESCE to nullable SUM, EnumerableProjectToCalcRule throws AssertionError
-
- Closed
-
-
CALCITE-6837 Invalid code generated for ROW_NUMBER function in Enumerable convention
-
- Closed
-
-
CALCITE-6281 Add test infrastructure to ensure that Calcite is consistent with the SQL dialects it is trying to emulate
-
- In Progress
-
-
CALCITE-6693 Add Source SQL Dialect to RelToSqlConverterTest
-
- Closed
-
- links to