Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-5529

Improve dialect tests

    XMLWordPrintableJSON

Details

    • Bug
    • Status: In Progress
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None

    Description

      The current dialect tests are quick and robust but not very thorough. In RelToSqlConverterTest there are about 400 tests (374 SQL statements and 30 relational expressions) that test translation to 538 SQL strings in target dialects (an average of 1.34 dialects per test). To run the 371 test methods takes about 12 seconds elapsed, 80 CPU seconds on my MacBook Pro. None of those SQL strings are executed, so we don't know whether they return the correct results or are even valid.

      Improvements:

      • Define a list of dialects that we wish to test (the "target dialects") and provide a means to connect to a database for each of these (typically a driver on the class path, a URL, user name and password);
      • In each test, for each target dialect, generate a SQL string. The generation should not throw.
      • In each test, the query is first executed against Calcite to create a sample. (For any test, this can be overridden to use a different reference dialect. Calcite will use a recording, if present.)
      • In each test, for each target dialect that has an enabled connection, execute the generated SQL against that connection. If there is a reference result for the test, check that the result equals the reference result.
      • In each test, for each target dialect that has a recording connection, execute the generated SQL against that connection. Record the outcome (error, column names and types, result) in the file. Proceed as for an enabled connection. (Check result against reference result, etc.)
      • In each test, for each target dialect that has a replaying connection, look up the outcome of the generated SQL. If the outcome is present, proceed as for an enabled connection. (Check result against reference result, etc.)
      • In each test, for each target dialect that has a replaying+recording connection, look up the outcome of the generated SQL. If the outcome is present, proceed as for an enabled connection. (Check result against reference result, etc.) If the outcome is not present, proceed as for a recording connection. (Execute the SQL, record the outcome, check against reference result, etc.)

      With these improvements, the test suite:

      • Remains fast and robust (because recordings are used most of the time);
      • Can be run offline (using recordings);
      • Can be run by contributors who do not necessarily have access to all of the supported databases;
      • Has full coverage (runs all tests against all supported dialects);
      • Can re-generate the 'golden' references. (Must be online, have the required driver and connection, and takes more time.)

      Design:

      • A new class DialectTestConfig describes the dialects to be tested, their names, connections, the connection state (enabled, recording, replaying, replaying+recording), and recording file name.
      • A mock JDBC driver mediates recording/replay.

      Attachments

        Issue Links

          Activity

            People

              julianhyde Julian Hyde
              julianhyde Julian Hyde
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated: