-
Notifications
You must be signed in to change notification settings - Fork 398
Implement EET oracle #1349
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Implement EET oracle #1349
Changes from all commits
Commits
Show all changes
14 commits
Select commit
Hold shift + click to select a range
eb5e013
Add EET (Equivalent Expression Transformation) oracle for MySQL
tlmorgan24 7c9f80f
Clarify that the E in the EET generics must extend the Expression class
tlmorgan24 95dcc8e
Refactor EET oracle structure for consistency and extensibility
tlmorgan24 657ec00
Move EET transformer ownership to EETOracle, removing EET-specific st…
tlmorgan24 65957ad
Rearrange MySQLExpressionGenerator code to group oracle-specific sect…
tlmorgan24 03c4874
Remove need for EETNodeFactory by merging it into EETTransformer
tlmorgan24 9285e91
Correct EET transformation rule comments
tlmorgan24 52ce866
Implement rules 3 and 4 for EET
tlmorgan24 86e5b71
Fix EET reproducer/reduction to port the same fixes to it that were e…
tlmorgan24 912197d
Add FLOAT and DECIMAL to MySQL EET target cast types
tlmorgan24 4d6a651
Fix treatment of unary minus in MySQL EET implementation
tlmorgan24 e3d9f61
Restrict creation of (M, D) columns to prevent false positives in EET
tlmorgan24 0a09fba
Fix checkstyle violations
tlmorgan24 c71d198
Refactor EETTransformer to encode the transformation rules as an enum
tlmorgan24 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| package sqlancer.common.gen; | ||
|
|
||
| import java.util.List; | ||
|
|
||
| import sqlancer.common.ast.newast.Expression; | ||
| import sqlancer.common.ast.newast.Join; | ||
| import sqlancer.common.ast.newast.Select; | ||
| import sqlancer.common.oracle.EETTransformer; | ||
| import sqlancer.common.schema.AbstractTable; | ||
| import sqlancer.common.schema.AbstractTableColumn; | ||
| import sqlancer.common.schema.AbstractTables; | ||
|
|
||
| /** | ||
| * Generator interface used by {@link sqlancer.common.oracle.EETOracle}. In addition to generating a random query (like | ||
| * the other oracle generators), an EET generator creates a DBMS-specific {@link EETTransformer} that the oracle uses to | ||
| * rewrite expressions into semantically equivalent ones. | ||
| * | ||
| * @param <S> | ||
| * the DBMS-specific SELECT statement class | ||
| * @param <J> | ||
| * the DBMS-specific JOIN clause class | ||
| * @param <E> | ||
| * the DBMS-specific expression class | ||
| * @param <T> | ||
| * the DBMS-specific table class | ||
| * @param <C> | ||
| * the DBMS-specific column class | ||
| */ | ||
| public interface EETGenerator<S extends Select<J, E, T, C>, J extends Join<E, T, C>, E extends Expression<C>, T extends AbstractTable<C, ?, ?>, C extends AbstractTableColumn<?, ?>> { | ||
|
|
||
| EETGenerator<S, J, E, T, C> setTablesAndColumns(AbstractTables<T, C> tables); | ||
|
|
||
| S generateSelect(); | ||
|
|
||
| List<J> getRandomJoinClauses(); | ||
|
|
||
| List<E> getTableRefs(); | ||
|
|
||
| List<E> generateFetchColumns(boolean shouldCreateDummy); | ||
|
|
||
| E generateBooleanExpression(); | ||
|
|
||
| /** | ||
| * Creates a DBMS-specific {@link EETTransformer} backed by this generator. Called once by | ||
| * {@link sqlancer.common.oracle.EETOracle} during construction; the oracle owns the returned transformer for the | ||
| * lifetime of the test run. | ||
| * | ||
| * @return a DBMS-specific {@link EETTransformer} backed by this generator | ||
| */ | ||
| EETTransformer<E, ?> createTransformer(); | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,201 @@ | ||
| package sqlancer.common.oracle; | ||
|
|
||
| import java.sql.SQLException; | ||
| import java.util.List; | ||
| import java.util.stream.Collectors; | ||
|
|
||
| import sqlancer.ComparatorHelper; | ||
| import sqlancer.Reproducer; | ||
| import sqlancer.SQLGlobalState; | ||
| import sqlancer.common.ast.newast.Expression; | ||
| import sqlancer.common.ast.newast.Join; | ||
| import sqlancer.common.ast.newast.Select; | ||
| import sqlancer.common.gen.EETGenerator; | ||
| import sqlancer.common.query.ExpectedErrors; | ||
| import sqlancer.common.schema.AbstractSchema; | ||
| import sqlancer.common.schema.AbstractTable; | ||
| import sqlancer.common.schema.AbstractTableColumn; | ||
| import sqlancer.common.schema.AbstractTables; | ||
|
|
||
| /** | ||
| * EET (Equivalent Expression Transformation) oracle, based on "Detecting Logic Bugs in Database Engines via Equivalent | ||
| * Expression Transformation" (Jiang & Su, OSDI'24). | ||
| * | ||
| * <p> | ||
| * The oracle generates a random query and then transforms its expressions (the WHERE predicate and the fetch columns) | ||
| * into semantically equivalent ones using {@link EETGenerator#transformExpression}. Because the transformation | ||
| * preserves semantics, the original and the transformed query must return the same result set; any discrepancy | ||
| * indicates a logic bug in the DBMS. | ||
| * | ||
| * @param <Z> | ||
| * the DBMS-specific SELECT statement class | ||
| * @param <J> | ||
| * the DBMS-specific JOIN clause class | ||
| * @param <E> | ||
| * the DBMS-specific expression class | ||
| * @param <S> | ||
| * the DBMS-specific schema class | ||
| * @param <T> | ||
| * the DBMS-specific table class | ||
| * @param <C> | ||
| * the DBMS-specific column class | ||
| * @param <G> | ||
| * the DBMS-specific global state class | ||
| */ | ||
| public class EETOracle<Z extends Select<J, E, T, C>, J extends Join<E, T, C>, E extends Expression<C>, S extends AbstractSchema<?, T>, T extends AbstractTable<C, ?, ?>, C extends AbstractTableColumn<?, ?>, G extends SQLGlobalState<?, S>> | ||
| implements TestOracle<G> { | ||
|
|
||
| private final G state; | ||
| private EETGenerator<Z, J, E, T, C> gen; | ||
| private final EETTransformer<E, ?> transformer; | ||
| private final ExpectedErrors errors; | ||
|
|
||
| private Reproducer<G> reproducer; | ||
| private String generatedQueryString; | ||
|
|
||
| private final class EETReproducer implements Reproducer<G> { | ||
| private final String originalQueryString; | ||
| // null if the original bug was a DBMS error on the original query alone | ||
| private final String transformedQueryString; | ||
| // null if the original bug is a result set mismatch; otherwise, the message of the | ||
| // unexpected DBMS error that the original or transformed query triggered | ||
| private final String expectedErrorMessage; | ||
|
|
||
| EETReproducer(String originalQueryString, String transformedQueryString, String expectedErrorMessage) { | ||
| this.originalQueryString = originalQueryString; | ||
| this.transformedQueryString = transformedQueryString; | ||
| this.expectedErrorMessage = expectedErrorMessage; | ||
| } | ||
|
|
||
| @Override | ||
| public boolean bugStillTriggers(G globalState) { | ||
| List<String> originalResultSet; | ||
| List<String> transformedResultSet; | ||
| try { | ||
| // Re-execute both queries against the current (reduced) database instead of comparing | ||
| // against a cached result set, which would be stale once statements have been removed. | ||
| originalResultSet = ComparatorHelper.getResultSetFirstColumnAsString(originalQueryString, errors, | ||
| globalState); | ||
| if (transformedQueryString == null) { | ||
| // the original bug was a DBMS error on the original query alone, which no | ||
| // longer occurs | ||
| return false; | ||
| } | ||
| transformedResultSet = ComparatorHelper.getResultSetFirstColumnAsString(transformedQueryString, errors, | ||
| globalState); | ||
| } catch (AssertionError unexpectedError) { | ||
| // a DBMS error reproduces the bug only if the original failure was the same error; | ||
| // other errors are artifacts of the reduction (e.g., a removed CREATE TABLE) | ||
| return expectedErrorMessage != null | ||
| && expectedErrorMessage.equals(TestOracleUtils.getUnexpectedErrorMessage(unexpectedError)); | ||
| } catch (SQLException | RuntimeException e) { | ||
| return false; | ||
| } | ||
| if (expectedErrorMessage != null) { | ||
| // the original bug was a DBMS error, which no longer occurs | ||
| return false; | ||
| } | ||
| try { | ||
| ComparatorHelper.assumeResultSetsAreEqual(originalResultSet, transformedResultSet, originalQueryString, | ||
| List.of(transformedQueryString), globalState); | ||
| } catch (AssertionError resultSetMismatch) { | ||
| return true; | ||
| } | ||
| return false; | ||
| } | ||
|
|
||
| @Override | ||
| public String getBugInformation() { | ||
| StringBuilder sb = new StringBuilder(); | ||
| if (expectedErrorMessage == null) { | ||
| sb.append("-- On the database set up by the statements above, the result sets of the following" | ||
| + " queries mismatch:").append(System.lineSeparator()); | ||
| } else { | ||
| sb.append("-- On the database set up by the statements above, the following queries trigger an" | ||
| + " unexpected error with message: ").append(expectedErrorMessage) | ||
| .append(System.lineSeparator()); | ||
| } | ||
| sb.append("-- original: ").append(originalQueryString).append(';').append(System.lineSeparator()); | ||
| if (transformedQueryString != null) { | ||
| sb.append("-- transformed: ").append(transformedQueryString).append(';').append(System.lineSeparator()); | ||
| } | ||
| return sb.toString(); | ||
| } | ||
| } | ||
|
|
||
| public EETOracle(G state, EETGenerator<Z, J, E, T, C> gen, ExpectedErrors expectedErrors) { | ||
| if (state == null || gen == null || expectedErrors == null) { | ||
| throw new IllegalArgumentException("Null variables used to initialize test oracle."); | ||
| } | ||
| this.state = state; | ||
| this.gen = gen; | ||
| this.transformer = gen.createTransformer(); | ||
| this.errors = expectedErrors; | ||
| } | ||
|
|
||
| @Override | ||
| public void check() throws SQLException { | ||
| reproducer = null; | ||
| S schema = state.getSchema(); | ||
| AbstractTables<T, C> targetTables = TestOracleUtils.getRandomTableNonEmptyTables(schema); | ||
| gen = gen.setTablesAndColumns(targetTables); | ||
|
|
||
| Z select = gen.generateSelect(); | ||
| select.setJoinClauses(gen.getRandomJoinClauses()); | ||
| select.setFromList(gen.getTableRefs()); | ||
| List<E> fetchColumns = gen.generateFetchColumns(true); | ||
| select.setFetchColumns(fetchColumns); | ||
| E whereClause = gen.generateBooleanExpression(); | ||
| select.setWhereClause(whereClause); | ||
|
|
||
| String originalQueryString = select.asString(); | ||
| generatedQueryString = originalQueryString; | ||
| List<String> originalResultSet; | ||
| try { | ||
| originalResultSet = ComparatorHelper.getResultSetFirstColumnAsString(originalQueryString, errors, state); | ||
| } catch (AssertionError unexpectedError) { | ||
| // an unexpected DBMS error on the original query alone is itself a bug worth reducing; | ||
| // transformedQueryString is null because no transformed query is involved | ||
| reproducer = new EETReproducer(originalQueryString, null, | ||
| TestOracleUtils.getUnexpectedErrorMessage(unexpectedError)); | ||
| throw unexpectedError; | ||
| } | ||
|
|
||
| // Transform the query's expressions into semantically equivalent ones. Fetch columns are scalar expressions, | ||
| // while the WHERE clause is evaluated in a boolean context. | ||
| List<E> transformedFetchColumns = fetchColumns.stream().map(c -> transformer.transform(c, false)) | ||
| .collect(Collectors.toList()); | ||
| select.setFetchColumns(transformedFetchColumns); | ||
| select.setWhereClause(transformer.transform(whereClause, true)); | ||
|
|
||
| String transformedQueryString = select.asString(); | ||
| List<String> transformedResultSet; | ||
| try { | ||
| transformedResultSet = ComparatorHelper.getResultSetFirstColumnAsString(transformedQueryString, errors, | ||
| state); | ||
| } catch (AssertionError unexpectedError) { | ||
| // the semantics-preserving transformation made the query trigger a DBMS error that the | ||
| // original did not, which is a bug worth reducing | ||
| reproducer = new EETReproducer(originalQueryString, transformedQueryString, | ||
| TestOracleUtils.getUnexpectedErrorMessage(unexpectedError)); | ||
| throw unexpectedError; | ||
| } | ||
|
|
||
| // Set the reproducer before the assertion: assumeResultSetsAreEqual throws when the bug is | ||
| // detected, so creating the reproducer afterwards would leave it null and prevent any reduction. | ||
| reproducer = new EETReproducer(originalQueryString, transformedQueryString, null); | ||
|
|
||
| ComparatorHelper.assumeResultSetsAreEqual(originalResultSet, transformedResultSet, originalQueryString, | ||
| List.of(transformedQueryString), state); | ||
| } | ||
|
|
||
| @Override | ||
| public Reproducer<G> getLastReproducer() { | ||
| return reproducer; | ||
| } | ||
|
|
||
| @Override | ||
| public String getLastQueryString() { | ||
| return generatedQueryString; | ||
| } | ||
| } | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks good, but I'm also wondering whether the error-handling functionality is something that we could factor out and reuse among all the test oracles?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, that would be good. I plan to implement testing of DML statements for EET first (initially without test case reduction). After that, I think the different requirements will be clearer for how best to factor out and reuse the reproducer/reducer logic across the whole of EET and the other oracles.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In fact, having thought about it, I will do a PR for this refactor now, before any PR for implementing additional DML support