PERF: Optimize execute() hot path: soft reset, prepare caching, and guarded diagnostics #528
Draft
bewithgaurav wants to merge 8 commits intomainfrom
Draft
PERF: Optimize execute() hot path: soft reset, prepare caching, and guarded diagnostics #528bewithgaurav wants to merge 8 commits intomainfrom
bewithgaurav wants to merge 8 commits intomainfrom
Conversation
…n, CI artifact pipeline - Add insertmanyvalues scenario (100K rows, 1000/batch, GH #500 repro) - Add --baseline flag for 3-way comparison (main vs PR vs pyodbc) - Add --json flag to save results for pipeline artifact consumption - Publish benchmark results as CI artifact on main merge - Download baseline artifact on PR runs for automatic regression detection - Fix timing: time.time() -> time.perf_counter() - Move connection setup outside timing window (measure execute+fetch only)
📊 Code Coverage Report
Diff CoverageDiff: main...HEAD, staged and unstaged changes
Summary
mssql_python/cursor.pyLines 1365-1373 1365 if reset_cursor:
1366 if self.hstmt:
1367 self._soft_reset_cursor()
1368 else:
! 1369 self._reset_cursor()
1370
1371 # Clear any previous messages
1372 self.messages = []mssql_python/pybind/ddbc_bindings.cppLines 1363-1376 1363 }
1364
1365 SQLRETURN SQLResetStmt_wrap(SqlHandlePtr statementHandle) {
1366 if (!SQLFreeStmt_ptr) {
! 1367 DriverLoader::getInstance().loadDriver();
! 1368 }
1369 SQLHANDLE hStmt = statementHandle->get();
1370 if (!hStmt) {
! 1371 return SQL_INVALID_HANDLE;
! 1372 }
1373
1374 SQLRETURN rc = SQLFreeStmt_ptr(hStmt, SQL_CLOSE);
1375 if (SQL_SUCCEEDED(rc)) {
1376 rc = SQLFreeStmt_ptr(hStmt, SQL_RESET_PARAMS);📋 Files Needing Attention📉 Files with overall lowest coverage (click to expand)mssql_python.pybind.logger_bridge.cpp: 59.2%
mssql_python.pybind.ddbc_bindings.h: 67.8%
mssql_python.row.py: 70.5%
mssql_python.pybind.logger_bridge.hpp: 70.8%
mssql_python.pybind.ddbc_bindings.cpp: 74.3%
mssql_python.pybind.connection.connection.cpp: 75.8%
mssql_python.__init__.py: 77.3%
mssql_python.ddbc_bindings.py: 79.6%
mssql_python.pybind.connection.connection_pool.cpp: 79.6%
mssql_python.connection.py: 85.2%🔗 Quick Links
|
- Download and restore AdventureWorks2022 via docker cp + sqlcmd - Install pyodbc and run perf-benchmarking.py with --baseline/--json - Publish/download baseline artifacts (perf-baseline-macOS-SQL2022/SQL2025) - Same pattern as Windows, adapted for Colima Docker
ab57bb8 to
6703b85
Compare
… handle empty baseline
6703b85 to
2ee0f68
Compare
…ux conditional baseline
2ee0f68 to
c113a03
Compare
c113a03 to
9ed69e4
Compare
- Add _soft_reset_cursor: SQL_CLOSE + SQL_RESET_PARAMS instead of full HSTMT free/realloc on each execute() call - Add DDBCSQLResetStmt C++ wrapper exposing lightweight reset via pybind11 - Skip SQLPrepare when re-executing the same SQL (prepare caching) - Skip detect_and_convert_parameters on repeated same-SQL calls - Guard DDBCSQLGetAllDiagRecords behind SQL_SUCCESS_WITH_INFO check - Guard per-parameter debug logging behind logger.isEnabledFor(DEBUG) - Fix benchmark script to strip Driver= from mssql-python connection string
5b92dc7 to
970ce4d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Work Item / Issue Reference
Summary
This pull request introduces performance benchmarking infrastructure improvements and optimizations to the
mssql_pythondriver, as well as documentation and workflow updates to support a newPERF:pull request prefix. The most significant changes are grouped below.Driver Performance Optimizations:
_soft_reset_cursorincursor.pyto reuse prepared statement handles, avoiding unnecessary SQLPrepare calls when executing the same SQL repeatedly. This improves performance for repeated statement execution. [1] [2] [3] [4]DDBCSQLResetStmtbinding in the C++ layer to support the lightweight reset operation, and exposed it to Python. [1] [2]