[PECOBLR-2086] Add comprehensive MST transaction metadata and edge case tests by vikrantpuppala · Pull Request #1 · vikrantpuppala/databricks-sql-python

vikrantpuppala · 2026-04-02T06:50:53Z

Summary

Adds comprehensive E2E transaction tests to tests/e2e/test_transactions.py covering gaps identified in the MST + xDBC Metadata RPCs audit
Removes @pytest.mark.skip from TestTransactions class so tests run against MST-enabled warehouses
Covers metadata operations in transactions (cursor.columns, cursor.tables, cursor.schemas, cursor.catalogs, SHOW/DESCRIBE statements)
Tests MSTCheckRule-blocked SQL statements and their error handling
Tests edge cases: empty transactions, DDL in transactions, connection close with pending txn, concurrent operations, batch operations, PreparedStatement-style parameterized queries
Renames test table to pysql_transaction_test_table to avoid collisions with JDBC driver tests
Fixes catalogOwned-preview → catalogManaged table property

Test plan

Run pytest tests/e2e/test_transactions.py against an MST-enabled warehouse
Verify all new tests pass alongside the existing tests

NO_CHANGELOG=true

This pull request was AI-assisted by Isaac.

…re (databricks#553) * Clean thrift files Signed-off-by: Vikrant Puppala <[email protected]>

…atabricks#544) This PR replaces the previous implementation of convert_decimals_in_arrow_table() with a more efficient approach that uses PyArrow's native casting operation instead of going through pandas conversion and array creation. - Remove conversion to pandas DataFrame via to_pandas() and apply() methods - Remove intermediate steps of creating array from decimal column and setting it back - Replace with direct type casting using PyArrow's cast() method - Build a new table with transformed columns rather than modifying the original table - Create a new schema based on the modified fields The new approach is more performant by avoiding pandas conversion overhead. The table below highlights substantial performance improvements when retrieving all rows from a table containing decimal columns, particularly when compression is disabled. Even greater gains were observed with compression enabled—showing approximately an 84% improvement (6 seconds compared to 39 seconds). Benchmarking was performed against e2-dogfood, with the client located in the us-west-2 region. ![image](https://github.com/user-attachments/assets/5407b651-8ab6-4c13-b525-cf912f503ba0) Signed-off-by: Jayant Singh <[email protected]>

…cks#551)

new codeowners

…t to prevent server side resource leaks (databricks#554) * Enhance Cursor close handling and context manager exception management * tests * fmt * Fix Cursor.close() to properly handle CursorAlreadyClosedError * Remove specific test message from Cursor.close() error handling * Improve error handling in connection and cursor context managers to ensure proper closure during exceptions, including KeyboardInterrupt. Add tests for nested cursor management and verify operation closure on server-side errors. * add * add

* PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <[email protected]> * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <[email protected]> * fixed format Signed-off-by: Sai Shree Pradhan <[email protected]> * used lazy logging Signed-off-by: Sai Shree Pradhan <[email protected]> * changed debug to error logs Signed-off-by: Sai Shree Pradhan <[email protected]> * used lazy logging Signed-off-by: Sai Shree Pradhan <[email protected]> --------- Signed-off-by: Sai Shree Pradhan <[email protected]>

More conditions to run github actions

* Added new table * Some more fixes

* [init] working code for corrected test Signed-off-by: varun-edachali-dbx <[email protected]> * excess comments Signed-off-by: varun-edachali-dbx <[email protected]> * use pytest parametrize instead of unittest subtests, switch to pytest instead of unittest primitives Signed-off-by: varun-edachali-dbx <[email protected]> * initialisations, function calls, then assertions Signed-off-by: varun-edachali-dbx <[email protected]> * remove pytest parametrize, move back to unittest subTest to allow keeping the test inside ClientTestSuite Signed-off-by: varun-edachali-dbx <[email protected]> * clean up docstring Signed-off-by: varun-edachali-dbx <[email protected]> * remove pytest import Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]>

* Basic testing * testing examples * Basic working prototype * ttypes fix * Refractored the ttypes * nit * Added inline support * Reordered boolean to be above int * Check Working e2e tests prototype * More tests * Added unit tests * refractor * nit * nit * nit * nit

* Added change log * Update CHANGELOG.md Co-authored-by: Vikrant Puppala <[email protected]> --------- Co-authored-by: Vikrant Puppala <[email protected]>

nit

* Add test to check thrift field IDs --------- Signed-off-by: Vikrant Puppala <[email protected]>

databricks#613) * Revert "Enhance Cursor close handling and context manager exception management to prevent server side resource leaks (databricks#554)" This reverts commit edfb283. * revert e2e

* Release version 4.0.5: Reverted cursor close handling changes to fix user errors. Updated version numbers in pyproject.toml and __init__.py. * Update CHANGELOG.md to include reference to issue databricks#613 for cursor close handling fix.

nit

…#621) * basic setup * Nit * working * moved pyjwt to code dependency * nit * nit * nit * nit * nit * nit * testing sdk * Refractor * logging * nit * nit * nit * nit * nit

* [PECO-197] Support Python 3.10 (#31) * Test with multiple python versions. * Update pyarrow to version 9.0.0 to address issue in relation to python 3.10 & a specific version of numpy being pulled in by pyarrow. Closes #26 Signed-off-by: David Black <[email protected]> * Update changelog and bump to v2.0.4 (#34) * Update changelog and bump to v2.0.4 * Specifically thank @dbaxa for this change. Signed-off-by: Jesse Whitehouse <[email protected]> * Bump to 2.0.5-dev on main (#35) Signed-off-by: Jesse Whitehouse <[email protected]> * On Pypi, display the "Project Links" sidebar. (#36) Signed-off-by: Jesse Whitehouse <[email protected]> * [ES-402013] Close cursors before closing connection (#38) * Add test: cursors are closed when connection closes Signed-off-by: Jesse Whitehouse <[email protected]> * Bump version to 2.0.5 and improve CHANGELOG (#40) Signed-off-by: Jesse Whitehouse <[email protected]> * fix dco issue Signed-off-by: Moe Derakhshani <[email protected]> * fix dco issue Signed-off-by: Moe Derakhshani <[email protected]> * dco tunning Signed-off-by: Moe Derakhshani <[email protected]> * dco tunning Signed-off-by: Moe Derakhshani <[email protected]> * Github workflows: run checks on pull requests from forks (#47) Signed-off-by: Jesse Whitehouse <[email protected]> * OAuth implementation (#15) This PR: * Adds the foundation for OAuth against Databricks account on AWS with BYOIDP. * It copies one internal module that Steve Weis @sweisdb wrote for Databricks CLI (oauth.py). Once ecosystem-dev team (Serge, Pieter) build a python sdk core we will move this code to their repo as a dependency. * the PR provides authenticators with visitor pattern format for stamping auth-token which later is intended to be moved to the repo owned by Serge @nfx and and Pieter @pietern * Automate deploys to Pypi (#48) Signed-off-by: Jesse Whitehouse <[email protected]> * [PECO-205] Add functional examples (#52) Signed-off-by: Jesse Whitehouse <[email protected]> * Bump version to 2.1.0 (#54) Bump to v2.1.0 and update changelog Signed-off-by: Jesse Whitehouse <[email protected]> * [SC-110400] Enabling compression in Python SQL Connector (#49) Signed-off-by: Mohit Singla <[email protected]> Co-authored-by: Moe Derakhshani <[email protected]> * Add tests for parameter sanitisation / escaping (#46) * Refactor so we can unit test `inject_parameters` * Add unit tests for inject_parameters * Remove inaccurate comment. Per #51, spark sql does not support escaping a single quote with a second single quote. * Closes #51 and adds unit tests plus the integration test provided in #56 Signed-off-by: Jesse Whitehouse <[email protected]> Co-authored-by: Courtney Holcomb (@courtneyholcomb) Co-authored-by: @mcannamela * Bump thrift dependency to 0.16.0 (#65) Addresses https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-13949 Signed-off-by: Jesse Whitehouse <[email protected]> * Bump version to 2.2.0 (#66) Signed-off-by: Jesse Whitehouse <[email protected]> * Support Python 3.11 (#60) Signed-off-by: Jesse Whitehouse <[email protected]> * Bump version to 2.2.1 (#70) Signed-off-by: Jesse Whitehouse <[email protected]> * Add none check on _oauth_persistence in DatabricksOAuthProvider (#71) Add none check on _oauth_persistence in DatabricksOAuthProvider to avoid app crash when _oauth_persistence is None. Signed-off-by: Jacky Hu <[email protected]> * Support custom oauth client id and redirect port (#75) * Support custom oauth client id and rediret port range PySQL is used by other tools/CLIs which have own oauth client id, we need to expose oauth_client_id and oauth_redirect_port_range as the connection parameters to support this customization. Signed-off-by: Jacky Hu <[email protected]> * Change oauth redirect port range to port Signed-off-by: Jacky Hu <[email protected]> * Fix type check issue Signed-off-by: Jacky Hu <[email protected]> Signed-off-by: Jacky Hu <[email protected]> * Bump version to 2.2.2 (#76) Signed-off-by: Jacky Hu <[email protected]> Signed-off-by: Jesse <[email protected]> * Merge staging ingestion into main (#78) Follow up to #67 and #64 * Regenerate TCLIService using latest TCLIService.thrift from DBR (#64) * SI: Implement GET, PUT, and REMOVE (#67) * Re-lock dependencies after merging `main` Signed-off-by: Jesse Whitehouse <[email protected]> * Bump version to 2.3.0 and update changelog (#80) Signed-off-by: Jesse Whitehouse <[email protected]> * Add pkgutil-style for the package (#84) Since the package is under databricks namespace. pip install this package will cause issue importing other packages under the same namespace like automl and feature store. Adding pkgutil style to resolve the issue. Signed-off-by: lu-wang-dl <[email protected]> * Add SQLAlchemy Dialect (#57) Signed-off-by: Jesse Whitehouse <[email protected]> * Bump to version 2.4.0(#89) Signed-off-by: Jesse Whitehouse <[email protected]> * Fix syntax in examples in root readme. (#92) Do this because the environment variable pulls did not have closing quotes on their string literals. * Less strict numpy and pyarrow dependencies (#90) Signed-off-by: Thomas Newton <[email protected]> Signed-off-by: Jesse Whitehouse <[email protected]> Co-authored-by: Thomas Newton <[email protected]> * Update example in docstring so query output is valid Spark SQL (#95) Signed-off-by: Jesse Whitehouse <[email protected]> * Bump version to 2.4.1 (#96) Per the sermver.org spec, updating the projects dependencies is considered a compatible change. https: //semver.org/#what-should-i-do-if-i-update-my-own-dependencies-without-changing-the-public-api Signed-off-by: Jesse Whitehouse <[email protected]> * Update CODEOWNERS (#97) * Add Andre to CODEOWNERS (#98) * Add Andre. Signed-off-by: Yunbo Deng <[email protected]> Signed-off-by: yunbodeng-db <[email protected]> * Revert the change temporarily so I can sign off. Signed-off-by: Yunbo Deng <[email protected]> Signed-off-by: yunbodeng-db <[email protected]> * Add Andre and sign off. Signed-off-by: Yunbo Deng <[email protected]> Signed-off-by: yunbodeng-db <[email protected]> * Remove redundant line Signed-off-by: yunbodeng-db <[email protected]> --------- Signed-off-by: yunbodeng-db <[email protected]> * Add external auth provider + example (#101) Signed-off-by: Andre Furlan <[email protected]> Signed-off-by: Jesse Whitehouse <[email protected]> Co-authored-by: Jesse Whitehouse <[email protected]> * Retry on connection timeout (#103) A lot of the time we see the error `[Errno 110] Connection timed out`. This happens a lot in Azure, particularly. In this PR I make it a retryable error as it is safe Signed-off-by: Andre Furlan <[email protected]> * [PECO-244] Make http proxies work (#81) Override thrift's proxy header encoding function. Uses the fix identified in https://github.com/apache/thrift/pull/2565 H/T @pspeter Signed-off-by: Jesse Whitehouse <[email protected]> * Bump to version 2.5.0 (#104) Signed-off-by: Jesse Whitehouse <[email protected]> * Fix changelog release date for version 2.5.0 Signed-off-by: Jesse Whitehouse <[email protected]> * Relax sqlalchemy requirement (#113) * Plus update docs about how to change dependency spec Signed-off-by: Jesse Whitehouse <[email protected]> * Update to version 2.5.1 (#114) Signed-off-by: Jesse Whitehouse <[email protected]> * Fix SQLAlchemy timestamp converter + docs (#117) --------- Signed-off-by: Jesse Whitehouse <[email protected]> * Relax pandas and alembic requirements (#119) Update dependencies for alembic and pandas per customer request Signed-off-by: Jesse Whitehouse <[email protected]> * Bump to version 2.5.2 (#118) Signed-off-by: Jesse Whitehouse <[email protected]> * Use urllib3 for thrift transport + reuse http connections (#131) Signed-off-by: Jesse Whitehouse <[email protected]> * Default socket timeout to 15 min (#137) Signed-off-by: Matthew Kim <[email protected]> * Bump version to 2.6.0 (#139) Signed-off-by: Jesse Whitehouse <[email protected]> * Fix: some thrift RPCs failed with BadStatusLine (#141) --------- Signed-off-by: Jesse Whitehouse <[email protected]> * Bump version to 2.6.1 (#142) Signed-off-by: Jesse Whitehouse <[email protected]> * [ES-706907] Retry GetOperationStatus for http errors (#145) Signed-off-by: Jesse Whitehouse <[email protected]> * Bump version to 2.6.2 (#147) Signed-off-by: Jesse Whitehouse <[email protected]> * [PECO-626] Support OAuth flow for Databricks Azure (#86) ## Summary Support OAuth flow for Databricks Azure ## Background Some OAuth endpoints (e.g. Open ID Configuration) and scopes are different between Databricks Azure and AWS. Current code only supports OAuth flow on Databricks in AWS ## What changes are proposed in this pull request? - Change `OAuthManager` to decouple Databricks AWS specific configuration from OAuth flow - Add `sql/auth/endpoint.py` that implements cloud specific OAuth endpoint configuration - Change `DatabricksOAuthProvider` to work with the OAuth configurations in different Databricks cloud (AWS, Azure) - Add the corresponding unit tests * Use a separate logger for unsafe thrift responses (#153) --------- Signed-off-by: Jesse Whitehouse <[email protected]> * Improve e2e test development ergonomics (#155) --------- Signed-off-by: Jesse Whitehouse <[email protected]> * Don't raise exception when closing a stale Thrift session (#159) Signed-off-by: Jesse Whitehouse <[email protected]> * Bump to version 2.7.0 (#161) Signed-off-by: Jesse Whitehouse <[email protected]> * Cloud Fetch download handler (#127) * Cloud Fetch download handler Signed-off-by: Matthew Kim <[email protected]> * Issue fix: final result link compressed data has multiple LZ4 end-of-frame markers Signed-off-by: Matthew Kim <[email protected]> * Addressing PR comments - Linting - Type annotations - Use response.ok - Log exception - Remove semaphore and only use threading.event - reset() flags method - Fix tests after removing semaphore - Link expiry logic should be in secs - Decompress data static function - link_expiry_buffer and static public methods - Docstrings and comments Signed-off-by: Matthew Kim <[email protected]> * Changing logger.debug to remove url Signed-off-by: Matthew Kim <[email protected]> * _reset() comment to docstring Signed-off-by: Matthew Kim <[email protected]> * link_expiry_buffer -> link_expiry_buffer_secs Signed-off-by: Matthew Kim <[email protected]> --------- Signed-off-by: Matthew Kim <[email protected]> * Cloud Fetch download manager (#146) * Cloud Fetch download manager Signed-off-by: Matthew Kim <[email protected]> * Bug fix: submit handler.run Signed-off-by: Matthew Kim <[email protected]> * Type annotations Signed-off-by: Matthew Kim <[email protected]> * Namedtuple -> dataclass Signed-off-by: Matthew Kim <[email protected]> * Shutdown thread pool and clear handlers Signed-off-by: Matthew Kim <[email protected]> * Docstrings and comments Signed-off-by: Matthew Kim <[email protected]> * handler.run is the correct call Signed-off-by: Matthew Kim <[email protected]> * Link expiry buffer in secs Signed-off-by: Matthew Kim <[email protected]> * Adding type annotations for download_handlers and downloadable_result_settings Signed-off-by: Matthew Kim <[email protected]> * Move DownloadableResultSettings to downloader.py to avoid circular import Signed-off-by: Matthew Kim <[email protected]> * Black linting Signed-off-by: Matthew Kim <[email protected]> * Timeout is never None Signed-off-by: Matthew Kim <[email protected]> --------- Signed-off-by: Matthew Kim <[email protected]> * Cloud fetch queue and integration (#151) * Cloud fetch queue and integration Signed-off-by: Matthew Kim <[email protected]> * Enable cloudfetch with direct results Signed-off-by: Matthew Kim <[email protected]> * Typing and style changes Signed-off-by: Matthew Kim <[email protected]> * Client-settable max_download_threads Signed-off-by: Matthew Kim <[email protected]> * Docstrings and comments Signed-off-by: Matthew Kim <[email protected]> * Increase default buffer size bytes to 104857600 Signed-off-by: Matthew Kim <[email protected]> * Move max_download_threads to kwargs of ThriftBackend, fix unit tests Signed-off-by: Matthew Kim <[email protected]> * Fix tests: staticmethod make_arrow_table mock not callable Signed-off-by: Matthew Kim <[email protected]> * cancel_futures in shutdown() only available in python >=3.9.0 Signed-off-by: Matthew Kim <[email protected]> * Black linting Signed-off-by: Matthew Kim <[email protected]> * Fix typing errors Signed-off-by: Matthew Kim <[email protected]> --------- Signed-off-by: Matthew Kim <[email protected]> * Cloud Fetch e2e tests (#154) * Cloud Fetch e2e tests Signed-off-by: Matthew Kim <[email protected]> * Test case works for e2-dogfood shared unity catalog Signed-off-by: Matthew Kim <[email protected]> * Moving test to LargeQueriesSuite and setting catalog to hive_metastore Signed-off-by: Matthew Kim <[email protected]> * Align default value of buffer_size_bytes in driver tests Signed-off-by: Matthew Kim <[email protected]> * Adding comment to specify what's needed to run successfully Signed-off-by: Matthew Kim <[email protected]> --------- Signed-off-by: Matthew Kim <[email protected]> * Update changelog for cloudfetch (#172) Signed-off-by: Matthew Kim <[email protected]> * Improve sqlalchemy backward compatibility with 1.3.24 (#173) Signed-off-by: Jesse Whitehouse <[email protected]> * OAuth: don't override auth headers with contents of .netrc file (#122) Signed-off-by: Jesse Whitehouse <[email protected]> * Fix proxy connection pool creation (#158) Signed-off-by: Sebastian Eckweiler <[email protected]> Signed-off-by: Jesse Whitehouse <[email protected]> Co-authored-by: Sebastian Eckweiler <[email protected]> Co-authored-by: Jesse Whitehouse <[email protected]> * Relax pandas dependency constraint to allow ^2.0.0 (#164) Signed-off-by: Daniel Segesdi <[email protected]> Signed-off-by: Jesse Whitehouse <[email protected]> Co-authored-by: Jesse Whitehouse <[email protected]> * Use hex string version of operation ID instead of bytes (#170) --------- Signed-off-by: Jesse Whitehouse <[email protected]> * SQLAlchemy: fix has_table so it honours schema= argument (#174) --------- Signed-off-by: Jesse Whitehouse <[email protected]> * Fix socket timeout test (#144) Signed-off-by: Jesse Whitehouse <[email protected]> Co-authored-by: Jesse Whitehouse <[email protected]> * Disable non_native_boolean_check_constraint (#120) --------- Signed-off-by: Bogdan Kyryliuk <[email protected]> Signed-off-by: Jesse Whitehouse <[email protected]> Co-authored-by: Jesse Whitehouse <[email protected]> * Remove unused import for SQLAlchemy 2 compatibility (#128) Signed-off-by: William Gentry <[email protected]> Signed-off-by: Jesse Whitehouse <[email protected]> Co-authored-by: Jesse Whitehouse <[email protected]> * Bump version to 2.8.0 (#178) Signed-off-by: Jesse Whitehouse <[email protected]> * Fix typo in python README quick start example (#186) --------- Co-authored-by: Jesse <[email protected]> * Configure autospec for mocked Client objects (#188) Resolves #187 Signed-off-by: Jesse Whitehouse <[email protected]> * Use urllib3 for retries (#182) Behaviour is gated behind `enable_v3_retries` config. This will be removed and become the default behaviour in a subsequent release. Signed-off-by: Jesse Whitehouse <[email protected]> * Bump version to 2.9.0 (#189) * Add note to changelog about using cloud_fetch Signed-off-by: Jesse Whitehouse <[email protected]> * Explicitly add urllib3 dependency (#191) Signed-off-by: Jacobus Herman <[email protected]> Co-authored-by: Jesse <[email protected]> Signed-off-by: Jesse Whitehouse <[email protected]> * Bump to 2.9.1 (#195) Signed-off-by: Jesse Whitehouse <[email protected]> * Make backwards compatible with urllib3~=1.0 (#197) Signed-off-by: Jesse Whitehouse <[email protected]> * Convenience improvements to v3 retry logic (#199) Signed-off-by: Jesse Whitehouse <[email protected]> * Bump version to 2.9.2 (#201) Signed-off-by: Jesse Whitehouse <[email protected]> * Github Actions Fix: poetry install fails for python 3.7 tests (#208) snok/install-poetry@v1 installs the latest version of Poetry The latest version of poetry released on 20 August 2023 (four days ago as of this commit) which drops support for Python 3.7, causing our github action to fail. Until we complete #207 we need to conditionally install the last version of poetry that supports Python 3.7 (poetry==1.5.1) Signed-off-by: Jesse Whitehouse <[email protected]> * Make backwards compatible with urllib3~=1.0 [Follow up #197] (#206) * Make retry policy backwards compatible with urllib3~=1.0.0 We already implement the equivalent of backoff_max so the behaviour will be the same for urllib3==1.x and urllib3==2.x We do not implement backoff jitter so the behaviour for urllib3==1.x will NOT include backoff jitter whereas urllib3==2.x WILL include jitter. --------- Signed-off-by: Jesse Whitehouse <[email protected]> * Bump version to 2.9.3 (#209) --------- Signed-off-by: Jesse Whitehouse <[email protected]> * Add note to sqlalchemy example: IDENTITY isn't supported yet (#212) ES-842237 Signed-off-by: Jesse Whitehouse <[email protected]> * [PECO-1029] Updated thrift compiler version (#216) * Updated thrift definitions Signed-off-by: nithinkdb <[email protected]> * Tried with a different thrift installation Signed-off-by: nithinkdb <[email protected]> * Reverted TCLI to previous Signed-off-by: nithinkdb <[email protected]> * Reverted to older thrift Signed-off-by: nithinkdb <[email protected]> * Updated version again Signed-off-by: nithinkdb <[email protected]> * Upgraded thrift Signed-off-by: nithinkdb <[email protected]> * Final commit Signed-off-by: nithinkdb <[email protected]> --------- Signed-off-by: nithinkdb <[email protected]> * [PECO-1055] Updated thrift defs to allow Tsparkparameters (#220) Updated thrift defs to most recent versions * Update changelog to indicate that 2.9.1 and 2.9.2 have been yanked. (#222) Signed-off-by: Jesse Whitehouse <[email protected]> * Fix changelog typo: _enable_v3_retries (#225) Closes #219 Signed-off-by: Jesse Whitehouse <[email protected]> * Introduce SQLAlchemy reusable dialog tests (#125) Signed-off-by: Jim Fulton <[email protected]> Co-Authored-By: Jesse Whitehouse <[email protected]> Signed-off-by: Jesse Whitehouse <[email protected]> * [PECO-1026] Add Parameterized Query support to Python (#217) * Initial commit Signed-off-by: nithinkdb <[email protected]> * Added tsparkparam handling Signed-off-by: nithinkdb <[email protected]> * Added basic test Signed-off-by: nithinkdb <[email protected]> * Addressed comments Signed-off-by: nithinkdb <[email protected]> * Addressed missed comments Signed-off-by: nithinkdb <[email protected]> * Resolved comments --------- Signed-off-by: nithinkdb <[email protected]> * Parameterized queries: Add e2e tests for inference (#227) * [PECO-1109] Parameterized Query: add suport for inferring decimal types (#228) Signed-off-by: Jesse Whitehouse <[email protected]> * SQLAlchemy 2: reorganise dialect files into a single directory (#231) Signed-off-by: Jesse Whitehouse <[email protected]> * [PECO-1083] Updated thrift files and added check for protocol version (#229) * Updated thrift files and added check for protocol version Signed-off-by: nithinkdb <[email protected]> * Made error message more clear Signed-off-by: nithinkdb <[email protected]> * Changed name of fn Signed-off-by: nithinkdb <[email protected]> * Ran linter Signed-off-by: nithinkdb <[email protected]> * Update src/databricks/sql/client.py Co-authored-by: Jesse <[email protected]> --------- Signed-off-by: nithinkdb <[email protected]> Co-authored-by: Jesse <[email protected]> * [PECO-840] Port staging ingestion behaviour to new UC Volumes (#235) Signed-off-by: Jesse Whitehouse <[email protected]> * Query parameters: implement support for binding NoneType parameters (#233) Signed-off-by: Jesse Whitehouse <[email protected]> * SQLAlchemy 2: Bump dependency version and update e2e tests for existing behaviour (#236) Signed-off-by: Jesse Whitehouse <[email protected]> * Revert "[PECO-1083] Updated thrift files and added check for protocol version" (#237) Reverts #229 as it causes all of our e2e tests to fail on some versions of DBR. We'll reimplement the protocol version check in a follow-up. This reverts commit 241e934a96737d506c2a1f77c7012e1ab8de967b. * SQLAlchemy 2: add type compilation for all CamelCase types (#238) Signed-off-by: Jesse Whitehouse <[email protected]> * SQLAlchemy 2: add type compilation for uppercase types (#240) Signed-off-by: Jesse Whitehouse <[email protected]> * SQLAlchemy 2: Stop skipping all type tests (#242) Signed-off-by: Jesse Whitehouse <[email protected]> * [PECO-1134] v3 Retries: allow users to bound the number of redirects to follow (#244) Signed-off-by: Jesse Whitehouse <[email protected]> * Parameters: Add type inference for BIGINT and TINYINT types (#246) Signed-off-by: Jesse Whitehouse <[email protected]> * SQLAlchemy 2: Stop skipping some non-type tests (#247) * Stop skipping TableDDLTest and permanent skip HasIndexTest We're now in the territory of features that aren't required for sqla2 compat as of pysql==3.0.0 but we may consider adding this in the future. In this case, table comment reflection needs to be manually implemented. Index reflection would require hooking into the compiler to reflect the partition strategy. test_suite.py::HasIndexTest_databricks+databricks::test_has_index[dialect] SKIPPED (Databricks does not support indexes.) test_suite.py::HasIndexTest_databricks+databricks::test_has_index[inspector] SKIPPED (Databricks does not support indexes.) test_suite.py::HasIndexTest_databricks+databricks::test_has_index_schema[dialect] SKIPPED (Databricks does not support indexes.) test_suite.py::HasIndexTest_databricks+databricks::test_has_index_schema[inspector] SKIPPED (Databricks does not support indexes.) test_suite.py::TableDDLTest_databricks+databricks::test_add_table_comment SKIPPED (Comment reflection is possible but not implemented in this dialect.) test_suite.py::TableDDLTest_databricks+databricks::test_create_index_if_not_exists SKIPPED (Databricks does not support indexes.) test_suite.py::TableDDLTest_databricks+databricks::test_create_table PASSED test_suite.py::TableDDLTest_databricks+databricks::test_create_table_if_not_exists PASSED test_suite.py::TableDDLTest_databricks+databricks::test_create_table_schema PASSED test_suite.py::TableDDLTest_databricks+databricks::test_drop_index_if_exists SKIPPED (Databricks does not support indexes.) test_suite.py::TableDDLTest_databricks+databricks::test_drop_table PASSED test_suite.py::TableDDLTest_databricks+databricks::test_drop_table_comment SKIPPED (Comment reflection is possible but not implemented in this dialect.) test_suite.py::TableDDLTest_databricks+databricks::test_drop_table_if_exists PASSED test_suite.py::TableDDLTest_databricks+databricks::test_underscore_names PASSED Signed-off-by: Jesse Whitehouse <[email protected]> * Permanently skip QuotedNameArgumentTest with comments The fixes to DESCRIBE TABLE and visit_xxx were necessary to get to the point where I could even determine that these tests wouldn't pass. But those changes are not currently tested in the dialect. If, in the course of reviewing the remaining tests in the compliance suite, I find that these visit_xxxx methods are not tested anywhere else then we should extend test_suite.py with our own tests to confirm the behaviour for ourselves. Signed-off-by: Jesse Whitehouse <[email protected]> * Move files from base.py to _ddl.py The presence of this pytest.ini file is _required_ to establish pytest's root_path https://docs.pytest.org/en/7.1.x/reference/customize.html#finding-the-rootdir Without it, the custom pytest plugin from SQLAlchemy can't read the contents of setup.cfg which makes none of the tests runnable. Signed-off-by: Jesse Whitehouse <[email protected]> * Emit a warning for certain constructs Signed-off-by: Jesse Whitehouse <[email protected]> * Stop skipping RowFetchTest Date type work fixed this test failure Signed-off-by: Jesse Whitehouse <[email protected]> * Revise infer_types logic to never infer a TINYINT This allows these SQLAlchemy tests to pass: test_suite.py::FetchLimitOffsetTest_databricks+databricks::test_bound_limit PASSED test_suite.py::FetchLimitOffsetTest_databricks+databricks::test_bound_limit_offset PASSED test_suite.py::FetchLimitOffsetTest_databricks+databricks::test_expr_limit_simple_offset PASSED test_suite.py::FetchLimitOffsetTest_databricks+databricks::test_simple_limit PASSED test_suite.py::FetchLimitOffsetTest_databricks+databricks::test_simple_limit_expr_offset PASSED test_suite.py::FetchLimitOffsetTest_databricks+databricks::test_simple_limit_offset[cases0] PASSED test_suite.py::FetchLimitOffsetTest_databricks+databricks::test_simple_limit_offset[cases1] PASSED test_suite.py::FetchLimitOffsetTest_databricks+databricks::test_simple_limit_offset[cases2] PASSED This partially reverts the change introduced in #246 Signed-off-by: Jesse Whitehouse <[email protected]> * Stop skipping FetchLimitOffsetTest I implemented our custom DatabricksStatementCompiler so we can override the default rendering of unbounded LIMIT clauses from `LIMIT -1` to `LIMIT ALL` We also explicitly skip the FETCH clause tests since Databricks doesn't support this syntax. Blacked all source code here too. Signed-off-by: Jesse Whitehouse <[email protected]> * Stop skipping FutureTableDDLTest Add meaningful skip markers for table comment reflection and indexes Signed-off-by: Jesse Whitehouse <[email protected]> * Stop skipping Identity column tests This closes https://github.com/databricks/databricks-sql-python/issues/175 Signed-off-by: Jesse Whitehouse <[email protected]> * Stop skipping HasTableTest Adding the @reflection.cache decorator to has_table is necessary to pass test_has_table_cache Caching calls to has_table improves the efficiency of the connector Signed-off-by: Jesse Whitehouse <[email protected]> * Permanently skip LongNameBlowoutTest Databricks constraint names are limited to 255 characters Signed-off-by: Jesse Whitehouse <[email protected]> * Stop skipping ExceptionTest Black test_suite.py Signed-off-by: Jesse Whitehouse <[email protected]> * Permanently skip LastrowidTest Signed-off-by: Jesse Whitehouse <[email protected]> * Implement PRIMARY KEY and FOREIGN KEY reflection and enable tests Signed-off-by: Jesse Whitehouse <[email protected]> * Skip all IdentityColumnTest tests Turns out that none of these can pass for the same reason that the first two seemed un-runnable in db6f52bb329f3f43a9215b5cd46b03c3459a302a Signed-off-by: Jesse Whitehouse <[email protected]> --------- Signed-off-by: Jesse Whitehouse <[email protected]> * SQLAlchemy 2: implement and refactor schema reflection methods (#249) Signed-off-by: Jesse Whitehouse <[email protected]> * Add GovCloud domain into AWS domains (#252) Signed-off-by: Jacky Hu <[email protected]> * SQLAlchemy 2: Refactor __init__.py into base.py (#250) Signed-off-by: Jesse Whitehouse <[email protected]> * SQLAlchemy 2: Finish implementing all of ComponentReflectionTest (#251) Signed-off-by: Jesse Whitehouse <[email protected]> * SQLAlchemy 2: Finish marking all tests in the suite (#253) Signed-off-by: Jesse Whitehouse <[email protected]> * SQLAlchemy 2: Finish organising compliance test suite (#256) Signed-off-by: Jesse Whitehouse <[email protected]> * SQLAlchemy 2: Fix failing mypy checks from development (#257) Signed-off-by: Jesse Whitehouse <[email protected]> * Enable cloud fetch by default (#258) Signed-off-by: Jesse Whitehouse <[email protected]> * [PECO-1137] Reintroduce protocol checking to Python test fw (#248) * Put in some unit tests, will add e2e Signed-off-by: nithinkdb <[email protected]> * Added e2e test Signed-off-by: nithinkdb <[email protected]> * Linted Signed-off-by: nithinkdb <[email protected]> * re-bumped thrift files Signed-off-by: nithinkdb <[email protected]> * Changed structure to store protocol version as feature of connection Signed-off-by: nithinkdb <[email protected]> * Fixed parameters test Signed-off-by: nithinkdb <[email protected]> * Fixed comments Signed-off-by: nithinkdb <[email protected]> * Update src/databricks/sql/client.py Co-authored-by: Jesse <[email protected]> Signed-off-by: nithinkdb <[email protected]> * Fixed comments Signed-off-by: nithinkdb <[email protected]> * Removed extra indent Signed-off-by: nithinkdb <[email protected]> --------- Signed-off-by: nithinkdb <[email protected]> Co-authored-by: Jesse <[email protected]> * sqla2 clean-up: make sqlalchemy optional and don't mangle the user-agent (#264) Signed-off-by: Jesse Whitehouse <[email protected]> * SQLAlchemy 2: Add support for TINYINT (#265) Closes #123 Signed-off-by: Jesse Whitehouse <[email protected]> * Add OAuth M2M example (#266) * Add OAuth M2M example Signed-off-by: Jacky Hu <[email protected]> * Native Parameters: reintroduce INLINE approach with tests (#267) Signed-off-by: Jesse Whitehouse <[email protected]> * Document behaviour of executemany (#213) Signed-off-by: Martin Rueckl <[email protected]> * SQLAlchemy 2: Expose TIMESTAMP and TIMESTAMP_NTZ types to users (#268) Signed-off-by: Jesse Whitehouse <[email protected]> * Drop Python 3.7 as a supported version (#270) Signed-off-by: Jesse Whitehouse <[email protected]> (cherry picked from commit 8d85fa8b33a70331141c0c6556196f641d1b8ed5) * GH Workflows: remove Python 3.7 from the matrix for _all_ workflows (#274) Remove Python 3.7 from the matrix for _all_ workflows This was missed in #270 Signed-off-by: Jesse Whitehouse <[email protected]> * Add README and updated example for SQLAlchemy usage (#273) Signed-off-by: Jesse Whitehouse <[email protected]> * Rewrite native parameter implementation with docs and tests (#281) Signed-off-by: Jesse Whitehouse <[email protected]> * Enable v3 retries by default (#282) Signed-off-by: Jesse Whitehouse <[email protected]> * security: bump pyarrow dependency to 14.0.1 (#284) pyarrow is currently compatible with Python 3.8 → Python 3.11 I also removed specifiers for when Python is 3.7 since this no longer applies. Signed-off-by: Jesse Whitehouse <[email protected]> * Bump package version to 3.0.0 (#285) Signed-off-by: Jesse Whitehouse <[email protected]> * Fix docstring about default parameter approach (#287) * [PECO-1286] Add tests for complex types in query results (#293) Signed-off-by: Jesse Whitehouse <[email protected]> * sqlalchemy: fix deprecation warning for dbapi classmethod (#294) Rename `dbapi` classmethod to `import_dbapi` as required by SQLAlchemy 2 Closes #289 Signed-off-by: Jesse Whitehouse <[email protected]> * [PECO-1297] sqlalchemy: fix: can't read columns for tables containing a TIMESTAMP_NTZ column (#296) Signed-off-by: Jesse Whitehouse <[email protected]> * Prepared 3.0.1 release (#297) Signed-off-by: Jesse Whitehouse <[email protected]> * Make contents of `__init__.py` equal across projects (#304) --------- Signed-off-by: Pieter Noordhuis <[email protected]> Signed-off-by: Jesse Whitehouse <[email protected]> Co-authored-by: Jesse Whitehouse <[email protected]> * Fix URI construction in ThriftBackend (#303) Signed-off-by: Jessica <[email protected]> Signed-off-by: Jesse Whitehouse <[email protected]> Co-authored-by: Jesse Whitehouse <[email protected]> * [sqlalchemy] Add table and column comment support (#329) Signed-off-by: Christophe Bornet <[email protected]> Signed-off-by: Jesse Whitehouse <[email protected]> Co-authored-by: Jesse Whitehouse <[email protected]> * Pin pandas and urllib3 versions to fix runtime issues in dbt-databricks (#330) Signed-off-by: Ben Cassell <[email protected]> Signed-off-by: Jesse Whitehouse <[email protected]> Co-authored-by: Jesse Whitehouse <[email protected]> * SQLAlchemy: TINYINT types didn't reflect properly (#315) Signed-off-by: Jesse Whitehouse <[email protected]> * [PECO-1435] Restore `tests.py` to the test suite (#331) --------- Signed-off-by: Jesse Whitehouse <[email protected]> * Bump to version 3.0.2 (#335) Signed-off-by: Jesse Whitehouse <[email protected]> * Update some outdated OAuth comments (#339) Signed-off-by: Jacky Hu <[email protected]> Signed-off-by: Jesse Whitehouse <[email protected]> Co-authored-by: Jesse Whitehouse <[email protected]> * Redact the URL query parameters from the urllib3.connectionpool logs (#341) * Redact the URL query parameters from the urllib3.connectionpool logs Signed-off-by: Mubashir Kazia <[email protected]> * Fix code formatting Signed-off-by: Mubashir Kazia <[email protected]> * Add str check for the log record message arg dict values Signed-off-by: Mubashir Kazia <[email protected]> --------- Signed-off-by: Mubashir Kazia <[email protected]> * Bump to version 3.0.3 (#344) Signed-off-by: Jacky Hu <[email protected]> * [PECO-1411] Support Databricks OAuth on GCP (#338) * [PECO-1411] Support OAuth InHouse on GCP Signed-off-by: Jacky Hu <[email protected]> * Update changelog Signed-off-by: Jesse Whitehouse <[email protected]> --------- Signed-off-by: Jacky Hu <[email protected]> Signed-off-by: Jesse Whitehouse <[email protected]> Co-authored-by: Jesse Whitehouse <[email protected]> * [PECO-1414] Support Databricks native OAuth in Azure (#351) * [PECO-1414] Support Databricks InHouse OAuth in Azure Signed-off-by: Jacky Hu <[email protected]> * Prep for Test Automation (#352) Getting ready for test automation Signed-off-by: Ben Cassell <[email protected]> * Update code owners (#345) * update owners Signed-off-by: yunbodeng-db <[email protected]> * update owners Signed-off-by: yunbodeng-db <[email protected]> * update owners Signed-off-by: yunbodeng-db <[email protected]> --------- Signed-off-by: yunbodeng-db <[email protected]> * Reverting retry behavior on 429s/503s to how it worked in 2.9.3 (#349) Signed-off-by: Ben Cassell <[email protected]> * Bump to version 3.1.0 (#358) Signed-off-by: Jacky Hu <[email protected]> * [PECO-1440] Expose current query id on cursor object (#364) * [PECO-1440] Expose current query id on cursor object Signed-off-by: Levko Kravets <[email protected]> * Clear `active_op_handle` when closing the cursor Signed-off-by: Levko Kravets <[email protected]> --------- Signed-off-by: Levko Kravets <[email protected]> * Add a default for retry after (#371) * Add a default for retry after Signed-off-by: Ben Cassell <[email protected]> * Applied black formatter Signed-off-by: Ben Cassell <[email protected]> * Fix boolean literals (#357) Set supports_native_boolean to True Signed-off-by: Alex Holyoke <[email protected]> * Don't retry network requests that fail with code 403 (#373) * Don't retry requests that fail with 404 Signed-off-by: Jesse Whitehouse <[email protected]> * Fix lint error Signed-off-by: Jesse Whitehouse <[email protected]> --------- Signed-off-by: Jesse Whitehouse <[email protected]> * Bump to 3.1.1 (#374) * bump to 3.1.1 Signed-off-by: Ben Cassell <[email protected]> * Fix cookie setting (#379) * fix cookie setting Signed-off-by: Ben Cassell <[email protected]> * Removing cookie code Signed-off-by: Ben Cassell <[email protected]> --------- Signed-off-by: Ben Cassell <[email protected]> * Fixing a couple type problems: how I would address most of #381 (#382) * Create py.typed Signed-off-by: wyattscarpenter <[email protected]> * add -> Connection annotation Signed-off-by: wyattscarpenter <[email protected]> * massage the code to appease the particular version of the project's mypy deps Signed-off-by: wyattscarpenter <[email protected]> * fix circular import problem Signed-off-by: wyattscarpenter <[email protected]> --------- Signed-off-by: wyattscarpenter <[email protected]> * fix the return types of the classes' __enter__ functions (#384) fix the return types of the classes' __enter__ functions so that the type information is preserved in context managers eg with-as blocks Signed-off-by: wyattscarpenter <[email protected]> * Add Kravets Levko to codeowners (#386) Signed-off-by: Levko Kravets <[email protected]> * Prepare for 3.1.2 (#387) Signed-off-by: Ben Cassell <[email protected]> * Update the proxy authentication (#354) changed authentication for proxy * Fix failing tests (#392) Signed-off-by: Levko Kravets <[email protected]> * Relax `pyarrow` pin (#389) * Relax `pyarrow` pin Signed-off-by: Dave Hirschfeld <[email protected]> * Allow `pyarrow` 16 Signed-off-by: Dave Hirschfeld <[email protected]> * Update `poetry.lock` Signed-off-by: Dave Hirschfeld <[email protected]> --------- Signed-off-by: Dave Hirschfeld <[email protected]> * Fix log error in oauth.py (#269) * Duplicate of applicable change from #93 Signed-off-by: Jesse Whitehouse <[email protected]> * Update changelog Signed-off-by: Jesse Whitehouse <[email protected]> * Fix after merge Signed-off-by: Levko Kravets <[email protected]> --------- Signed-off-by: Jesse Whitehouse <[email protected]> Signed-off-by: Levko Kravets <[email protected]> Co-authored-by: Levko Kravets <[email protected]> * Enable `delta.feature.allowColumnDefaults` for all tables (#343) * Enable `delta.feature.allowColumnDefaults` for all tables * Code style Signed-off-by: Levko Kravets <[email protected]> --------- Signed-off-by: Levko Kravets <[email protected]> Co-authored-by: Levko Kravets <[email protected]> * Fix SQLAlchemy tests (#393) Signed-off-by: Levko Kravets <[email protected]> * Add more debug logging for CloudFetch (#395) Signed-off-by: Levko Kravets <[email protected]> * Update Thrift package (#397) Signed-off-by: Milan Lukac <[email protected]> * Prepare release 3.2.0 (#396) * Prepare release 3.2.0 Signed-off-by: Levko Kravets <[email protected]> * Update changelog Signed-off-by: Levko Kravets <[email protected]> --------- Signed-off-by: Levko Kravets <[email protected]> * move py.typed to correct places (#403) * move py.typed to correct places https://peps.python.org/pep-0561/ says 'For namespace packages (see PEP 420), the py.typed file should be in the submodules of the namespace, to avoid conflicts and for clarity.'. Previously, when I added the py.typed file to this project, https://github.com/databricks/databricks-sql-python/pull/382 , I was unaware this was a namespace package (although, curiously, it seems I had done it right initially and then changed to the wrong way). As PEP 561 warns us, this does create conflicts; other libraries in the databricks namespace package (such as, in my case, databricks-vectorsearch) are then treated as though they are typed, which they are not. This commit moves the py.typed file to the correct places, the submodule folders, fixing that problem. Signed-off-by: wyattscarpenter <[email protected]> * change target of mypy to src/databricks instead of src. I think this might fix the CI code-quality checks failure, but unfortunately I can't replicate that failure locally and the error message is unhelpful Signed-off-by: wyattscarpenter <[email protected]> * Possible workaround for bad error message 'error: --install-types failed (no mypy cache directory)'; see https://github.com/python/mypy/issues/10768#issuecomment-2178450153 Signed-off-by: wyattscarpenter <[email protected]> * fix invalid yaml syntax Signed-off-by: wyattscarpenter <[email protected]> * Best fix (#3) Fixes the problem by cding and supplying a flag to mypy (that mypy needs this flag is seemingly fixed/changed in later versions of mypy; but that's another pr altogether...). Also fixes a type error that was somehow in the arguments of the program (?!) (I guess this is because you guys are still using implicit optional) --------- Signed-off-by: wyattscarpenter <[email protected]> * return the old result_links default (#5) Return the old result_links default, make the type optional, & I'm pretty sure the original problem is that add_file_links can't take a None, so these statements should be in the body of the if-statement that ensures it is not None Signed-off-by: wyattscarpenter <[email protected]> * Update src/databricks/sql/utils.py "self.download_manager is unconditionally used later, so must be created. Looks this part of code is totally not covered with tests 🤔" Co-authored-by: Levko Kravets <[email protected]> Signed-off-by: wyattscarpenter <[email protected]> --------- Signed-off-by: wyattscarpenter <[email protected]> Co-authored-by: Levko Kravets <[email protected]> * Upgrade mypy (#406) * Upgrade mypy This commit removes the flag (and cd step) from https://github.com/databricks/databricks-sql-python/commit/f53aa37a34dc37026d430e71b5e0d1b871bc5ac1 which we added to get mypy to treat namespaces correctly. This was apparently a bug in mypy, or behavior they decided to change. To get the new behavior, we must upgrade mypy. (This also allows us to remove a couple `# type: ignore` comment that are no longer needed.) This commit runs changes the version of mypy and runs `poetry lock`. It also conforms the whitespace of files in this project to the expectations of various tools and standard (namely: removing trailing whitespace as expected by git and enforcing the existence of one and only one newline at the end of a file as expected by unix and github.) It also uses https://github.com/hauntsaninja/no_implicit_optional to automatically upgrade codebase due to a change in mypy behavior. For a similar reason, it also fixes a new type (or otherwise) errors: * "Return type 'Retry' of 'new' incompatible with return type 'DatabricksRetryPolicy' in supertype 'Retry'" * databricks/sql/auth/retry.py:225: error: object has no attribute update [attr-defined] * /test_param_escaper.py:31: DeprecationWarning: invalid escape sequence \) [as it happens, I think it was also wrong for the string not to be raw, because I'm pretty sure it wants all of its backslashed single-quotes to appear literally with the backslashes, which wasn't happening until now] * ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject [this is like a numpy version thing, which I fixed by being stricter about numpy version] --------- Signed-off-by: wyattscarpenter <[email protected]> * Incorporate suggestion. I decided the most expedient way of dealing with this type error was just adding the type ignore comment back in, but with a `[attr-defined]` specifier this time. I mean, otherwise I would have to restructure the code or figure out the proper types for a TypedDict for the dict and I don't think that's worth it at the moment. Signed-off-by: wyattscarpenter <[email protected]> --------- Signed-off-by: wyattscarpenter <[email protected]> * Do not retry failing requests with status code 401 (#408) - Raises NonRecoverableNetworkError when request results in 401 status code Signed-off-by: Tor Hødnebø <[email protected]> Signed-off-by: Tor Hødnebø <[email protected]> * [PECO-1715] Remove username/password (BasicAuth) auth option (#409) Signed-off-by: Jacky Hu <[email protected]> * [PECO-1751] Refactor CloudFetch downloader: handle files sequentially (#405) * [PECO-1751] Refactor CloudFetch downloader: handle files sequentially; utilize Futures Signed-off-by: Levko Kravets <[email protected]> * Retry failed CloudFetch downloads Signed-off-by: Levko Kravets <[email protected]> * Update tests Signed-off-by: Levko Kravets <[email protected]> --------- Signed-off-by: Levko Kravets <[email protected]> * Fix CloudFetch retry policy to be compatible with all `urllib3` versions we support (#412) Signed-off-by: Levko Kravets <[email protected]> * Disable SSL verification for CloudFetch links (#414) * Disable SSL verification for CloudFetch links Signed-off-by: Levko Kravets <[email protected]> * Use existing `_tls_no_verify` option in CloudFetch downloader Signed-off-by: Levko Kravets <[email protected]> * Update tests Signed-off-by: Levko Kravets <[email protected]> --------- Signed-off-by: Levko Kravets <[email protected]> * Prepare relese 3.3.0 (#415) * Prepare relese 3.3.0 Signed-off-by: Levko Kravets <[email protected]> * Remove @arikfr from CODEOWNERS Signed-off-by: Levko Kravets <[email protected]> --------- Signed-off-by: Levko Kravets <[email protected]> * Fix pandas 2.2.2 support (#416) * Support pandas 2.2.2 See release note numpy 2.2.2: https://pandas.pydata.org/docs/dev/whatsnew/v2.2.0.html#to-numpy-for-numpy-nullable-and-arrow-types-converts-to-suitable-numpy-dtype * Allow pandas 2.2.2 in pyproject.toml * Update poetry.lock, poetry lock --no-update * Code style Signed-off-by: Levko Kravets <[email protected]> --------- Signed-off-by: Levko Kravets <[email protected]> Co-authored-by: Levko Kravets <[email protected]> * [PECO-1801] Make OAuth as the default authenticator if no authentication setting is provided (#419) * [PECO-1801] Make OAuth as the default authenticator if no authentication setting is provided Signed-off-by: Jacky Hu <[email protected]> * [PECO-1857] Use SSL options with HTTPS connection pool (#425) * [PECO-1857] Use SSL options with HTTPS connection pool Signed-off-by: Levko Kravets <[email protected]> * Some cleanup Signed-off-by: Levko Kravets <[email protected]> * Resolve circular dependencies Signed-off-by: Levko Kravets <[email protected]> * Update existing tests Signed-off-by: Levko Kravets <[email protected]> * Fix MyPy issues Signed-off-by: Levko Kravets <[email protected]> * Fix `_tls_no_verify` handling Signed-off-by: Levko Kravets <[email protected]> * Add tests Signed-off-by: Levko Kravets <[email protected]> --------- Signed-off-by: Levko Kravets <[email protected]> * Prepare release v3.4.0 (#430) Prepare release 3.4.0 Signed-off-by: Levko Kravets <[email protected]> * [PECO-1926] Create a non pyarrow flow to handle small results for the column set (#440) * Implemented the columnar flow for non arrow users * Minor fixes * Introduced the Column Table structure * Added test for the new column table * Minor fix * Removed unnecessory fikes * [PECO-1961] On non-retryable error, ensure PySQL includes useful information in error (#447) * added error info on non-retryable error * Reformatted all the files using black (#448) Reformatted the files using black * Prepare release v3.5.0 (#457) Prepare release 3.5.0 Signed-off-by: Jacky Hu <[email protected]> * [PECO-2051] Add custom auth headers into cloud fetch request (#460) Signed-off-by: Jacky Hu <[email protected]> * Prepare release 3.6.0 (#461) Signed-off-by: Jacky Hu <[email protected]> * [ PECO - 1768 ] PySQL: adjust HTTP retry logic to align with Go and Nodejs drivers (#467) * Added the exponential backoff code * Added the exponential backoff algorithm and refractored the code * Added jitter and added unit tests * Reformatted * Fixed the test_retry_exponential_backoff integration test * [ PECO-2065 ] Create the async execution flow for the PySQL Connector (#463) * Built the basic flow for the async pipeline - testing is remaining * Implemented the flow for the get_execution_result, but the problem of invalid operation handle still persists * Missed adding some files in previous commit * Working prototype of execute_async, get_query_state and get_execution_result * Added integration tests for execute_async * add docs for functions * Refractored the async code * Fixed java doc * Reformatted * Fix for check_types github action failing (#472) Fixed the chekc_types failing * Remove upper caps on dependencies (#452) * Remove upper caps on numpy and pyarrow versions * Updated the doc to specify native parameters in PUT operation is not supported from >=3.x connector (#477) Added doc update * Incorrect rows in inline fetch result (#479) * Raised error when incorrect Row offset it returned * Changed error type * grammar fix * Added unit tests and modified the code * Updated error message * Updated the non retying to only inline case * Updated fix * Changed the flow * Minor update * Updated the retryable condition * Minor test fix * Added extra space * Bumped up to version 3.7.0 (#482) * bumped up version * Updated to version 3.7.0 * Grammar fix * Minor fix * PySQL Connector split into connector and sqlalchemy (#444) * Modified the gitignore file to not have .idea file * [PECO-1803] Splitting the PySql connector into the core and the non core part (#417) * Implemented ColumnQueue to test the fetchall without pyarrow Removed token removed token * order of fields in row corrected * Changed the folder structure and tested the basic setup to work * Refractored the code to make connector to work * Basic Setup of connector, core and sqlalchemy is working * Basic integration of core, connect and sqlalchemy is working * Setup working dynamic change from ColumnQueue to ArrowQueue * Refractored the test code and moved to respective folders * Added the unit test for column_queue Fixed __version__ Fix * venv_main added to git ignore * Added code for merging columnar table * Merging code for columnar * Fixed the retry_close sesssion test issue with logging * Fixed the databricks_sqlalchemy tests and introduced pytest.ini for the sqla_testing * Added pyarrow_test mark on pytest * Fixed databricks.sqlalchemy to databricks_sqlalchemy imports * Added poetry.lock * Added dist folder * Changed the pyproject.toml * Minor Fix * Added the pyarrow skip tag on unit tests and tested their working * Fixed the Decimal and timestamp conversion issue in non arrow pipeline * Removed not required files and reformatted * Fixed test_retry error * Changed the folder structure to src / databricks * Removed the columnar non arrow flow to another PR * Moved the README to the root * removed columnQueue instance * Revmoved databricks_sqlalchemy dependency in core * Changed the pysql_supports_arrow predicate, introduced changes in the pyproject.toml * Ran the black formatter with the original version * Extra .py removed from all the __init__.py files names * Undo formatting check * Check * Check * Check * Check * Check * Check * Check * Check * Check * Check * Check * Check * Check * Check * BIG UPDATE * Refeactor code * Refractor * Fixed versioning * Minor refractoring * Minor refractoring * Changed the folder structure such that sqlalchemy has not reference here * Fixed README.md and CONTRIBUTING.md * Added manual publish * On push trigger added * Manually setting the publish step * Changed versioning in pyproject.toml * Bumped up the version to 4.0.0.b3 and also changed the structure to have pyarrow as optional * Removed the sqlalchemy tests from integration.yml file * [PECO-1803] Print warning message if pyarrow is not installed (#468) Print warning message if pyarrow is not installed Signed-off-by: Jacky Hu <[email protected]> * [PECO-1803] Remove sqlalchemy and update README.md (#469) Remove sqlalchemy and update README.md Signed-off-by: Jacky Hu <[email protected]> * Removed all sqlalchemy related stuff * generated the lock file * Fixed failing tests * removed poetry.lock * Updated the lock file * Fixed poetry numpy 2.2.2 issue * Workflow fixes --------- Signed-off-by: Jacky Hu <[email protected]> Co-authored-by: Jacky Hu <[email protected]> * Removed CI CD for python3.8 (#490) * Removed python3.8 support * Minor fix * Added CI CD upto python 3.12 (#491) Support for Py till 3.12 * Merging changes from v3.7.1 release (#488) * Increased the number of retry attempts allowed (#486) Updated the number of attempts allowed * bump version to 3.7.1 (#487) bumped up version * Refractore * Minor change * Bumped up to version 4.0.0 (#493) bumped up the version * Updated action's version (#455) Updated actions version. Signed-off-by: Arata Hatori <[email protected]> * Support Python 3.13 and update deps (#510) * Remove upper caps on dependencies (#452) * Remove upper caps on numpy and pyarrow versions Signed-off-by: David Black <[email protected]> * Added CI CD upto python 3.13 Signed-off-by: David Black <[email protected]> * Specify pandas 2.2.3 as the lower bound for python 3.13 Signed-off-by: David Black <[email protected]> * Specify pyarrow 18.0.0 as the lower bound for python 3.13 Signed-off-by: David Black <[email protected]> * Move `numpy` to dev dependencies Signed-off-by: Dave Hirschfeld <[email protected]> * Updated lockfile Signed-off-by: Dave Hirschfeld <[email protected]> --------- Signed-off-by: David Black <[email protected]> Signed-off-by: Dave Hirschfeld <[email protected]> Co-authored-by: David Black <[email protected]> * Improve debugging + fix PR review template (#514) * Improve debugging + add PR review template * case sensitivity of PR template * Forward porting all changes into 4.x.x. uptil v3.7.3 (#529) * Base changes * Black formatter * Cache version fix * Added the changed test_retry.py file * retry_test_mixins changes * Updated the CODEOWNERS (#531) Updated the codeowners * Add version check for urllib3 in backoff calculation (#526) Signed-off-by: Shivam Raj <[email protected]> * [ES-1372353] make user_agent_header part of public API (#530) * make user_agent_header part of public API * removed user_agent_entry from list of internal params * add backward compatibility * Updates runner used to run DCO check to use databricks-protected-runner (#521) * commit 1 Signed-off-by: Madhav Sainanee <[email protected]> * commit 1 Signed-off-by: Madhav Sainanee <[email protected]> * updates runner for dco check Signed-off-by: Madhav Sainanee <[email protected]> * removes contributing file changes Signed-off-by: Madhav Sainanee <[email protected]> --------- Signed-off-by: Madhav Sainanee <[email protected]> * Support multiple timestamp formats in non arrow flow (#533) * Added check for 2 formats * Wrote unit tests * Added more supporting formats * Added the T format datetime * Added more timestamp formats * Added python-dateutil library * prepare release for v4.0.1 (#534) Signed-off-by: Shivam Raj <[email protected]> * Relaxed bound for python-dateutil (#538) Changed bound for python-datetutil * Bumped up the version for 4.0.2 (#539) * Added example for async execute query (#537) Added examples and fixed the async execute not working without pyarrow * Added urllib3 version check (#547) * Added version check * Removed packaging * Bump version to 4.0.3 (#549) Updated the version to 4.0.3 * Cleanup fields as they might be deprecated/removed/change in the future (#553) * Clean thrift files Signed-off-by: Vikrant Puppala <[email protected]> * Refactor decimal conversion in PyArrow tables to use direct casting (#544) This PR replaces the previous implementation of convert_decimals_in_arrow_table() with a more efficient approach that uses PyArrow's native casting operation instead of going through pandas conversion and array creation. - Remove conversion to pandas DataFrame via to_pandas() and apply() methods - Remove intermediate steps of creating array from decimal column and setting it back - Replace with direct type casting using PyArrow's cast() method - Build a new table with transformed columns rather than modifying the original table - Create a new schema based on the modified fields The new approach is more performant by avoiding pandas conversion overhead. The table below highlights substantial performance improvements when retrieving all rows from a table containing decimal columns, particularly when compression is disabled. Even greater gains were observed with compression enabled—showing approximately an 84% improvement (6 seconds compared to 39 seconds). Benchmarking was performed against e2-dogfood, with the client located in the us-west-2 region. ![image](https://github.com/user-attachments/assets/5407b651-8ab6-4c13-b525-cf912f503ba0) Signed-off-by: Jayant Singh <[email protected]> * [PECOBLR-361] convert column table to arrow if arrow present (#551) * Update CODEOWNERS (#562) new codeowners * Enhance Cursor close handling and context manager exception management to prevent server side resource leaks (#554) * Enhance Cursor close handling and context manager exception management * tests * fmt * Fix Cursor.close() to properly handle CursorAlreadyClosedError * Remove specific test message from Cursor.close() error handling * Improve error handling in connection and cursor context managers to ensure proper closure during exceptions, including KeyboardInterrupt. Add tests for nested cursor management and verify operation closure on server-side errors. * add * add * PECOBLR-86 improve logging on python driver (#556) * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <[email protected]> * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <[email protected]> * fixed format Signed-off-by: Sai Shree Pradhan <[email protected]> * used lazy logging Signed-off-by: Sai Shree Pradhan <[email protected]> * changed debug to error logs Signed-off-by: Sai Shree Pradhan <[email protected]> * used lazy logging Signed-off-by: Sai Shree Pradhan <[email protected]> --------- Signed-off-by: Sai Shree Pradhan <[email protected]> * Update github actions run conditions (#569) More conditions to run github actions * Added classes required for telemetry (#572) * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <[email protected]> * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <[email protected]> * fixed format Signed-off-by: Sai Shree Pradhan <[email protected]> * used lazy logging Signed-off-by: Sai Shree Pradhan <[email protected]> * changed debug to error logs Signed-off-by: Sai Shree Pradhan <[email protected]> * added classes required for telemetry Signed-off-by: Sai Shree Pradhan <[email protected]> * removed TelemetryHelper Signed-off-by: Sai Shree Pradhan <[email protected]> * [PECOBLR-361] convert column table to arrow if arrow present (#551) Signed-off-by: Sai Shree Pradhan <[email protected]> * Update CODEOWNERS (#562) new codeowners Signed-off-by: Sai Shree Pradhan <[email protected]> * Enhance Cursor close handling and context manager exception management to prevent server side resource leaks (#554) * Enhance Cursor close handling and context manager exception management * tests * fmt * Fix Cursor.close() to properly handle CursorAlreadyClosedError * Remove specific test message from Cursor.close() error handling * Improve error handling in connection and cursor context managers to ensure proper closure during exceptions, including KeyboardInterrupt. Add tests for nested cursor management and verify operation closure on server-side errors. * add * add Signed-off-by: Sai Shree Pradhan <[email protected]> * PECOBLR-86 improve logging on python driver (#556) * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <[email protected]> * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <[email protected]> * fixed format Signed-off-by: Sai Shree Pradhan <[email protected]> * used lazy logging Signed-off-by: Sai Shree Pradhan <[email protected]> * changed debug to error logs Signed-off-by: Sai Shree Pradhan <[email protected]> * used lazy logging Signed-off-by: Sai Shree Pradhan <[email protected]> --------- Signed-off-by: Sai Shree Pradhan <[email protected]> * Update github actions run conditions (#569) More conditions to run github actions Signed-off-by: Sai Shree Pradhan <[email protected]> * Added classes required for telemetry Signed-off-by: Sai Shree Pradhan <[email protected]> * fixed example Signed-off-by: Sai Shree Pradhan <[email protected]> * chan…

* add close() for Queue, add ResultSet invocation Signed-off-by: varun-edachali-dbx <[email protected]> * move Queue closure to finally: block to ensure client-side cleanup regardless of server side state Signed-off-by: varun-edachali-dbx <[email protected]> * add unit test assertions to ensure Queue closure Signed-off-by: varun-edachali-dbx <[email protected]> * move results closure to try block Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]>

* Separate Session related functionality from Connection class (databricks#571) * decouple session class from existing Connection ensure maintenance of current APIs of Connection while delegating responsibility Signed-off-by: varun-edachali-dbx <[email protected]> * add open property to Connection to ensure maintenance of existing API Signed-off-by: varun-edachali-dbx <[email protected]> * update unit tests to address ThriftBackend through session instead of through Connection Signed-off-by: varun-edachali-dbx <[email protected]> * chore: move session specific tests from test_client to test_session Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) as in CONTRIBUTING.md Signed-off-by: varun-edachali-dbx <[email protected]> * use connection open property instead of long chain through session Signed-off-by: varun-edachali-dbx <[email protected]> * trigger integration workflow Signed-off-by: varun-edachali-dbx <[email protected]> * fix: ensure open attribute of Connection never fails in case the openSession takes long, the initialisation of the session will not complete immediately. This could make the session attribute inaccessible. If the Connection is deleted in this time, the open() check will throw because the session attribute does not exist. Thus, we default to the Connection being closed in this case. This was not an issue before because open was a direct attribute of the Connection class. Caught in the integration tests. Signed-off-by: varun-edachali-dbx <[email protected]> * fix: de-complicate earlier connection open logic earlier, one of the integration tests was failing because 'session was not an attribute of Connection'. This is likely tied to a local configuration issue related to unittest that was causing an error in the test suite itself. The tests are now passing without checking for the session attribute. databricks@c676f9b Signed-off-by: varun-edachali-dbx <[email protected]> * Revert "fix: de-complicate earlier connection open logic" This reverts commit d6b1b19. Signed-off-by: varun-edachali-dbx <[email protected]> * [empty commit] attempt to trigger ci e2e workflow Signed-off-by: varun-edachali-dbx <[email protected]> * Update CODEOWNERS (databricks#562) new codeowners Signed-off-by: varun-edachali-dbx <[email protected]> * Enhance Cursor close handling and context manager exception management to prevent server side resource leaks (databricks#554) * Enhance Cursor close handling and context manager exception management * tests * fmt * Fix Cursor.close() to properly handle CursorAlreadyClosedError * Remove specific test message from Cursor.close() error handling * Improve error handling in connection and cursor context managers to ensure proper closure during exceptions, including KeyboardInterrupt. Add tests for nested cursor management and verify operation closure on server-side errors. * add * add Signed-off-by: varun-edachali-dbx <[email protected]> * PECOBLR-86 improve logging on python driver (databricks#556) * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <[email protected]> * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <[email protected]> * fixed format Signed-off-by: Sai Shree Pradhan <[email protected]> * used lazy logging Signed-off-by: Sai Shree Pradhan <[email protected]> * changed debug to error logs Signed-off-by: Sai Shree Pradhan <[email protected]> * used lazy logging Signed-off-by: Sai Shree Pradhan <[email protected]> --------- Signed-off-by: Sai Shree Pradhan <[email protected]> Signed-off-by: varun-edachali-dbx <[email protected]> * Revert "Merge remote-tracking branch 'upstream/sea-migration' into decouple-session" This reverts commit dbb2ec5, reversing changes made to 7192f11. Signed-off-by: varun-edachali-dbx <[email protected]> * Reapply "Merge remote-tracking branch 'upstream/sea-migration' into decouple-session" This reverts commit bdb8381. Signed-off-by: varun-edachali-dbx <[email protected]> * fix: separate session opening logic from instantiation ensures correctness of self.session.open call in Connection Signed-off-by: varun-edachali-dbx <[email protected]> * fix: use is_open attribute to denote session availability Signed-off-by: varun-edachali-dbx <[email protected]> * fix: access thrift backend through session Signed-off-by: varun-edachali-dbx <[email protected]> * chore: use get_handle() instead of private session attribute in client Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * fix: remove accidentally removed assertions Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> Signed-off-by: Sai Shree Pradhan <[email protected]> Co-authored-by: Jothi Prakash <[email protected]> Co-authored-by: Madhav Sainanee <[email protected]> Co-authored-by: Sai Shree Pradhan <[email protected]> * Introduce Backend Interface (DatabricksClient) (databricks#573) NOTE: the `test_complex_types` e2e test was not working at the time of this merge. The test must be triggered when the test is back up and running as intended. * remove excess logs, assertions, instantiations large merge artifacts Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) + remove excess log (merge artifact) Signed-off-by: varun-edachali-dbx <[email protected]> * fix typing Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary check Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary replace call Signed-off-by: varun-edachali-dbx <[email protected]> * introduce __str__ methods for CommandId and SessionId Signed-off-by: varun-edachali-dbx <[email protected]> * docstrings for DatabricksClient interface Signed-off-by: varun-edachali-dbx <[email protected]> * stronger typing of Cursor and ExecuteResponse Signed-off-by: varun-edachali-dbx <[email protected]> * remove utility functions from backend interface, fix circular import Signed-off-by: varun-edachali-dbx <[email protected]> * rename info to properties Signed-off-by: varun-edachali-dbx <[email protected]> * newline for cleanliness Signed-off-by: varun-edachali-dbx <[email protected]> * fix circular import Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * to_hex_id -> get_hex_id Signed-off-by: varun-edachali-dbx <[email protected]> * better comment on protocol version getter Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * move guid to hex id to new utils module Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * move staging allowed local path to connection props Signed-off-by: varun-edachali-dbx <[email protected]> * add strong return type for execute_command Signed-off-by: varun-edachali-dbx <[email protected]> * skip auth, error handling in databricksclient interface Signed-off-by: varun-edachali-dbx <[email protected]> * chore: docstring + line width Signed-off-by: varun-edachali-dbx <[email protected]> * get_id -> get_guid Signed-off-by: varun-edachali-dbx <[email protected]> * chore: docstring Signed-off-by: varun-edachali-dbx <[email protected]> * fix: to_hex_id -> to_hex_guid Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> * Implement ResultSet Abstraction (backend interfaces for fetch phase) (databricks#574) * ensure backend client returns a ResultSet type in backend tests Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * newline for cleanliness Signed-off-by: varun-edachali-dbx <[email protected]> * fix circular import Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * to_hex_id -> get_hex_id Signed-off-by: varun-edachali-dbx <[email protected]> * better comment on protocol version getter Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * stricter typing for cursor Signed-off-by: varun-edachali-dbx <[email protected]> * correct typing Signed-off-by: varun-edachali-dbx <[email protected]> * correct tests and merge artifacts Signed-off-by: varun-edachali-dbx <[email protected]> * remove accidentally modified workflow files remnants of old merge Signed-off-by: varun-edachali-dbx <[email protected]> * chore: remove accidentally modified workflow files Signed-off-by: varun-edachali-dbx <[email protected]> * add back accidentally removed docstrings Signed-off-by: varun-edachali-dbx <[email protected]> * clean up docstrings Signed-off-by: varun-edachali-dbx <[email protected]> * log hex Signed-off-by: varun-edachali-dbx <[email protected]> * remove unnecessary _replace call Signed-off-by: varun-edachali-dbx <[email protected]> * add __str__ for CommandId Signed-off-by: varun-edachali-dbx <[email protected]> * take TOpenSessionResp in get_protocol_version to maintain existing interface Signed-off-by: varun-edachali-dbx <[email protected]> * active_op_handle -> active_mmand_id Signed-off-by: varun-edachali-dbx <[email protected]> * ensure None returned for close_command Signed-off-by: varun-edachali-dbx <[email protected]> * account for ResultSet return in new pydocs Signed-off-by: varun-edachali-dbx <[email protected]> * pydoc for types Signed-off-by: varun-edachali-dbx <[email protected]> * move common state to ResultSet aprent Signed-off-by: varun-edachali-dbx <[email protected]> * stronger typing in resultSet behaviour Signed-off-by: varun-edachali-dbx <[email protected]> * remove redundant patch in test Signed-off-by: varun-edachali-dbx <[email protected]> * add has_been_closed_server_side assertion Signed-off-by: varun-edachali-dbx <[email protected]> * remove redundancies in tests Signed-off-by: varun-edachali-dbx <[email protected]> * more robust close check Signed-off-by: varun-edachali-dbx <[email protected]> * use normalised state in e2e test Signed-off-by: varun-edachali-dbx <[email protected]> * simplify corrected test Signed-off-by: varun-edachali-dbx <[email protected]> * add line gaps after multi-line pydocs for consistency Signed-off-by: varun-edachali-dbx <[email protected]> * use normalised CommandState type in ExecuteResponse Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary initialisation assertions Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary line break s Signed-off-by: varun-edachali-dbx <[email protected]> * more un-necessary line breaks Signed-off-by: varun-edachali-dbx <[email protected]> * constrain diff of test_closing_connection_closes_commands Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff of test_closing_connection_closes_commands Signed-off-by: varun-edachali-dbx <[email protected]> * use pytest-like assertions for test_closing_connection_closes_commands Signed-off-by: varun-edachali-dbx <[email protected]> * ensure command_id is not None Signed-off-by: varun-edachali-dbx <[email protected]> * line breaks after multi-line pyfocs Signed-off-by: varun-edachali-dbx <[email protected]> * ensure non null operationHandle for commandId creation Signed-off-by: varun-edachali-dbx <[email protected]> * use command_id methods instead of explicit guid_to_hex_id conversion Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary artifacts in test_session, add back assertion Signed-off-by: varun-edachali-dbx <[email protected]> * add from __future__ import annotations to remove string literals around forward refs, remove some unused imports Signed-off-by: varun-edachali-dbx <[email protected]> * move docstring of DatabricksClient within class Signed-off-by: varun-edachali-dbx <[email protected]> * move ThriftResultSet import to top of file Signed-off-by: varun-edachali-dbx <[email protected]> * make backend/utils __init__ file empty Signed-off-by: varun-edachali-dbx <[email protected]> * use from __future__ import annotations to remove string literals around Cursor Signed-off-by: varun-edachali-dbx <[email protected]> * use lazy logging Signed-off-by: varun-edachali-dbx <[email protected]> * replace getters with property tag Signed-off-by: varun-edachali-dbx <[email protected]> * set active_command_id to None, not active_op_handle Signed-off-by: varun-edachali-dbx <[email protected]> * align test_session with pytest instead of unittest Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * remove repetition from Session.__init__ Signed-off-by: varun-edachali-dbx <[email protected]> * mention that if catalog / schema name is None, we fetch across all Signed-off-by: varun-edachali-dbx <[email protected]> * mention fetching across all tables if null table name Signed-off-by: varun-edachali-dbx <[email protected]> * remove lazy import of ThriftResultSet Signed-off-by: varun-edachali-dbx <[email protected]> * remove unused import Signed-off-by: varun-edachali-dbx <[email protected]> * better docstrings Signed-off-by: varun-edachali-dbx <[email protected]> * clarified role of cursor in docstring Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]>

* Minor fix * Perf update * more * test fix

* send telemetry to unauth endpoint in case of connection/auth errors Signed-off-by: Sai Shree Pradhan <[email protected]> * formatting Signed-off-by: Sai Shree Pradhan <[email protected]> * added unit test for send_connection_error_telemetry Signed-off-by: Sai Shree Pradhan <[email protected]> * retry errors Signed-off-by: Sai Shree Pradhan <[email protected]> * Add functionality for export of latency logs via telemetry (databricks#608) * added functionality for export of failure logs Signed-off-by: Sai Shree Pradhan <[email protected]> * changed logger.error to logger.debug in exc.py Signed-off-by: Sai Shree Pradhan <[email protected]> * Fix telemetry loss during Python shutdown Signed-off-by: Sai Shree Pradhan <[email protected]> * unit tests for export_failure_log Signed-off-by: Sai Shree Pradhan <[email protected]> * try-catch blocks to make telemetry failures non-blocking for connector operations Signed-off-by: Sai Shree Pradhan <[email protected]> * removed redundant try/catch blocks, added try/catch block to initialize and get telemetry client Signed-off-by: Sai Shree Pradhan <[email protected]> * skip null fields in telemetry request Signed-off-by: Sai Shree Pradhan <[email protected]> * removed dup import, renamed func, changed a filter_null_values to lamda Signed-off-by: Sai Shree Pradhan <[email protected]> * removed unnecassary class variable and a redundant try/except block Signed-off-by: Sai Shree Pradhan <[email protected]> * public functions defined at interface level Signed-off-by: Sai Shree Pradhan <[email protected]> * changed export_event and flush to private functions Signed-off-by: Sai Shree Pradhan <[email protected]> * formatting Signed-off-by: Sai Shree Pradhan <[email protected]> * changed connection_uuid to thread local in thrift backend Signed-off-by: Sai Shree Pradhan <[email protected]> * made errors more specific Signed-off-by: Sai Shree Pradhan <[email protected]> * revert change to connection_uuid Signed-off-by: Sai Shree Pradhan <[email protected]> * reverting change in close in telemetry client Signed-off-by: Sai Shree Pradhan <[email protected]> * JsonSerializableMixin Signed-off-by: Sai Shree Pradhan <[email protected]> * isdataclass check in JsonSerializableMixin Signed-off-by: Sai Shree Pradhan <[email protected]> * convert TelemetryClientFactory to module-level functions, replace NoopTelemetryClient class with NOOP_TELEMETRY_CLIENT singleton, updated tests accordingly Signed-off-by: Sai Shree Pradhan <[email protected]> * renamed connection_uuid as session_id_hex Signed-off-by: Sai Shree Pradhan <[email protected]> * added NotImplementedError to abstract class, added unit tests Signed-off-by: Sai Shree Pradhan <[email protected]> * formatting Signed-off-by: Sai Shree Pradhan <[email protected]> * added PEP-249 link, changed NoopTelemetryClient implementation Signed-off-by: Sai Shree Pradhan <[email protected]> * removed unused import Signed-off-by: Sai Shree Pradhan <[email protected]> * made telemetry client close a module-level function Signed-off-by: Sai Shree Pradhan <[email protected]> * unit tests verbose Signed-off-by: Sai Shree Pradhan <[email protected]> * debug logs in unit tests Signed-off-by: Sai Shree Pradhan <[email protected]> * debug logs in unit tests Signed-off-by: Sai Shree Pradhan <[email protected]> * removed ABC from mixin, added try/catch block around executor shutdown Signed-off-by: Sai Shree Pradhan <[email protected]> * checking stuff Signed-off-by: Sai Shree Pradhan <[email protected]> * finding out * finding out more * more more finding out more nice * locks are useless anyways * haha * normal * := looks like walrus horizontally * one more * walrus again * old stuff without walrus seems to fail * manually do the walrussing * change 3.13t, v2 Signed-off-by: Sai Shree Pradhan <[email protected]> * formatting, added walrus Signed-off-by: Sai Shree Pradhan <[email protected]> * formatting Signed-off-by: Sai Shree Pradhan <[email protected]> * removed walrus, removed test before stalling test Signed-off-by: Sai Shree Pradhan <[email protected]> * changed order of stalling test Signed-off-by: Sai Shree Pradhan <[email protected]> * removed debugging, added TelemetryClientFactory Signed-off-by: Sai Shree Pradhan <[email protected]> * remove more debugging Signed-off-by: Sai Shree Pradhan <[email protected]> * latency logs funcitionality Signed-off-by: Sai Shree Pradhan <[email protected]> * fixed type of return value in get_session_id_hex() in thrift backend Signed-off-by: Sai Shree Pradhan <[email protected]> * debug on TelemetryClientFactory lock Signed-off-by: Sai Shree Pradhan <[email protected]> * formatting Signed-off-by: Sai Shree Pradhan <[email protected]> * type notation for _waiters Signed-off-by: Sai Shree Pradhan <[email protected]> * called connection.close() in test_arraysize_buffer_size_passthrough Signed-off-by: Sai Shree Pradhan <[email protected]> * run all unit tests Signed-off-by: Sai Shree Pradhan <[email protected]> * more debugging Signed-off-by: Sai Shree Pradhan <[email protected]> * removed the connection.close() from that test, put debug statement before and after TelemetryClientFactory lock Signed-off-by: Sai Shree Pradhan <[email protected]> * more debug Signed-off-by: Sai Shree Pradhan <[email protected]> * more more more Signed-off-by: Sai Shree Pradhan <[email protected]> * why Signed-off-by: Sai Shree Pradhan <[email protected]> * whywhy Signed-off-by: Sai Shree Pradhan <[email protected]> * thread name Signed-off-by: Sai Shree Pradhan <[email protected]> * added teardown to all tests except finalizer test (gc collect) Signed-off-by: Sai Shree Pradhan <[email protected]> * added the get_attribute functions to the classes Signed-off-by: Sai Shree Pradhan <[email protected]> * removed tearDown, added connection.close() to first test Signed-off-by: Sai Shree Pradhan <[email protected]> * finally Signed-off-by: Sai Shree Pradhan <[email protected]> * remove debugging Signed-off-by: Sai Shree Pradhan <[email protected]> * added test for export_latency_log, made mock of thrift backend with retry policy Signed-off-by: Sai Shree Pradhan <[email protected]> * added multi threaded tests Signed-off-by: Sai Shree Pradhan <[email protected]> * formatting Signed-off-by: Sai Shree Pradhan <[email protected]> * added TelemetryExtractor, removed multithreaded tests Signed-off-by: Sai Shree Pradhan <[email protected]> * formatting Signed-off-by: Sai Shree Pradhan <[email protected]> * fixes in test Signed-off-by: Sai Shree Pradhan <[email protected]> * fix in telemetry extractor Signed-off-by: Sai Shree Pradhan <[email protected]> * added doc strings to latency_logger, abstracted export_telemetry_log Signed-off-by: Sai Shree Pradhan <[email protected]> * statement type, unit test fix Signed-off-by: Sai Shree Pradhan <[email protected]> * unit test fix Signed-off-by: Sai Shree Pradhan <[email protected]> * statement type changes Signed-off-by: Sai Shree Pradhan <[email protected]> * test_fetches fix Signed-off-by: Sai Shree Pradhan <[email protected]> * added mocks to resolve the errors caused by log_latency decorator in tests Signed-off-by: Sai Shree Pradhan <[email protected]> * removed function in test_fetches cuz it is only used once Signed-off-by: Sai Shree Pradhan <[email protected]> * added _safe_call which returns None in case of errors in the get functions Signed-off-by: Sai Shree Pradhan <[email protected]> * removed the changes in test_client and test_fetches Signed-off-by: Sai Shree Pradhan <[email protected]> * removed the changes in test_fetches Signed-off-by: Sai Shree Pradhan <[email protected]> * test_telemetry Signed-off-by: Sai Shree Pradhan <[email protected]> * removed test Signed-off-by: Sai Shree Pradhan <[email protected]> --------- Signed-off-by: Sai Shree Pradhan <[email protected]> * MaxRetryDurationError Signed-off-by: Sai Shree Pradhan <[email protected]> * main changes Signed-off-by: Sai Shree Pradhan <[email protected]> * formatting Signed-off-by: Sai Shree Pradhan <[email protected]> * import json Signed-off-by: Sai Shree Pradhan <[email protected]> * without the max retry errors Signed-off-by: Sai Shree Pradhan <[email protected]> * unauth telemetry client Signed-off-by: Sai Shree Pradhan <[email protected]> * remove duplicate code setting telemetry_enabled Signed-off-by: Sai Shree Pradhan <[email protected]> * removed unused errors Signed-off-by: Sai Shree Pradhan <[email protected]> * merge with main changes Signed-off-by: Sai Shree Pradhan <[email protected]> * test Signed-off-by: Sai Shree Pradhan <[email protected]> * without try/catch block Signed-off-by: Sai Shree Pradhan <[email protected]> * - Signed-off-by: Sai Shree Pradhan <[email protected]> * error log for auth provider, ThriftDatabricksClient Signed-off-by: Sai Shree Pradhan <[email protected]> * error log for session.open Signed-off-by: Sai Shree Pradhan <[email protected]> * retry tests fix Signed-off-by: Sai Shree Pradhan <[email protected]> * test connection failure log Signed-off-by: Sai Shree Pradhan <[email protected]> * check types fix Signed-off-by: Sai Shree Pradhan <[email protected]> * test Signed-off-by: Sai Shree Pradhan <[email protected]> * rephrase import Signed-off-by: Sai Shree Pradhan <[email protected]> --------- Signed-off-by: Sai Shree Pradhan <[email protected]>

* Separate Session related functionality from Connection class (#571) * decouple session class from existing Connection ensure maintenance of current APIs of Connection while delegating responsibility Signed-off-by: varun-edachali-dbx <[email protected]> * add open property to Connection to ensure maintenance of existing API Signed-off-by: varun-edachali-dbx <[email protected]> * update unit tests to address ThriftBackend through session instead of through Connection Signed-off-by: varun-edachali-dbx <[email protected]> * chore: move session specific tests from test_client to test_session Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) as in CONTRIBUTING.md Signed-off-by: varun-edachali-dbx <[email protected]> * use connection open property instead of long chain through session Signed-off-by: varun-edachali-dbx <[email protected]> * trigger integration workflow Signed-off-by: varun-edachali-dbx <[email protected]> * fix: ensure open attribute of Connection never fails in case the openSession takes long, the initialisation of the session will not complete immediately. This could make the session attribute inaccessible. If the Connection is deleted in this time, the open() check will throw because the session attribute does not exist. Thus, we default to the Connection being closed in this case. This was not an issue before because open was a direct attribute of the Connection class. Caught in the integration tests. Signed-off-by: varun-edachali-dbx <[email protected]> * fix: de-complicate earlier connection open logic earlier, one of the integration tests was failing because 'session was not an attribute of Connection'. This is likely tied to a local configuration issue related to unittest that was causing an error in the test suite itself. The tests are now passing without checking for the session attribute. https://github.com/databricks/databricks-sql-python/pull/567/commits/c676f9b0281cc3e4fe9c6d8216cc62fc75eade3b Signed-off-by: varun-edachali-dbx <[email protected]> * Revert "fix: de-complicate earlier connection open logic" This reverts commit d6b1b196c98a6e9d8e593a88c34bbde010519ef4. Signed-off-by: varun-edachali-dbx <[email protected]> * [empty commit] attempt to trigger ci e2e workflow Signed-off-by: varun-edachali-dbx <[email protected]> * Update CODEOWNERS (#562) new codeowners Signed-off-by: varun-edachali-dbx <[email protected]> * Enhance Cursor close handling and context manager exception management to prevent server side resource leaks (#554) * Enhance Cursor close handling and context manager exception management * tests * fmt * Fix Cursor.close() to properly handle CursorAlreadyClosedError * Remove specific test message from Cursor.close() error handling * Improve error handling in connection and cursor context managers to ensure proper closure during exceptions, including KeyboardInterrupt. Add tests for nested cursor management and verify operation closure on server-side errors. * add * add Signed-off-by: varun-edachali-dbx <[email protected]> * PECOBLR-86 improve logging on python driver (#556) * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <[email protected]> * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <[email protected]> * fixed format Signed-off-by: Sai Shree Pradhan <[email protected]> * used lazy logging Signed-off-by: Sai Shree Pradhan <[email protected]> * changed debug to error logs Signed-off-by: Sai Shree Pradhan <[email protected]> * used lazy logging Signed-off-by: Sai Shree Pradhan <[email protected]> --------- Signed-off-by: Sai Shree Pradhan <[email protected]> Signed-off-by: varun-edachali-dbx <[email protected]> * Revert "Merge remote-tracking branch 'upstream/sea-migration' into decouple-session" This reverts commit dbb2ec52306b91072a2ee842270c7113aece9aff, reversing changes made to 7192f117279d4f0adcbafcdf2238c18663324515. Signed-off-by: varun-edachali-dbx <[email protected]> * Reapply "Merge remote-tracking branch 'upstream/sea-migration' into decouple-session" This reverts commit bdb83817f49e1d88a01679b11da8e55e8e80b42f. Signed-off-by: varun-edachali-dbx <[email protected]> * fix: separate session opening logic from instantiation ensures correctness of self.session.open call in Connection Signed-off-by: varun-edachali-dbx <[email protected]> * fix: use is_open attribute to denote session availability Signed-off-by: varun-edachali-dbx <[email protected]> * fix: access thrift backend through session Signed-off-by: varun-edachali-dbx <[email protected]> * chore: use get_handle() instead of private session attribute in client Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * fix: remove accidentally removed assertions Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> Signed-off-by: Sai Shree Pradhan <[email protected]> Co-authored-by: Jothi Prakash <[email protected]> Co-authored-by: Madhav Sainanee <[email protected]> Co-authored-by: Sai Shree Pradhan <[email protected]> * Introduce Backend Interface (DatabricksClient) (#573) NOTE: the `test_complex_types` e2e test was not working at the time of this merge. The test must be triggered when the test is back up and running as intended. * remove excess logs, assertions, instantiations large merge artifacts Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) + remove excess log (merge artifact) Signed-off-by: varun-edachali-dbx <[email protected]> * fix typing Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary check Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary replace call Signed-off-by: varun-edachali-dbx <[email protected]> * introduce __str__ methods for CommandId and SessionId Signed-off-by: varun-edachali-dbx <[email protected]> * docstrings for DatabricksClient interface Signed-off-by: varun-edachali-dbx <[email protected]> * stronger typing of Cursor and ExecuteResponse Signed-off-by: varun-edachali-dbx <[email protected]> * remove utility functions from backend interface, fix circular import Signed-off-by: varun-edachali-dbx <[email protected]> * rename info to properties Signed-off-by: varun-edachali-dbx <[email protected]> * newline for cleanliness Signed-off-by: varun-edachali-dbx <[email protected]> * fix circular import Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * to_hex_id -> get_hex_id Signed-off-by: varun-edachali-dbx <[email protected]> * better comment on protocol version getter Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * move guid to hex id to new utils module Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * move staging allowed local path to connection props Signed-off-by: varun-edachali-dbx <[email protected]> * add strong return type for execute_command Signed-off-by: varun-edachali-dbx <[email protected]> * skip auth, error handling in databricksclient interface Signed-off-by: varun-edachali-dbx <[email protected]> * chore: docstring + line width Signed-off-by: varun-edachali-dbx <[email protected]> * get_id -> get_guid Signed-off-by: varun-edachali-dbx <[email protected]> * chore: docstring Signed-off-by: varun-edachali-dbx <[email protected]> * fix: to_hex_id -> to_hex_guid Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> * Implement ResultSet Abstraction (backend interfaces for fetch phase) (#574) * ensure backend client returns a ResultSet type in backend tests Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * newline for cleanliness Signed-off-by: varun-edachali-dbx <[email protected]> * fix circular import Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * to_hex_id -> get_hex_id Signed-off-by: varun-edachali-dbx <[email protected]> * better comment on protocol version getter Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * stricter typing for cursor Signed-off-by: varun-edachali-dbx <[email protected]> * correct typing Signed-off-by: varun-edachali-dbx <[email protected]> * correct tests and merge artifacts Signed-off-by: varun-edachali-dbx <[email protected]> * remove accidentally modified workflow files remnants of old merge Signed-off-by: varun-edachali-dbx <[email protected]> * chore: remove accidentally modified workflow files Signed-off-by: varun-edachali-dbx <[email protected]> * add back accidentally removed docstrings Signed-off-by: varun-edachali-dbx <[email protected]> * clean up docstrings Signed-off-by: varun-edachali-dbx <[email protected]> * log hex Signed-off-by: varun-edachali-dbx <[email protected]> * remove unnecessary _replace call Signed-off-by: varun-edachali-dbx <[email protected]> * add __str__ for CommandId Signed-off-by: varun-edachali-dbx <[email protected]> * take TOpenSessionResp in get_protocol_version to maintain existing interface Signed-off-by: varun-edachali-dbx <[email protected]> * active_op_handle -> active_mmand_id Signed-off-by: varun-edachali-dbx <[email protected]> * ensure None returned for close_command Signed-off-by: varun-edachali-dbx <[email protected]> * account for ResultSet return in new pydocs Signed-off-by: varun-edachali-dbx <[email protected]> * pydoc for types Signed-off-by: varun-edachali-dbx <[email protected]> * move common state to ResultSet aprent Signed-off-by: varun-edachali-dbx <[email protected]> * stronger typing in resultSet behaviour Signed-off-by: varun-edachali-dbx <[email protected]> * remove redundant patch in test Signed-off-by: varun-edachali-dbx <[email protected]> * add has_been_closed_server_side assertion Signed-off-by: varun-edachali-dbx <[email protected]> * remove redundancies in tests Signed-off-by: varun-edachali-dbx <[email protected]> * more robust close check Signed-off-by: varun-edachali-dbx <[email protected]> * use normalised state in e2e test Signed-off-by: varun-edachali-dbx <[email protected]> * simplify corrected test Signed-off-by: varun-edachali-dbx <[email protected]> * add line gaps after multi-line pydocs for consistency Signed-off-by: varun-edachali-dbx <[email protected]> * use normalised CommandState type in ExecuteResponse Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> * Introduce Sea HTTP Client and test script (#583) * introduce http client (temp) and sea test file Signed-off-by: varun-edachali-dbx <[email protected]> * reduce verbosity Signed-off-by: varun-edachali-dbx <[email protected]> * redundant comment Signed-off-by: varun-edachali-dbx <[email protected]> * reduce redundancy, params and data separate Signed-off-by: varun-edachali-dbx <[email protected]> * rename client Signed-off-by: varun-edachali-dbx <[email protected]> * fix type issues Signed-off-by: varun-edachali-dbx <[email protected]> * reduce repetition in request calls Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary elifs Signed-off-by: varun-edachali-dbx <[email protected]> * add newline at EOF Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> * Introduce `SeaDatabricksClient` (Session Implementation) (#582) * [squashed from prev branch] introduce sea client with session open and close functionality Signed-off-by: varun-edachali-dbx <[email protected]> * remove accidental changes to workflows (merge artifacts) Signed-off-by: varun-edachali-dbx <[email protected]> * pass test_input to get_protocol_version instead of session_id to maintain previous API Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black + line gaps after multi-line pydocs) Signed-off-by: varun-edachali-dbx <[email protected]> * use factory for backend instantiation Signed-off-by: varun-edachali-dbx <[email protected]> * fix type issues Signed-off-by: varun-edachali-dbx <[email protected]> * remove redundant comments Signed-off-by: varun-edachali-dbx <[email protected]> * introduce models for requests and responses Signed-off-by: varun-edachali-dbx <[email protected]> * remove http client and test script to prevent diff from showing up post http-client merge Signed-off-by: varun-edachali-dbx <[email protected]> * Introduce Sea HTTP Client and test script (#583) * introduce http client (temp) and sea test file Signed-off-by: varun-edachali-dbx <[email protected]> * reduce verbosity Signed-off-by: varun-edachali-dbx <[email protected]> * redundant comment Signed-off-by: varun-edachali-dbx <[email protected]> * reduce redundancy, params and data separate Signed-off-by: varun-edachali-dbx <[email protected]> * rename client Signed-off-by: varun-edachali-dbx <[email protected]> * fix type issues Signed-off-by: varun-edachali-dbx <[email protected]> * reduce repetition in request calls Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary elifs Signed-off-by: varun-edachali-dbx <[email protected]> * add newline at EOF Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> * CustomHttpClient -> SeaHttpClient Signed-off-by: varun-edachali-dbx <[email protected]> * redundant comment in backend client Signed-off-by: varun-edachali-dbx <[email protected]> * regex for warehouse_id instead of .split, remove excess imports and behaviour Signed-off-by: varun-edachali-dbx <[email protected]> * remove redundant attributes Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * [nit] reduce nested code Signed-off-by: varun-edachali-dbx <[email protected]> * line gap after multi-line pydoc Signed-off-by: varun-edachali-dbx <[email protected]> * redundant imports Signed-off-by: varun-edachali-dbx <[email protected]> * move sea backend and models into separate sea/ dir Signed-off-by: varun-edachali-dbx <[email protected]> * move http client into separate sea/ dir Signed-off-by: varun-edachali-dbx <[email protected]> * change commands to include ones in docs Signed-off-by: varun-edachali-dbx <[email protected]> * add link to sql-ref-parameters for session-confs Signed-off-by: varun-edachali-dbx <[email protected]> * add client side filtering for session confs, add note on warehouses over endoints Signed-off-by: varun-edachali-dbx <[email protected]> * test unimplemented methods and max_download_threads prop Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> * Normalise Execution Response (clean backend interfaces) (#587) * [squash from exec-sea] bring over execution phase changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess test Signed-off-by: varun-edachali-dbx <[email protected]> * add docstring Signed-off-by: varun-edachali-dbx <[email protected]> * remvoe exec func in sea backend Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess files Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess models Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess sea backend tests Signed-off-by: varun-edachali-dbx <[email protected]> * cleanup Signed-off-by: varun-edachali-dbx <[email protected]> * re-introduce get_schema_desc Signed-off-by: varun-edachali-dbx <[email protected]> * remove SeaResultSet Signed-off-by: varun-edachali-dbx <[email protected]> * clean imports and attributes Signed-off-by: varun-edachali-dbx <[email protected]> * pass CommandId to ExecResp Signed-off-by: varun-edachali-dbx <[email protected]> * remove changes in types Signed-off-by: varun-edachali-dbx <[email protected]> * add back essential types (ExecResponse, from_sea_state) Signed-off-by: varun-edachali-dbx <[email protected]> * fix fetch types Signed-off-by: varun-edachali-dbx <[email protected]> * excess imports Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff by maintaining logs Signed-off-by: varun-edachali-dbx <[email protected]> * fix int test types Signed-off-by: varun-edachali-dbx <[email protected]> * move guid_to_hex_id import to utils Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff in guid utils import Signed-off-by: varun-edachali-dbx <[email protected]> * move arrow_schema_bytes back into ExecuteResult Signed-off-by: varun-edachali-dbx <[email protected]> * maintain log Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary assignment Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary tuple response Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-ncessary verbose mocking Signed-off-by: varun-edachali-dbx <[email protected]> * move Queue construction to ResultSert Signed-off-by: varun-edachali-dbx <[email protected]> * move description to List[Tuple] Signed-off-by: varun-edachali-dbx <[email protected]> * frmatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff (remove explicit tuple conversion) Signed-off-by: varun-edachali-dbx <[email protected]> * remove has_more_rows from ExecuteResponse Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary has_more_rows aclc Signed-off-by: varun-edachali-dbx <[email protected]> * default has_more_rows to True Signed-off-by: varun-edachali-dbx <[email protected]> * return has_more_rows from ExecResponse conversion during GetRespMetadata Signed-off-by: varun-edachali-dbx <[email protected]> * remove unnecessary replacement Signed-off-by: varun-edachali-dbx <[email protected]> * better mocked backend naming Signed-off-by: varun-edachali-dbx <[email protected]> * remove has_more_rows test in ExecuteResponse Signed-off-by: varun-edachali-dbx <[email protected]> * introduce replacement of original has_more_rows read test Signed-off-by: varun-edachali-dbx <[email protected]> * call correct method in test_use_arrow_schema Signed-off-by: varun-edachali-dbx <[email protected]> * call correct method in test_fall_back_to_hive_schema Signed-off-by: varun-edachali-dbx <[email protected]> * re-introduce result response read test Signed-off-by: varun-edachali-dbx <[email protected]> * simplify test Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess fetch_results mocks Signed-off-by: varun-edachali-dbx <[email protected]> * more minimal changes to thrift_backend tests Signed-off-by: varun-edachali-dbx <[email protected]> * move back to old table types Signed-off-by: varun-edachali-dbx <[email protected]> * remove outdated arrow_schema_bytes return Signed-off-by: varun-edachali-dbx <[email protected]> * remove duplicate import Signed-off-by: varun-edachali-dbx <[email protected]> * rephrase model docstrings to explicitly denote that they are representations and not used over the wire Signed-off-by: varun-edachali-dbx <[email protected]> * has_more_rows -> is_direct_results Signed-off-by: varun-edachali-dbx <[email protected]> * switch docstring format to align with Connection class Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> * Introduce models for `SeaDatabricksClient` (#595) * [squash from exec-sea] bring over execution phase changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess test Signed-off-by: varun-edachali-dbx <[email protected]> * add docstring Signed-off-by: varun-edachali-dbx <[email protected]> * remvoe exec func in sea backend Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess files Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess models Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess sea backend tests Signed-off-by: varun-edachali-dbx <[email protected]> * cleanup Signed-off-by: varun-edachali-dbx <[email protected]> * re-introduce get_schema_desc Signed-off-by: varun-edachali-dbx <[email protected]> * remove SeaResultSet Signed-off-by: varun-edachali-dbx <[email protected]> * clean imports and attributes Signed-off-by: varun-edachali-dbx <[email protected]> * pass CommandId to ExecResp Signed-off-by: varun-edachali-dbx <[email protected]> * remove changes in types Signed-off-by: varun-edachali-dbx <[email protected]> * add back essential types (ExecResponse, from_sea_state) Signed-off-by: varun-edachali-dbx <[email protected]> * fix fetch types Signed-off-by: varun-edachali-dbx <[email protected]> * excess imports Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff by maintaining logs Signed-off-by: varun-edachali-dbx <[email protected]> * fix int test types Signed-off-by: varun-edachali-dbx <[email protected]> * [squashed from exec-sea] init execution func Signed-off-by: varun-edachali-dbx <[email protected]> * remove irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove ResultSetFilter functionality Signed-off-by: varun-edachali-dbx <[email protected]> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * even more irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove sea response as init option Signed-off-by: varun-edachali-dbx <[email protected]> * exec test example scripts Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * [squashed from sea-exec] merge sea stuffs Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess removed docstring Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess changes in backend Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess imports Signed-off-by: varun-edachali-dbx <[email protected]> * remove accidentally removed _get_schema_desc Signed-off-by: varun-edachali-dbx <[email protected]> * remove unnecessary init with sea_response tests Signed-off-by: varun-edachali-dbx <[email protected]> * rmeove unnecessary changes Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * improved models and filters from cloudfetch-sea branch Signed-off-by: varun-edachali-dbx <[email protected]> * filters stuff (align with JDBC) Signed-off-by: varun-edachali-dbx <[email protected]> * backend from cloudfetch-sea Signed-off-by: varun-edachali-dbx <[email protected]> * remove filtering, metadata ops Signed-off-by: varun-edachali-dbx <[email protected]> * raise NotImplementedErrror for metadata ops Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary backend cahnges Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-needed GetChunksResponse only relevant in Fetch phase Signed-off-by: varun-edachali-dbx <[email protected]> * reduce code duplication Signed-off-by: varun-edachali-dbx <[email protected]> * more clear docstrings Signed-off-by: varun-edachali-dbx <[email protected]> * introduce strongly typed ChunkInfo Signed-off-by: varun-edachali-dbx <[email protected]> * remove is_volume_operation from response Signed-off-by: varun-edachali-dbx <[email protected]> * add is_volume_op and more ResultData fields Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> * Introduce preliminary SEA Result Set (#588) * [squash from exec-sea] bring over execution phase changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess test Signed-off-by: varun-edachali-dbx <[email protected]> * add docstring Signed-off-by: varun-edachali-dbx <[email protected]> * remvoe exec func in sea backend Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess files Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess models Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess sea backend tests Signed-off-by: varun-edachali-dbx <[email protected]> * cleanup Signed-off-by: varun-edachali-dbx <[email protected]> * re-introduce get_schema_desc Signed-off-by: varun-edachali-dbx <[email protected]> * remove SeaResultSet Signed-off-by: varun-edachali-dbx <[email protected]> * clean imports and attributes Signed-off-by: varun-edachali-dbx <[email protected]> * pass CommandId to ExecResp Signed-off-by: varun-edachali-dbx <[email protected]> * remove changes in types Signed-off-by: varun-edachali-dbx <[email protected]> * add back essential types (ExecResponse, from_sea_state) Signed-off-by: varun-edachali-dbx <[email protected]> * fix fetch types Signed-off-by: varun-edachali-dbx <[email protected]> * excess imports Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff by maintaining logs Signed-off-by: varun-edachali-dbx <[email protected]> * fix int test types Signed-off-by: varun-edachali-dbx <[email protected]> * [squashed from exec-sea] init execution func Signed-off-by: varun-edachali-dbx <[email protected]> * remove irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove ResultSetFilter functionality Signed-off-by: varun-edachali-dbx <[email protected]> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * even more irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove sea response as init option Signed-off-by: varun-edachali-dbx <[email protected]> * move guid_to_hex_id import to utils Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff in guid utils import Signed-off-by: varun-edachali-dbx <[email protected]> * move arrow_schema_bytes back into ExecuteResult Signed-off-by: varun-edachali-dbx <[email protected]> * maintain log Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary assignment Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary tuple response Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-ncessary verbose mocking Signed-off-by: varun-edachali-dbx <[email protected]> * move Queue construction to ResultSert Signed-off-by: varun-edachali-dbx <[email protected]> * move description to List[Tuple] Signed-off-by: varun-edachali-dbx <[email protected]> * frmatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff (remove explicit tuple conversion) Signed-off-by: varun-edachali-dbx <[email protected]> * remove has_more_rows from ExecuteResponse Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary has_more_rows aclc Signed-off-by: varun-edachali-dbx <[email protected]> * default has_more_rows to True Signed-off-by: varun-edachali-dbx <[email protected]> * return has_more_rows from ExecResponse conversion during GetRespMetadata Signed-off-by: varun-edachali-dbx <[email protected]> * remove unnecessary replacement Signed-off-by: varun-edachali-dbx <[email protected]> * better mocked backend naming Signed-off-by: varun-edachali-dbx <[email protected]> * remove has_more_rows test in ExecuteResponse Signed-off-by: varun-edachali-dbx <[email protected]> * introduce replacement of original has_more_rows read test Signed-off-by: varun-edachali-dbx <[email protected]> * call correct method in test_use_arrow_schema Signed-off-by: varun-edachali-dbx <[email protected]> * call correct method in test_fall_back_to_hive_schema Signed-off-by: varun-edachali-dbx <[email protected]> * re-introduce result response read test Signed-off-by: varun-edachali-dbx <[email protected]> * simplify test Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess fetch_results mocks Signed-off-by: varun-edachali-dbx <[email protected]> * more minimal changes to thrift_backend tests Signed-off-by: varun-edachali-dbx <[email protected]> * move back to old table types Signed-off-by: varun-edachali-dbx <[email protected]> * remove outdated arrow_schema_bytes return Signed-off-by: varun-edachali-dbx <[email protected]> * align SeaResultSet with new structure Signed-off-by: varun-edachali-dbx <[email protected]> * correct sea res set tests Signed-off-by: varun-edachali-dbx <[email protected]> * remove duplicate import Signed-off-by: varun-edachali-dbx <[email protected]> * rephrase model docstrings to explicitly denote that they are representations and not used over the wire Signed-off-by: varun-edachali-dbx <[email protected]> * has_more_rows -> is_direct_results Signed-off-by: varun-edachali-dbx <[email protected]> * switch docstring format to align with Connection class Signed-off-by: varun-edachali-dbx <[email protected]> * has_more_rows -> is_direct_results Signed-off-by: varun-edachali-dbx <[email protected]> * fix type errors with arrow_schema_bytes Signed-off-by: varun-edachali-dbx <[email protected]> * spaces after multi line pydocs Signed-off-by: varun-edachali-dbx <[email protected]> * remove duplicate queue init (merge artifact) Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff (remove newlines) Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary changes covered by #588 anyway Signed-off-by: varun-edachali-dbx <[email protected]> * Revert "remove un-necessary changes" This reverts commit a70a6cee277db44d6951604e890f91cae9f92f32. Signed-off-by: varun-edachali-dbx <[email protected]> * b"" -> None Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> * remove invalid ExecuteResponse import Signed-off-by: varun-edachali-dbx <[email protected]> * Separate Session related functionality from Connection class (#571) * decouple session class from existing Connection ensure maintenance of current APIs of Connection while delegating responsibility Signed-off-by: varun-edachali-dbx <[email protected]> * add open property to Connection to ensure maintenance of existing API Signed-off-by: varun-edachali-dbx <[email protected]> * update unit tests to address ThriftBackend through session instead of through Connection Signed-off-by: varun-edachali-dbx <[email protected]> * chore: move session specific tests from test_client to test_session Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) as in CONTRIBUTING.md Signed-off-by: varun-edachali-dbx <[email protected]> * use connection open property instead of long chain through session Signed-off-by: varun-edachali-dbx <[email protected]> * trigger integration workflow Signed-off-by: varun-edachali-dbx <[email protected]> * fix: ensure open attribute of Connection never fails in case the openSession takes long, the initialisation of the session will not complete immediately. This could make the session attribute inaccessible. If the Connection is deleted in this time, the open() check will throw because the session attribute does not exist. Thus, we default to the Connection being closed in this case. This was not an issue before because open was a direct attribute of the Connection class. Caught in the integration tests. Signed-off-by: varun-edachali-dbx <[email protected]> * fix: de-complicate earlier connection open logic earlier, one of the integration tests was failing because 'session was not an attribute of Connection'. This is likely tied to a local configuration issue related to unittest that was causing an error in the test suite itself. The tests are now passing without checking for the session attribute. https://github.com/databricks/databricks-sql-python/pull/567/commits/c676f9b0281cc3e4fe9c6d8216cc62fc75eade3b Signed-off-by: varun-edachali-dbx <[email protected]> * Revert "fix: de-complicate earlier connection open logic" This reverts commit d6b1b196c98a6e9d8e593a88c34bbde010519ef4. Signed-off-by: varun-edachali-dbx <[email protected]> * [empty commit] attempt to trigger ci e2e workflow Signed-off-by: varun-edachali-dbx <[email protected]> * Update CODEOWNERS (#562) new codeowners Signed-off-by: varun-edachali-dbx <[email protected]> * Enhance Cursor close handling and context manager exception management to prevent server side resource leaks (#554) * Enhance Cursor close handling and context manager exception management * tests * fmt * Fix Cursor.close() to properly handle CursorAlreadyClosedError * Remove specific test message from Cursor.close() error handling * Improve error handling in connection and cursor context managers to ensure proper closure during exceptions, including KeyboardInterrupt. Add tests for nested cursor management and verify operation closure on server-side errors. * add * add Signed-off-by: varun-edachali-dbx <[email protected]> * PECOBLR-86 improve logging on python driver (#556) * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <[email protected]> * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <[email protected]> * fixed format Signed-off-by: Sai Shree Pradhan <[email protected]> * used lazy logging Signed-off-by: Sai Shree Pradhan <[email protected]> * changed debug to error logs Signed-off-by: Sai Shree Pradhan <[email protected]> * used lazy logging Signed-off-by: Sai Shree Pradhan <[email protected]> --------- Signed-off-by: Sai Shree Pradhan <[email protected]> Signed-off-by: varun-edachali-dbx <[email protected]> * Revert "Merge remote-tracking branch 'upstream/sea-migration' into decouple-session" This reverts commit dbb2ec52306b91072a2ee842270c7113aece9aff, reversing changes made to 7192f117279d4f0adcbafcdf2238c18663324515. Signed-off-by: varun-edachali-dbx <[email protected]> * Reapply "Merge remote-tracking branch 'upstream/sea-migration' into decouple-session" This reverts commit bdb83817f49e1d88a01679b11da8e55e8e80b42f. Signed-off-by: varun-edachali-dbx <[email protected]> * fix: separate session opening logic from instantiation ensures correctness of self.session.open call in Connection Signed-off-by: varun-edachali-dbx <[email protected]> * fix: use is_open attribute to denote session availability Signed-off-by: varun-edachali-dbx <[email protected]> * fix: access thrift backend through session Signed-off-by: varun-edachali-dbx <[email protected]> * chore: use get_handle() instead of private session attribute in client Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * fix: remove accidentally removed assertions Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> Signed-off-by: Sai Shree Pradhan <[email protected]> Co-authored-by: Jothi Prakash <[email protected]> Co-authored-by: Madhav Sainanee <[email protected]> Co-authored-by: Sai Shree Pradhan <[email protected]> * Introduce Backend Interface (DatabricksClient) (#573) NOTE: the `test_complex_types` e2e test was not working at the time of this merge. The test must be triggered when the test is back up and running as intended. * remove excess logs, assertions, instantiations large merge artifacts Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) + remove excess log (merge artifact) Signed-off-by: varun-edachali-dbx <[email protected]> * fix typing Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary check Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary replace call Signed-off-by: varun-edachali-dbx <[email protected]> * introduce __str__ methods for CommandId and SessionId Signed-off-by: varun-edachali-dbx <[email protected]> * docstrings for DatabricksClient interface Signed-off-by: varun-edachali-dbx <[email protected]> * stronger typing of Cursor and ExecuteResponse Signed-off-by: varun-edachali-dbx <[email protected]> * remove utility functions from backend interface, fix circular import Signed-off-by: varun-edachali-dbx <[email protected]> * rename info to properties Signed-off-by: varun-edachali-dbx <[email protected]> * newline for cleanliness Signed-off-by: varun-edachali-dbx <[email protected]> * fix circular import Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * to_hex_id -> get_hex_id Signed-off-by: varun-edachali-dbx <[email protected]> * better comment on protocol version getter Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * move guid to hex id to new utils module Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * move staging allowed local path to connection props Signed-off-by: varun-edachali-dbx <[email protected]> * add strong return type for execute_command Signed-off-by: varun-edachali-dbx <[email protected]> * skip auth, error handling in databricksclient interface Signed-off-by: varun-edachali-dbx <[email protected]> * chore: docstring + line width Signed-off-by: varun-edachali-dbx <[email protected]> * get_id -> get_guid Signed-off-by: varun-edachali-dbx <[email protected]> * chore: docstring Signed-off-by: varun-edachali-dbx <[email protected]> * fix: to_hex_id -> to_hex_guid Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> * Implement ResultSet Abstraction (backend interfaces for fetch phase) (#574) * ensure backend client returns a ResultSet type in backend tests Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * newline for cleanliness Signed-off-by: varun-edachali-dbx <[email protected]> * fix circular import Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * to_hex_id -> get_hex_id Signed-off-by: varun-edachali-dbx <[email protected]> * better comment on protocol version getter Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * stricter typing for cursor Signed-off-by: varun-edachali-dbx <[email protected]> * correct typing Signed-off-by: varun-edachali-dbx <[email protected]> * correct tests and merge artifacts Signed-off-by: varun-edachali-dbx <[email protected]> * remove accidentally modified workflow files remnants of old merge Signed-off-by: varun-edachali-dbx <[email protected]> * chore: remove accidentally modified workflow files Signed-off-by: varun-edachali-dbx <[email protected]> * add back accidentally removed docstrings Signed-off-by: varun-edachali-dbx <[email protected]> * clean up docstrings Signed-off-by: varun-edachali-dbx <[email protected]> * log hex Signed-off-by: varun-edachali-dbx <[email protected]> * remove unnecessary _replace call Signed-off-by: varun-edachali-dbx <[email protected]> * add __str__ for CommandId Signed-off-by: varun-edachali-dbx <[email protected]> * take TOpenSessionResp in get_protocol_version to maintain existing interface Signed-off-by: varun-edachali-dbx <[email protected]> * active_op_handle -> active_mmand_id Signed-off-by: varun-edachali-dbx <[email protected]> * ensure None returned for close_command Signed-off-by: varun-edachali-dbx <[email protected]> * account for ResultSet return in new pydocs Signed-off-by: varun-edachali-dbx <[email protected]> * pydoc for types Signed-off-by: varun-edachali-dbx <[email protected]> * move common state to ResultSet aprent Signed-off-by: varun-edachali-dbx <[email protected]> * stronger typing in resultSet behaviour Signed-off-by: varun-edachali-dbx <[email protected]> * remove redundant patch in test Signed-off-by: varun-edachali-dbx <[email protected]> * add has_been_closed_server_side assertion Signed-off-by: varun-edachali-dbx <[email protected]> * remove redundancies in tests Signed-off-by: varun-edachali-dbx <[email protected]> * more robust close check Signed-off-by: varun-edachali-dbx <[email protected]> * use normalised state in e2e test Signed-off-by: varun-edachali-dbx <[email protected]> * simplify corrected test Signed-off-by: varun-edachali-dbx <[email protected]> * add line gaps after multi-line pydocs for consistency Signed-off-by: varun-edachali-dbx <[email protected]> * use normalised CommandState type in ExecuteResponse Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary initialisation assertions Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary line break s Signed-off-by: varun-edachali-dbx <[email protected]> * more un-necessary line breaks Signed-off-by: varun-edachali-dbx <[email protected]> * constrain diff of test_closing_connection_closes_commands Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff of test_closing_connection_closes_commands Signed-off-by: varun-edachali-dbx <[email protected]> * use pytest-like assertions for test_closing_connection_closes_commands Signed-off-by: varun-edachali-dbx <[email protected]> * ensure command_id is not None Signed-off-by: varun-edachali-dbx <[email protected]> * line breaks after multi-line pyfocs Signed-off-by: varun-edachali-dbx <[email protected]> * ensure non null operationHandle for commandId creation Signed-off-by: varun-edachali-dbx <[email protected]> * use command_id methods instead of explicit guid_to_hex_id conversion Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary artifacts in test_session, add back assertion Signed-off-by: varun-edachali-dbx <[email protected]> * Implement SeaDatabricksClient (Complete Execution Spec) (#590) * [squash from exec-sea] bring over execution phase changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess test Signed-off-by: varun-edachali-dbx <[email protected]> * add docstring Signed-off-by: varun-edachali-dbx <[email protected]> * remvoe exec func in sea backend Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess files Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess models Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess sea backend tests Signed-off-by: varun-edachali-dbx <[email protected]> * cleanup Signed-off-by: varun-edachali-dbx <[email protected]> * re-introduce get_schema_desc Signed-off-by: varun-edachali-dbx <[email protected]> * remove SeaResultSet Signed-off-by: varun-edachali-dbx <[email protected]> * clean imports and attributes Signed-off-by: varun-edachali-dbx <[email protected]> * pass CommandId to ExecResp Signed-off-by: varun-edachali-dbx <[email protected]> * remove changes in types Signed-off-by: varun-edachali-dbx <[email protected]> * add back essential types (ExecResponse, from_sea_state) Signed-off-by: varun-edachali-dbx <[email protected]> * fix fetch types Signed-off-by: varun-edachali-dbx <[email protected]> * excess imports Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff by maintaining logs Signed-off-by: varun-edachali-dbx <[email protected]> * fix int test types Signed-off-by: varun-edachali-dbx <[email protected]> * [squashed from exec-sea] init execution func Signed-off-by: varun-edachali-dbx <[email protected]> * remove irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove ResultSetFilter functionality Signed-off-by: varun-edachali-dbx <[email protected]> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * even more irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove sea response as init option Signed-off-by: varun-edachali-dbx <[email protected]> * exec test example scripts Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * [squashed from sea-exec] merge sea stuffs Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess removed docstring Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess changes in backend Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess imports Signed-off-by: varun-edachali-dbx <[email protected]> * remove accidentally removed _get_schema_desc Signed-off-by: varun-edachali-dbx <[email protected]> * remove unnecessary init with sea_response tests Signed-off-by: varun-edachali-dbx <[email protected]> * rmeove unnecessary changes Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * improved models and filters from cloudfetch-sea branch Signed-off-by: varun-edachali-dbx <[email protected]> * filters stuff (align with JDBC) Signed-off-by: varun-edachali-dbx <[email protected]> * backend from cloudfetch-sea Signed-off-by: varun-edachali-dbx <[email protected]> * remove filtering, metadata ops Signed-off-by: varun-edachali-dbx <[email protected]> * raise NotImplementedErrror for metadata ops Signed-off-by: varun-edachali-dbx <[email protected]> * change to valid table name Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary changes covered by #588 Signed-off-by: varun-edachali-dbx <[email protected]> * simplify test module Signed-off-by: varun-edachali-dbx <[email protected]> * logging -> debug level Signed-off-by: varun-edachali-dbx <[email protected]> * change table name in log Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary backend cahnges Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-needed GetChunksResponse Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-needed GetChunksResponse only relevant in Fetch phase Signed-off-by: varun-edachali-dbx <[email protected]> * reduce code duplication in response parsing Signed-off-by: varun-edachali-dbx <[email protected]> * reduce code duplication Signed-off-by: varun-edachali-dbx <[email protected]> * more clear docstrings Signed-off-by: varun-edachali-dbx <[email protected]> * introduce strongly typed ChunkInfo Signed-off-by: varun-edachali-dbx <[email protected]> * remove is_volume_operation from response Signed-off-by: varun-edachali-dbx <[email protected]> * add is_volume_op and more ResultData fields Signed-off-by: varun-edachali-dbx <[email protected]> * add test scripts Signed-off-by: varun-edachali-dbx <[email protected]> * Revert "Merge branch 'exec-models-sea' into exec-phase-sea" This reverts commit be1997e0d6b6cf0f5499db2381971ec3a015a2f7, reversing changes made to 37813ba6d1fe06d7f9f10d510a059b88dc552496. * change logging level Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove _get_schema_bytes (for now) Signed-off-by: varun-edachali-dbx <[email protected]> * redundant comments Signed-off-by: varun-edachali-dbx <[email protected]> * remove fetch phase methods Signed-off-by: varun-edachali-dbx <[email protected]> * reduce code repetititon + introduce gaps after multi line pydocs Signed-off-by: varun-edachali-dbx <[email protected]> * remove unused imports Signed-off-by: varun-edachali-dbx <[email protected]> * move description extraction to helper func Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * add more unit tests Signed-off-by: varun-edachali-dbx <[email protected]> * streamline unit tests Signed-off-by: varun-edachali-dbx <[email protected]> * test getting the list of allowed configurations Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff Signed-off-by: varun-edachali-dbx <[email protected]> * house constants in enums for readability and immutability Signed-off-by: varun-edachali-dbx <[email protected]> * add note on hybrid disposition Signed-off-by: varun-edachali-dbx <[email protected]> * remove redundant note on arrow_schema_bytes Signed-off-by: varun-edachali-dbx <[email protected]> * remove invalid import Signed-off-by: varun-edachali-dbx <[email protected]> * add strong typing for manifest in _extract_description Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary column skipping Signed-off-by: varun-edachali-dbx <[email protected]> * remove parsing in backend Signed-off-by: varun-edachali-dbx <[email protected]> * fix: convert sea statement id to CommandId type Signed-off-by: varun-edachali-dbx <[email protected]> * make polling interval a separate constant Signed-off-by: varun-edachali-dbx <[email protected]> * align state checking with Thrift implementation Signed-off-by: varun-edachali-dbx <[email protected]> * update unit tests according to changes Signed-off-by: varun-edachali-dbx <[email protected]> * add unit tests for added methods Signed-off-by: varun-edachali-dbx <[email protected]> * add spec to description extraction docstring, add strong typing to params Signed-off-by: varun-edachali-dbx <[email protected]> * add strong typing for backend parameters arg Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> * add from __future__ import annotations to remove string literals around forward refs, remove some unused imports Signed-off-by: varun-edachali-dbx <[email protected]> * move docstring of DatabricksClient within class Signed-off-by: varun-edachali-dbx <[email protected]> * move ThriftResultSet import to top of file Signed-off-by: varun-edachali-dbx <[email protected]> * make backend/utils __init__ file empty Signed-off-by: varun-edachali-dbx <[email protected]> * use from __future__ import annotations to remove string literals around Cursor Signed-off-by: varun-edachali-dbx <[email protected]> * use lazy logging Signed-off-by: varun-edachali-dbx <[email protected]> * replace getters with property tag Signed-off-by: varun-edachali-dbx <[email protected]> * set active_command_id to None, not active_op_handle Signed-off-by: varun-edachali-dbx <[email protected]> * align test_session with pytest instead of unittest Signed-off-by: varun-edachali-dbx <[email protected]> * remove duplicate test, correct active_command_id attribute Signed-off-by: varun-edachali-dbx <[email protected]> * SeaDatabricksClient: Add Metadata Commands (#593) * [squash from exec-sea] bring over execution phase changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess test Signed-off-by: varun-edachali-dbx <[email protected]> * add docstring Signed-off-by: varun-edachali-dbx <[email protected]> * remvoe exec func in sea backend Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess files Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess models Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess sea backend tests Signed-off-by: varun-edachali-dbx <[email protected]> * cleanup Signed-off-by: varun-edachali-dbx <[email protected]> * re-introduce get_schema_desc Signed-off-by: varun-edachali-dbx <[email protected]> * remove SeaResultSet Signed-off-by: varun-edachali-dbx <[email protected]> * clean imports and attributes Signed-off-by: varun-edachali-dbx <[email protected]> * pass CommandId to ExecResp Signed-off-by: varun-edachali-dbx <[email protected]> * remove changes in types Signed-off-by: varun-edachali-dbx <[email protected]> * add back essential types (ExecResponse, from_sea_state) Signed-off-by: varun-edachali-dbx <[email protected]> * fix fetch types Signed-off-by: varun-edachali-dbx <[email protected]> * excess imports Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff by maintaining logs Signed-off-by: varun-edachali-dbx <[email protected]> * fix int test types Signed-off-by: varun-edachali-dbx <[email protected]> * [squashed from exec-sea] init execution func Signed-off-by: varun-edachali-dbx <[email protected]> * remove irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove ResultSetFilter functionality Signed-off-by: varun-edachali-dbx <[email protected]> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * even more irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove sea response as init option Signed-off-by: varun-edachali-dbx <[email protected]> * exec test example scripts Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * [squashed from sea-exec] merge sea stuffs Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess removed docstring Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess changes in backend Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess imports Signed-off-by: varun-edachali-dbx <[email protected]> * remove accidentally removed _get_schema_desc Signed-off-by: varun-edachali-dbx <[email protected]> * remove unnecessary init with sea_response tests Signed-off-by: varun-edachali-dbx <[email protected]> * rmeove unnecessary changes Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * improved models and filters from cloudfetch-sea branch Signed-off-by: varun-edachali-dbx <[email protected]> * filters stuff (align with JDBC) Signed-off-by: varun-edachali-dbx <[email protected]> * backend from cloudfetch-sea Signed-off-by: varun-edachali-dbx <[email protected]> * remove filtering, metadata ops Signed-off-by: varun-edachali-dbx <[email protected]> * raise NotImplementedErrror for metadata ops Signed-off-by: varun-edachali-dbx <[email protected]> * add metadata commands Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * add metadata command unit tests Signed-off-by: varun-edachali-dbx <[email protected]> * change to valid table name Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary changes covered by #588 Signed-off-by: varun-edachali-dbx <[email protected]> * simplify test module Signed-off-by: varun-edachali-dbx <[email protected]> * logging -> debug level Signed-off-by: varun-edachali-dbx <[email protected]> * change table name in log Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary backend cahnges Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-needed GetChunksResponse Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-needed GetChunksResponse only relevant in Fetch phase Signed-off-by: varun-edachali-dbx <[email protected]> * reduce code duplication in response parsing Signed-off-by: varun-edachali-dbx <[email protected]> * reduce code duplication Signed-off-by: varun-edachali-dbx <[email protected]> * more clear docstrings Signed-off-by: varun-edachali-dbx <[email protected]> * introduce strongly typed ChunkInfo Signed-off-by: varun-edachali-dbx <[email protected]> * remove is_volume_operation from response Signed-off-by: varun-edachali-dbx <[email protected]> * add is_volume_op and more ResultData fields Signed-off-by: varun-edachali-dbx <[email protected]> * add test scripts Signed-off-by: varun-edachali-dbx <[email protected]> * Revert "Merge branch 'sea-migration' into exec-models-sea" This reverts commit 8bd12d829ea13abf8fc1507fff8cb21751001c67, reversing changes made to 030edf8df3db487b7af8d910ee51240d1339229e. * Revert "Merge branch 'exec-models-sea' into exec-phase-sea" This reverts commit be1997e0d6b6cf0f5499db2381971ec3a015a2f7, reversing changes made to 37813ba6d1fe06d7f9f10d510a059b88dc552496. * change logging level Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove _get_schema_bytes (for now) Signed-off-by: varun-edachali-dbx <[email protected]> * redundant comment…

* telemetry retry Signed-off-by: Sai Shree Pradhan <[email protected]> * shifted tests to unit test, removed unused imports Signed-off-by: Sai Shree Pradhan <[email protected]> * tests Signed-off-by: Sai Shree Pradhan <[email protected]> --------- Signed-off-by: Sai Shree Pradhan <[email protected]>

* Separate Session related functionality from Connection class (#571) * decouple session class from existing Connection ensure maintenance of current APIs of Connection while delegating responsibility Signed-off-by: varun-edachali-dbx <[email protected]> * add open property to Connection to ensure maintenance of existing API Signed-off-by: varun-edachali-dbx <[email protected]> * update unit tests to address ThriftBackend through session instead of through Connection Signed-off-by: varun-edachali-dbx <[email protected]> * chore: move session specific tests from test_client to test_session Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) as in CONTRIBUTING.md Signed-off-by: varun-edachali-dbx <[email protected]> * use connection open property instead of long chain through session Signed-off-by: varun-edachali-dbx <[email protected]> * trigger integration workflow Signed-off-by: varun-edachali-dbx <[email protected]> * fix: ensure open attribute of Connection never fails in case the openSession takes long, the initialisation of the session will not complete immediately. This could make the session attribute inaccessible. If the Connection is deleted in this time, the open() check will throw because the session attribute does not exist. Thus, we default to the Connection being closed in this case. This was not an issue before because open was a direct attribute of the Connection class. Caught in the integration tests. Signed-off-by: varun-edachali-dbx <[email protected]> * fix: de-complicate earlier connection open logic earlier, one of the integration tests was failing because 'session was not an attribute of Connection'. This is likely tied to a local configuration issue related to unittest that was causing an error in the test suite itself. The tests are now passing without checking for the session attribute. https://github.com/databricks/databricks-sql-python/pull/567/commits/c676f9b0281cc3e4fe9c6d8216cc62fc75eade3b Signed-off-by: varun-edachali-dbx <[email protected]> * Revert "fix: de-complicate earlier connection open logic" This reverts commit d6b1b196c98a6e9d8e593a88c34bbde010519ef4. Signed-off-by: varun-edachali-dbx <[email protected]> * [empty commit] attempt to trigger ci e2e workflow Signed-off-by: varun-edachali-dbx <[email protected]> * Update CODEOWNERS (#562) new codeowners Signed-off-by: varun-edachali-dbx <[email protected]> * Enhance Cursor close handling and context manager exception management to prevent server side resource leaks (#554) * Enhance Cursor close handling and context manager exception management * tests * fmt * Fix Cursor.close() to properly handle CursorAlreadyClosedError * Remove specific test message from Cursor.close() error handling * Improve error handling in connection and cursor context managers to ensure proper closure during exceptions, including KeyboardInterrupt. Add tests for nested cursor management and verify operation closure on server-side errors. * add * add Signed-off-by: varun-edachali-dbx <[email protected]> * PECOBLR-86 improve logging on python driver (#556) * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <[email protected]> * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <[email protected]> * fixed format Signed-off-by: Sai Shree Pradhan <[email protected]> * used lazy logging Signed-off-by: Sai Shree Pradhan <[email protected]> * changed debug to error logs Signed-off-by: Sai Shree Pradhan <[email protected]> * used lazy logging Signed-off-by: Sai Shree Pradhan <[email protected]> --------- Signed-off-by: Sai Shree Pradhan <[email protected]> Signed-off-by: varun-edachali-dbx <[email protected]> * Revert "Merge remote-tracking branch 'upstream/sea-migration' into decouple-session" This reverts commit dbb2ec52306b91072a2ee842270c7113aece9aff, reversing changes made to 7192f117279d4f0adcbafcdf2238c18663324515. Signed-off-by: varun-edachali-dbx <[email protected]> * Reapply "Merge remote-tracking branch 'upstream/sea-migration' into decouple-session" This reverts commit bdb83817f49e1d88a01679b11da8e55e8e80b42f. Signed-off-by: varun-edachali-dbx <[email protected]> * fix: separate session opening logic from instantiation ensures correctness of self.session.open call in Connection Signed-off-by: varun-edachali-dbx <[email protected]> * fix: use is_open attribute to denote session availability Signed-off-by: varun-edachali-dbx <[email protected]> * fix: access thrift backend through session Signed-off-by: varun-edachali-dbx <[email protected]> * chore: use get_handle() instead of private session attribute in client Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * fix: remove accidentally removed assertions Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> Signed-off-by: Sai Shree Pradhan <[email protected]> Co-authored-by: Jothi Prakash <[email protected]> Co-authored-by: Madhav Sainanee <[email protected]> Co-authored-by: Sai Shree Pradhan <[email protected]> * Introduce Backend Interface (DatabricksClient) (#573) NOTE: the `test_complex_types` e2e test was not working at the time of this merge. The test must be triggered when the test is back up and running as intended. * remove excess logs, assertions, instantiations large merge artifacts Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) + remove excess log (merge artifact) Signed-off-by: varun-edachali-dbx <[email protected]> * fix typing Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary check Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary replace call Signed-off-by: varun-edachali-dbx <[email protected]> * introduce __str__ methods for CommandId and SessionId Signed-off-by: varun-edachali-dbx <[email protected]> * docstrings for DatabricksClient interface Signed-off-by: varun-edachali-dbx <[email protected]> * stronger typing of Cursor and ExecuteResponse Signed-off-by: varun-edachali-dbx <[email protected]> * remove utility functions from backend interface, fix circular import Signed-off-by: varun-edachali-dbx <[email protected]> * rename info to properties Signed-off-by: varun-edachali-dbx <[email protected]> * newline for cleanliness Signed-off-by: varun-edachali-dbx <[email protected]> * fix circular import Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * to_hex_id -> get_hex_id Signed-off-by: varun-edachali-dbx <[email protected]> * better comment on protocol version getter Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * move guid to hex id to new utils module Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * move staging allowed local path to connection props Signed-off-by: varun-edachali-dbx <[email protected]> * add strong return type for execute_command Signed-off-by: varun-edachali-dbx <[email protected]> * skip auth, error handling in databricksclient interface Signed-off-by: varun-edachali-dbx <[email protected]> * chore: docstring + line width Signed-off-by: varun-edachali-dbx <[email protected]> * get_id -> get_guid Signed-off-by: varun-edachali-dbx <[email protected]> * chore: docstring Signed-off-by: varun-edachali-dbx <[email protected]> * fix: to_hex_id -> to_hex_guid Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> * Implement ResultSet Abstraction (backend interfaces for fetch phase) (#574) * ensure backend client returns a ResultSet type in backend tests Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * newline for cleanliness Signed-off-by: varun-edachali-dbx <[email protected]> * fix circular import Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * to_hex_id -> get_hex_id Signed-off-by: varun-edachali-dbx <[email protected]> * better comment on protocol version getter Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * stricter typing for cursor Signed-off-by: varun-edachali-dbx <[email protected]> * correct typing Signed-off-by: varun-edachali-dbx <[email protected]> * correct tests and merge artifacts Signed-off-by: varun-edachali-dbx <[email protected]> * remove accidentally modified workflow files remnants of old merge Signed-off-by: varun-edachali-dbx <[email protected]> * chore: remove accidentally modified workflow files Signed-off-by: varun-edachali-dbx <[email protected]> * add back accidentally removed docstrings Signed-off-by: varun-edachali-dbx <[email protected]> * clean up docstrings Signed-off-by: varun-edachali-dbx <[email protected]> * log hex Signed-off-by: varun-edachali-dbx <[email protected]> * remove unnecessary _replace call Signed-off-by: varun-edachali-dbx <[email protected]> * add __str__ for CommandId Signed-off-by: varun-edachali-dbx <[email protected]> * take TOpenSessionResp in get_protocol_version to maintain existing interface Signed-off-by: varun-edachali-dbx <[email protected]> * active_op_handle -> active_mmand_id Signed-off-by: varun-edachali-dbx <[email protected]> * ensure None returned for close_command Signed-off-by: varun-edachali-dbx <[email protected]> * account for ResultSet return in new pydocs Signed-off-by: varun-edachali-dbx <[email protected]> * pydoc for types Signed-off-by: varun-edachali-dbx <[email protected]> * move common state to ResultSet aprent Signed-off-by: varun-edachali-dbx <[email protected]> * stronger typing in resultSet behaviour Signed-off-by: varun-edachali-dbx <[email protected]> * remove redundant patch in test Signed-off-by: varun-edachali-dbx <[email protected]> * add has_been_closed_server_side assertion Signed-off-by: varun-edachali-dbx <[email protected]> * remove redundancies in tests Signed-off-by: varun-edachali-dbx <[email protected]> * more robust close check Signed-off-by: varun-edachali-dbx <[email protected]> * use normalised state in e2e test Signed-off-by: varun-edachali-dbx <[email protected]> * simplify corrected test Signed-off-by: varun-edachali-dbx <[email protected]> * add line gaps after multi-line pydocs for consistency Signed-off-by: varun-edachali-dbx <[email protected]> * use normalised CommandState type in ExecuteResponse Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> * Introduce Sea HTTP Client and test script (#583) * introduce http client (temp) and sea test file Signed-off-by: varun-edachali-dbx <[email protected]> * reduce verbosity Signed-off-by: varun-edachali-dbx <[email protected]> * redundant comment Signed-off-by: varun-edachali-dbx <[email protected]> * reduce redundancy, params and data separate Signed-off-by: varun-edachali-dbx <[email protected]> * rename client Signed-off-by: varun-edachali-dbx <[email protected]> * fix type issues Signed-off-by: varun-edachali-dbx <[email protected]> * reduce repetition in request calls Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary elifs Signed-off-by: varun-edachali-dbx <[email protected]> * add newline at EOF Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> * Introduce `SeaDatabricksClient` (Session Implementation) (#582) * [squashed from prev branch] introduce sea client with session open and close functionality Signed-off-by: varun-edachali-dbx <[email protected]> * remove accidental changes to workflows (merge artifacts) Signed-off-by: varun-edachali-dbx <[email protected]> * pass test_input to get_protocol_version instead of session_id to maintain previous API Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black + line gaps after multi-line pydocs) Signed-off-by: varun-edachali-dbx <[email protected]> * use factory for backend instantiation Signed-off-by: varun-edachali-dbx <[email protected]> * fix type issues Signed-off-by: varun-edachali-dbx <[email protected]> * remove redundant comments Signed-off-by: varun-edachali-dbx <[email protected]> * introduce models for requests and responses Signed-off-by: varun-edachali-dbx <[email protected]> * remove http client and test script to prevent diff from showing up post http-client merge Signed-off-by: varun-edachali-dbx <[email protected]> * Introduce Sea HTTP Client and test script (#583) * introduce http client (temp) and sea test file Signed-off-by: varun-edachali-dbx <[email protected]> * reduce verbosity Signed-off-by: varun-edachali-dbx <[email protected]> * redundant comment Signed-off-by: varun-edachali-dbx <[email protected]> * reduce redundancy, params and data separate Signed-off-by: varun-edachali-dbx <[email protected]> * rename client Signed-off-by: varun-edachali-dbx <[email protected]> * fix type issues Signed-off-by: varun-edachali-dbx <[email protected]> * reduce repetition in request calls Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary elifs Signed-off-by: varun-edachali-dbx <[email protected]> * add newline at EOF Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> * CustomHttpClient -> SeaHttpClient Signed-off-by: varun-edachali-dbx <[email protected]> * redundant comment in backend client Signed-off-by: varun-edachali-dbx <[email protected]> * regex for warehouse_id instead of .split, remove excess imports and behaviour Signed-off-by: varun-edachali-dbx <[email protected]> * remove redundant attributes Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * [nit] reduce nested code Signed-off-by: varun-edachali-dbx <[email protected]> * line gap after multi-line pydoc Signed-off-by: varun-edachali-dbx <[email protected]> * redundant imports Signed-off-by: varun-edachali-dbx <[email protected]> * move sea backend and models into separate sea/ dir Signed-off-by: varun-edachali-dbx <[email protected]> * move http client into separate sea/ dir Signed-off-by: varun-edachali-dbx <[email protected]> * change commands to include ones in docs Signed-off-by: varun-edachali-dbx <[email protected]> * add link to sql-ref-parameters for session-confs Signed-off-by: varun-edachali-dbx <[email protected]> * add client side filtering for session confs, add note on warehouses over endoints Signed-off-by: varun-edachali-dbx <[email protected]> * test unimplemented methods and max_download_threads prop Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> * Normalise Execution Response (clean backend interfaces) (#587) * [squash from exec-sea] bring over execution phase changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess test Signed-off-by: varun-edachali-dbx <[email protected]> * add docstring Signed-off-by: varun-edachali-dbx <[email protected]> * remvoe exec func in sea backend Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess files Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess models Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess sea backend tests Signed-off-by: varun-edachali-dbx <[email protected]> * cleanup Signed-off-by: varun-edachali-dbx <[email protected]> * re-introduce get_schema_desc Signed-off-by: varun-edachali-dbx <[email protected]> * remove SeaResultSet Signed-off-by: varun-edachali-dbx <[email protected]> * clean imports and attributes Signed-off-by: varun-edachali-dbx <[email protected]> * pass CommandId to ExecResp Signed-off-by: varun-edachali-dbx <[email protected]> * remove changes in types Signed-off-by: varun-edachali-dbx <[email protected]> * add back essential types (ExecResponse, from_sea_state) Signed-off-by: varun-edachali-dbx <[email protected]> * fix fetch types Signed-off-by: varun-edachali-dbx <[email protected]> * excess imports Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff by maintaining logs Signed-off-by: varun-edachali-dbx <[email protected]> * fix int test types Signed-off-by: varun-edachali-dbx <[email protected]> * move guid_to_hex_id import to utils Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff in guid utils import Signed-off-by: varun-edachali-dbx <[email protected]> * move arrow_schema_bytes back into ExecuteResult Signed-off-by: varun-edachali-dbx <[email protected]> * maintain log Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary assignment Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary tuple response Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-ncessary verbose mocking Signed-off-by: varun-edachali-dbx <[email protected]> * move Queue construction to ResultSert Signed-off-by: varun-edachali-dbx <[email protected]> * move description to List[Tuple] Signed-off-by: varun-edachali-dbx <[email protected]> * frmatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff (remove explicit tuple conversion) Signed-off-by: varun-edachali-dbx <[email protected]> * remove has_more_rows from ExecuteResponse Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary has_more_rows aclc Signed-off-by: varun-edachali-dbx <[email protected]> * default has_more_rows to True Signed-off-by: varun-edachali-dbx <[email protected]> * return has_more_rows from ExecResponse conversion during GetRespMetadata Signed-off-by: varun-edachali-dbx <[email protected]> * remove unnecessary replacement Signed-off-by: varun-edachali-dbx <[email protected]> * better mocked backend naming Signed-off-by: varun-edachali-dbx <[email protected]> * remove has_more_rows test in ExecuteResponse Signed-off-by: varun-edachali-dbx <[email protected]> * introduce replacement of original has_more_rows read test Signed-off-by: varun-edachali-dbx <[email protected]> * call correct method in test_use_arrow_schema Signed-off-by: varun-edachali-dbx <[email protected]> * call correct method in test_fall_back_to_hive_schema Signed-off-by: varun-edachali-dbx <[email protected]> * re-introduce result response read test Signed-off-by: varun-edachali-dbx <[email protected]> * simplify test Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess fetch_results mocks Signed-off-by: varun-edachali-dbx <[email protected]> * more minimal changes to thrift_backend tests Signed-off-by: varun-edachali-dbx <[email protected]> * move back to old table types Signed-off-by: varun-edachali-dbx <[email protected]> * remove outdated arrow_schema_bytes return Signed-off-by: varun-edachali-dbx <[email protected]> * remove duplicate import Signed-off-by: varun-edachali-dbx <[email protected]> * rephrase model docstrings to explicitly denote that they are representations and not used over the wire Signed-off-by: varun-edachali-dbx <[email protected]> * has_more_rows -> is_direct_results Signed-off-by: varun-edachali-dbx <[email protected]> * switch docstring format to align with Connection class Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> * Introduce models for `SeaDatabricksClient` (#595) * [squash from exec-sea] bring over execution phase changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess test Signed-off-by: varun-edachali-dbx <[email protected]> * add docstring Signed-off-by: varun-edachali-dbx <[email protected]> * remvoe exec func in sea backend Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess files Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess models Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess sea backend tests Signed-off-by: varun-edachali-dbx <[email protected]> * cleanup Signed-off-by: varun-edachali-dbx <[email protected]> * re-introduce get_schema_desc Signed-off-by: varun-edachali-dbx <[email protected]> * remove SeaResultSet Signed-off-by: varun-edachali-dbx <[email protected]> * clean imports and attributes Signed-off-by: varun-edachali-dbx <[email protected]> * pass CommandId to ExecResp Signed-off-by: varun-edachali-dbx <[email protected]> * remove changes in types Signed-off-by: varun-edachali-dbx <[email protected]> * add back essential types (ExecResponse, from_sea_state) Signed-off-by: varun-edachali-dbx <[email protected]> * fix fetch types Signed-off-by: varun-edachali-dbx <[email protected]> * excess imports Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff by maintaining logs Signed-off-by: varun-edachali-dbx <[email protected]> * fix int test types Signed-off-by: varun-edachali-dbx <[email protected]> * [squashed from exec-sea] init execution func Signed-off-by: varun-edachali-dbx <[email protected]> * remove irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove ResultSetFilter functionality Signed-off-by: varun-edachali-dbx <[email protected]> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * even more irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove sea response as init option Signed-off-by: varun-edachali-dbx <[email protected]> * exec test example scripts Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * [squashed from sea-exec] merge sea stuffs Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess removed docstring Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess changes in backend Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess imports Signed-off-by: varun-edachali-dbx <[email protected]> * remove accidentally removed _get_schema_desc Signed-off-by: varun-edachali-dbx <[email protected]> * remove unnecessary init with sea_response tests Signed-off-by: varun-edachali-dbx <[email protected]> * rmeove unnecessary changes Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * improved models and filters from cloudfetch-sea branch Signed-off-by: varun-edachali-dbx <[email protected]> * filters stuff (align with JDBC) Signed-off-by: varun-edachali-dbx <[email protected]> * backend from cloudfetch-sea Signed-off-by: varun-edachali-dbx <[email protected]> * remove filtering, metadata ops Signed-off-by: varun-edachali-dbx <[email protected]> * raise NotImplementedErrror for metadata ops Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary backend cahnges Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-needed GetChunksResponse only relevant in Fetch phase Signed-off-by: varun-edachali-dbx <[email protected]> * reduce code duplication Signed-off-by: varun-edachali-dbx <[email protected]> * more clear docstrings Signed-off-by: varun-edachali-dbx <[email protected]> * introduce strongly typed ChunkInfo Signed-off-by: varun-edachali-dbx <[email protected]> * remove is_volume_operation from response Signed-off-by: varun-edachali-dbx <[email protected]> * add is_volume_op and more ResultData fields Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> * Introduce preliminary SEA Result Set (#588) * [squash from exec-sea] bring over execution phase changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess test Signed-off-by: varun-edachali-dbx <[email protected]> * add docstring Signed-off-by: varun-edachali-dbx <[email protected]> * remvoe exec func in sea backend Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess files Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess models Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess sea backend tests Signed-off-by: varun-edachali-dbx <[email protected]> * cleanup Signed-off-by: varun-edachali-dbx <[email protected]> * re-introduce get_schema_desc Signed-off-by: varun-edachali-dbx <[email protected]> * remove SeaResultSet Signed-off-by: varun-edachali-dbx <[email protected]> * clean imports and attributes Signed-off-by: varun-edachali-dbx <[email protected]> * pass CommandId to ExecResp Signed-off-by: varun-edachali-dbx <[email protected]> * remove changes in types Signed-off-by: varun-edachali-dbx <[email protected]> * add back essential types (ExecResponse, from_sea_state) Signed-off-by: varun-edachali-dbx <[email protected]> * fix fetch types Signed-off-by: varun-edachali-dbx <[email protected]> * excess imports Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff by maintaining logs Signed-off-by: varun-edachali-dbx <[email protected]> * fix int test types Signed-off-by: varun-edachali-dbx <[email protected]> * [squashed from exec-sea] init execution func Signed-off-by: varun-edachali-dbx <[email protected]> * remove irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove ResultSetFilter functionality Signed-off-by: varun-edachali-dbx <[email protected]> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * even more irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove sea response as init option Signed-off-by: varun-edachali-dbx <[email protected]> * move guid_to_hex_id import to utils Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff in guid utils import Signed-off-by: varun-edachali-dbx <[email protected]> * move arrow_schema_bytes back into ExecuteResult Signed-off-by: varun-edachali-dbx <[email protected]> * maintain log Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary assignment Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary tuple response Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-ncessary verbose mocking Signed-off-by: varun-edachali-dbx <[email protected]> * move Queue construction to ResultSert Signed-off-by: varun-edachali-dbx <[email protected]> * move description to List[Tuple] Signed-off-by: varun-edachali-dbx <[email protected]> * frmatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff (remove explicit tuple conversion) Signed-off-by: varun-edachali-dbx <[email protected]> * remove has_more_rows from ExecuteResponse Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary has_more_rows aclc Signed-off-by: varun-edachali-dbx <[email protected]> * default has_more_rows to True Signed-off-by: varun-edachali-dbx <[email protected]> * return has_more_rows from ExecResponse conversion during GetRespMetadata Signed-off-by: varun-edachali-dbx <[email protected]> * remove unnecessary replacement Signed-off-by: varun-edachali-dbx <[email protected]> * better mocked backend naming Signed-off-by: varun-edachali-dbx <[email protected]> * remove has_more_rows test in ExecuteResponse Signed-off-by: varun-edachali-dbx <[email protected]> * introduce replacement of original has_more_rows read test Signed-off-by: varun-edachali-dbx <[email protected]> * call correct method in test_use_arrow_schema Signed-off-by: varun-edachali-dbx <[email protected]> * call correct method in test_fall_back_to_hive_schema Signed-off-by: varun-edachali-dbx <[email protected]> * re-introduce result response read test Signed-off-by: varun-edachali-dbx <[email protected]> * simplify test Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess fetch_results mocks Signed-off-by: varun-edachali-dbx <[email protected]> * more minimal changes to thrift_backend tests Signed-off-by: varun-edachali-dbx <[email protected]> * move back to old table types Signed-off-by: varun-edachali-dbx <[email protected]> * remove outdated arrow_schema_bytes return Signed-off-by: varun-edachali-dbx <[email protected]> * align SeaResultSet with new structure Signed-off-by: varun-edachali-dbx <[email protected]> * correct sea res set tests Signed-off-by: varun-edachali-dbx <[email protected]> * remove duplicate import Signed-off-by: varun-edachali-dbx <[email protected]> * rephrase model docstrings to explicitly denote that they are representations and not used over the wire Signed-off-by: varun-edachali-dbx <[email protected]> * has_more_rows -> is_direct_results Signed-off-by: varun-edachali-dbx <[email protected]> * switch docstring format to align with Connection class Signed-off-by: varun-edachali-dbx <[email protected]> * has_more_rows -> is_direct_results Signed-off-by: varun-edachali-dbx <[email protected]> * fix type errors with arrow_schema_bytes Signed-off-by: varun-edachali-dbx <[email protected]> * spaces after multi line pydocs Signed-off-by: varun-edachali-dbx <[email protected]> * remove duplicate queue init (merge artifact) Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff (remove newlines) Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary changes covered by #588 anyway Signed-off-by: varun-edachali-dbx <[email protected]> * Revert "remove un-necessary changes" This reverts commit a70a6cee277db44d6951604e890f91cae9f92f32. Signed-off-by: varun-edachali-dbx <[email protected]> * b"" -> None Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> * remove invalid ExecuteResponse import Signed-off-by: varun-edachali-dbx <[email protected]> * Separate Session related functionality from Connection class (#571) * decouple session class from existing Connection ensure maintenance of current APIs of Connection while delegating responsibility Signed-off-by: varun-edachali-dbx <[email protected]> * add open property to Connection to ensure maintenance of existing API Signed-off-by: varun-edachali-dbx <[email protected]> * update unit tests to address ThriftBackend through session instead of through Connection Signed-off-by: varun-edachali-dbx <[email protected]> * chore: move session specific tests from test_client to test_session Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) as in CONTRIBUTING.md Signed-off-by: varun-edachali-dbx <[email protected]> * use connection open property instead of long chain through session Signed-off-by: varun-edachali-dbx <[email protected]> * trigger integration workflow Signed-off-by: varun-edachali-dbx <[email protected]> * fix: ensure open attribute of Connection never fails in case the openSession takes long, the initialisation of the session will not complete immediately. This could make the session attribute inaccessible. If the Connection is deleted in this time, the open() check will throw because the session attribute does not exist. Thus, we default to the Connection being closed in this case. This was not an issue before because open was a direct attribute of the Connection class. Caught in the integration tests. Signed-off-by: varun-edachali-dbx <[email protected]> * fix: de-complicate earlier connection open logic earlier, one of the integration tests was failing because 'session was not an attribute of Connection'. This is likely tied to a local configuration issue related to unittest that was causing an error in the test suite itself. The tests are now passing without checking for the session attribute. https://github.com/databricks/databricks-sql-python/pull/567/commits/c676f9b0281cc3e4fe9c6d8216cc62fc75eade3b Signed-off-by: varun-edachali-dbx <[email protected]> * Revert "fix: de-complicate earlier connection open logic" This reverts commit d6b1b196c98a6e9d8e593a88c34bbde010519ef4. Signed-off-by: varun-edachali-dbx <[email protected]> * [empty commit] attempt to trigger ci e2e workflow Signed-off-by: varun-edachali-dbx <[email protected]> * Update CODEOWNERS (#562) new codeowners Signed-off-by: varun-edachali-dbx <[email protected]> * Enhance Cursor close handling and context manager exception management to prevent server side resource leaks (#554) * Enhance Cursor close handling and context manager exception management * tests * fmt * Fix Cursor.close() to properly handle CursorAlreadyClosedError * Remove specific test message from Cursor.close() error handling * Improve error handling in connection and cursor context managers to ensure proper closure during exceptions, including KeyboardInterrupt. Add tests for nested cursor management and verify operation closure on server-side errors. * add * add Signed-off-by: varun-edachali-dbx <[email protected]> * PECOBLR-86 improve logging on python driver (#556) * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <[email protected]> * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <[email protected]> * fixed format Signed-off-by: Sai Shree Pradhan <[email protected]> * used lazy logging Signed-off-by: Sai Shree Pradhan <[email protected]> * changed debug to error logs Signed-off-by: Sai Shree Pradhan <[email protected]> * used lazy logging Signed-off-by: Sai Shree Pradhan <[email protected]> --------- Signed-off-by: Sai Shree Pradhan <[email protected]> Signed-off-by: varun-edachali-dbx <[email protected]> * Revert "Merge remote-tracking branch 'upstream/sea-migration' into decouple-session" This reverts commit dbb2ec52306b91072a2ee842270c7113aece9aff, reversing changes made to 7192f117279d4f0adcbafcdf2238c18663324515. Signed-off-by: varun-edachali-dbx <[email protected]> * Reapply "Merge remote-tracking branch 'upstream/sea-migration' into decouple-session" This reverts commit bdb83817f49e1d88a01679b11da8e55e8e80b42f. Signed-off-by: varun-edachali-dbx <[email protected]> * fix: separate session opening logic from instantiation ensures correctness of self.session.open call in Connection Signed-off-by: varun-edachali-dbx <[email protected]> * fix: use is_open attribute to denote session availability Signed-off-by: varun-edachali-dbx <[email protected]> * fix: access thrift backend through session Signed-off-by: varun-edachali-dbx <[email protected]> * chore: use get_handle() instead of private session attribute in client Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * fix: remove accidentally removed assertions Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> Signed-off-by: Sai Shree Pradhan <[email protected]> Co-authored-by: Jothi Prakash <[email protected]> Co-authored-by: Madhav Sainanee <[email protected]> Co-authored-by: Sai Shree Pradhan <[email protected]> * Introduce Backend Interface (DatabricksClient) (#573) NOTE: the `test_complex_types` e2e test was not working at the time of this merge. The test must be triggered when the test is back up and running as intended. * remove excess logs, assertions, instantiations large merge artifacts Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) + remove excess log (merge artifact) Signed-off-by: varun-edachali-dbx <[email protected]> * fix typing Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary check Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary replace call Signed-off-by: varun-edachali-dbx <[email protected]> * introduce __str__ methods for CommandId and SessionId Signed-off-by: varun-edachali-dbx <[email protected]> * docstrings for DatabricksClient interface Signed-off-by: varun-edachali-dbx <[email protected]> * stronger typing of Cursor and ExecuteResponse Signed-off-by: varun-edachali-dbx <[email protected]> * remove utility functions from backend interface, fix circular import Signed-off-by: varun-edachali-dbx <[email protected]> * rename info to properties Signed-off-by: varun-edachali-dbx <[email protected]> * newline for cleanliness Signed-off-by: varun-edachali-dbx <[email protected]> * fix circular import Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * to_hex_id -> get_hex_id Signed-off-by: varun-edachali-dbx <[email protected]> * better comment on protocol version getter Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * move guid to hex id to new utils module Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * move staging allowed local path to connection props Signed-off-by: varun-edachali-dbx <[email protected]> * add strong return type for execute_command Signed-off-by: varun-edachali-dbx <[email protected]> * skip auth, error handling in databricksclient interface Signed-off-by: varun-edachali-dbx <[email protected]> * chore: docstring + line width Signed-off-by: varun-edachali-dbx <[email protected]> * get_id -> get_guid Signed-off-by: varun-edachali-dbx <[email protected]> * chore: docstring Signed-off-by: varun-edachali-dbx <[email protected]> * fix: to_hex_id -> to_hex_guid Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> * Implement ResultSet Abstraction (backend interfaces for fetch phase) (#574) * ensure backend client returns a ResultSet type in backend tests Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * newline for cleanliness Signed-off-by: varun-edachali-dbx <[email protected]> * fix circular import Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * to_hex_id -> get_hex_id Signed-off-by: varun-edachali-dbx <[email protected]> * better comment on protocol version getter Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * stricter typing for cursor Signed-off-by: varun-edachali-dbx <[email protected]> * correct typing Signed-off-by: varun-edachali-dbx <[email protected]> * correct tests and merge artifacts Signed-off-by: varun-edachali-dbx <[email protected]> * remove accidentally modified workflow files remnants of old merge Signed-off-by: varun-edachali-dbx <[email protected]> * chore: remove accidentally modified workflow files Signed-off-by: varun-edachali-dbx <[email protected]> * add back accidentally removed docstrings Signed-off-by: varun-edachali-dbx <[email protected]> * clean up docstrings Signed-off-by: varun-edachali-dbx <[email protected]> * log hex Signed-off-by: varun-edachali-dbx <[email protected]> * remove unnecessary _replace call Signed-off-by: varun-edachali-dbx <[email protected]> * add __str__ for CommandId Signed-off-by: varun-edachali-dbx <[email protected]> * take TOpenSessionResp in get_protocol_version to maintain existing interface Signed-off-by: varun-edachali-dbx <[email protected]> * active_op_handle -> active_mmand_id Signed-off-by: varun-edachali-dbx <[email protected]> * ensure None returned for close_command Signed-off-by: varun-edachali-dbx <[email protected]> * account for ResultSet return in new pydocs Signed-off-by: varun-edachali-dbx <[email protected]> * pydoc for types Signed-off-by: varun-edachali-dbx <[email protected]> * move common state to ResultSet aprent Signed-off-by: varun-edachali-dbx <[email protected]> * stronger typing in resultSet behaviour Signed-off-by: varun-edachali-dbx <[email protected]> * remove redundant patch in test Signed-off-by: varun-edachali-dbx <[email protected]> * add has_been_closed_server_side assertion Signed-off-by: varun-edachali-dbx <[email protected]> * remove redundancies in tests Signed-off-by: varun-edachali-dbx <[email protected]> * more robust close check Signed-off-by: varun-edachali-dbx <[email protected]> * use normalised state in e2e test Signed-off-by: varun-edachali-dbx <[email protected]> * simplify corrected test Signed-off-by: varun-edachali-dbx <[email protected]> * add line gaps after multi-line pydocs for consistency Signed-off-by: varun-edachali-dbx <[email protected]> * use normalised CommandState type in ExecuteResponse Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary initialisation assertions Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary line break s Signed-off-by: varun-edachali-dbx <[email protected]> * more un-necessary line breaks Signed-off-by: varun-edachali-dbx <[email protected]> * constrain diff of test_closing_connection_closes_commands Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff of test_closing_connection_closes_commands Signed-off-by: varun-edachali-dbx <[email protected]> * use pytest-like assertions for test_closing_connection_closes_commands Signed-off-by: varun-edachali-dbx <[email protected]> * ensure command_id is not None Signed-off-by: varun-edachali-dbx <[email protected]> * line breaks after multi-line pyfocs Signed-off-by: varun-edachali-dbx <[email protected]> * ensure non null operationHandle for commandId creation Signed-off-by: varun-edachali-dbx <[email protected]> * use command_id methods instead of explicit guid_to_hex_id conversion Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary artifacts in test_session, add back assertion Signed-off-by: varun-edachali-dbx <[email protected]> * Implement SeaDatabricksClient (Complete Execution Spec) (#590) * [squash from exec-sea] bring over execution phase changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess test Signed-off-by: varun-edachali-dbx <[email protected]> * add docstring Signed-off-by: varun-edachali-dbx <[email protected]> * remvoe exec func in sea backend Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess files Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess models Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess sea backend tests Signed-off-by: varun-edachali-dbx <[email protected]> * cleanup Signed-off-by: varun-edachali-dbx <[email protected]> * re-introduce get_schema_desc Signed-off-by: varun-edachali-dbx <[email protected]> * remove SeaResultSet Signed-off-by: varun-edachali-dbx <[email protected]> * clean imports and attributes Signed-off-by: varun-edachali-dbx <[email protected]> * pass CommandId to ExecResp Signed-off-by: varun-edachali-dbx <[email protected]> * remove changes in types Signed-off-by: varun-edachali-dbx <[email protected]> * add back essential types (ExecResponse, from_sea_state) Signed-off-by: varun-edachali-dbx <[email protected]> * fix fetch types Signed-off-by: varun-edachali-dbx <[email protected]> * excess imports Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff by maintaining logs Signed-off-by: varun-edachali-dbx <[email protected]> * fix int test types Signed-off-by: varun-edachali-dbx <[email protected]> * [squashed from exec-sea] init execution func Signed-off-by: varun-edachali-dbx <[email protected]> * remove irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove ResultSetFilter functionality Signed-off-by: varun-edachali-dbx <[email protected]> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * even more irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove sea response as init option Signed-off-by: varun-edachali-dbx <[email protected]> * exec test example scripts Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * [squashed from sea-exec] merge sea stuffs Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess removed docstring Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess changes in backend Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess imports Signed-off-by: varun-edachali-dbx <[email protected]> * remove accidentally removed _get_schema_desc Signed-off-by: varun-edachali-dbx <[email protected]> * remove unnecessary init with sea_response tests Signed-off-by: varun-edachali-dbx <[email protected]> * rmeove unnecessary changes Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * improved models and filters from cloudfetch-sea branch Signed-off-by: varun-edachali-dbx <[email protected]> * filters stuff (align with JDBC) Signed-off-by: varun-edachali-dbx <[email protected]> * backend from cloudfetch-sea Signed-off-by: varun-edachali-dbx <[email protected]> * remove filtering, metadata ops Signed-off-by: varun-edachali-dbx <[email protected]> * raise NotImplementedErrror for metadata ops Signed-off-by: varun-edachali-dbx <[email protected]> * change to valid table name Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary changes covered by #588 Signed-off-by: varun-edachali-dbx <[email protected]> * simplify test module Signed-off-by: varun-edachali-dbx <[email protected]> * logging -> debug level Signed-off-by: varun-edachali-dbx <[email protected]> * change table name in log Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary backend cahnges Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-needed GetChunksResponse Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-needed GetChunksResponse only relevant in Fetch phase Signed-off-by: varun-edachali-dbx <[email protected]> * reduce code duplication in response parsing Signed-off-by: varun-edachali-dbx <[email protected]> * reduce code duplication Signed-off-by: varun-edachali-dbx <[email protected]> * more clear docstrings Signed-off-by: varun-edachali-dbx <[email protected]> * introduce strongly typed ChunkInfo Signed-off-by: varun-edachali-dbx <[email protected]> * remove is_volume_operation from response Signed-off-by: varun-edachali-dbx <[email protected]> * add is_volume_op and more ResultData fields Signed-off-by: varun-edachali-dbx <[email protected]> * add test scripts Signed-off-by: varun-edachali-dbx <[email protected]> * Revert "Merge branch 'exec-models-sea' into exec-phase-sea" This reverts commit be1997e0d6b6cf0f5499db2381971ec3a015a2f7, reversing changes made to 37813ba6d1fe06d7f9f10d510a059b88dc552496. * change logging level Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove _get_schema_bytes (for now) Signed-off-by: varun-edachali-dbx <[email protected]> * redundant comments Signed-off-by: varun-edachali-dbx <[email protected]> * remove fetch phase methods Signed-off-by: varun-edachali-dbx <[email protected]> * reduce code repetititon + introduce gaps after multi line pydocs Signed-off-by: varun-edachali-dbx <[email protected]> * remove unused imports Signed-off-by: varun-edachali-dbx <[email protected]> * move description extraction to helper func Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * add more unit tests Signed-off-by: varun-edachali-dbx <[email protected]> * streamline unit tests Signed-off-by: varun-edachali-dbx <[email protected]> * test getting the list of allowed configurations Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff Signed-off-by: varun-edachali-dbx <[email protected]> * house constants in enums for readability and immutability Signed-off-by: varun-edachali-dbx <[email protected]> * add note on hybrid disposition Signed-off-by: varun-edachali-dbx <[email protected]> * remove redundant note on arrow_schema_bytes Signed-off-by: varun-edachali-dbx <[email protected]> * remove invalid import Signed-off-by: varun-edachali-dbx <[email protected]> * add strong typing for manifest in _extract_description Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary column skipping Signed-off-by: varun-edachali-dbx <[email protected]> * remove parsing in backend Signed-off-by: varun-edachali-dbx <[email protected]> * fix: convert sea statement id to CommandId type Signed-off-by: varun-edachali-dbx <[email protected]> * make polling interval a separate constant Signed-off-by: varun-edachali-dbx <[email protected]> * align state checking with Thrift implementation Signed-off-by: varun-edachali-dbx <[email protected]> * update unit tests according to changes Signed-off-by: varun-edachali-dbx <[email protected]> * add unit tests for added methods Signed-off-by: varun-edachali-dbx <[email protected]> * add spec to description extraction docstring, add strong typing to params Signed-off-by: varun-edachali-dbx <[email protected]> * add strong typing for backend parameters arg Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> * add from __future__ import annotations to remove string literals around forward refs, remove some unused imports Signed-off-by: varun-edachali-dbx <[email protected]> * move docstring of DatabricksClient within class Signed-off-by: varun-edachali-dbx <[email protected]> * move ThriftResultSet import to top of file Signed-off-by: varun-edachali-dbx <[email protected]> * make backend/utils __init__ file empty Signed-off-by: varun-edachali-dbx <[email protected]> * use from __future__ import annotations to remove string literals around Cursor Signed-off-by: varun-edachali-dbx <[email protected]> * use lazy logging Signed-off-by: varun-edachali-dbx <[email protected]> * replace getters with property tag Signed-off-by: varun-edachali-dbx <[email protected]> * set active_command_id to None, not active_op_handle Signed-off-by: varun-edachali-dbx <[email protected]> * align test_session with pytest instead of unittest Signed-off-by: varun-edachali-dbx <[email protected]> * remove duplicate test, correct active_command_id attribute Signed-off-by: varun-edachali-dbx <[email protected]> * SeaDatabricksClient: Add Metadata Commands (#593) * [squash from exec-sea] bring over execution phase changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess test Signed-off-by: varun-edachali-dbx <[email protected]> * add docstring Signed-off-by: varun-edachali-dbx <[email protected]> * remvoe exec func in sea backend Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess files Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess models Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess sea backend tests Signed-off-by: varun-edachali-dbx <[email protected]> * cleanup Signed-off-by: varun-edachali-dbx <[email protected]> * re-introduce get_schema_desc Signed-off-by: varun-edachali-dbx <[email protected]> * remove SeaResultSet Signed-off-by: varun-edachali-dbx <[email protected]> * clean imports and attributes Signed-off-by: varun-edachali-dbx <[email protected]> * pass CommandId to ExecResp Signed-off-by: varun-edachali-dbx <[email protected]> * remove changes in types Signed-off-by: varun-edachali-dbx <[email protected]> * add back essential types (ExecResponse, from_sea_state) Signed-off-by: varun-edachali-dbx <[email protected]> * fix fetch types Signed-off-by: varun-edachali-dbx <[email protected]> * excess imports Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff by maintaining logs Signed-off-by: varun-edachali-dbx <[email protected]> * fix int test types Signed-off-by: varun-edachali-dbx <[email protected]> * [squashed from exec-sea] init execution func Signed-off-by: varun-edachali-dbx <[email protected]> * remove irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove ResultSetFilter functionality Signed-off-by: varun-edachali-dbx <[email protected]> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * even more irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove sea response as init option Signed-off-by: varun-edachali-dbx <[email protected]> * exec test example scripts Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * [squashed from sea-exec] merge sea stuffs Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess removed docstring Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess changes in backend Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess imports Signed-off-by: varun-edachali-dbx <[email protected]> * remove accidentally removed _get_schema_desc Signed-off-by: varun-edachali-dbx <[email protected]> * remove unnecessary init with sea_response tests Signed-off-by: varun-edachali-dbx <[email protected]> * rmeove unnecessary changes Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * improved models and filters from cloudfetch-sea branch Signed-off-by: varun-edachali-dbx <[email protected]> * filters stuff (align with JDBC) Signed-off-by: varun-edachali-dbx <[email protected]> * backend from cloudfetch-sea Signed-off-by: varun-edachali-dbx <[email protected]> * remove filtering, metadata ops Signed-off-by: varun-edachali-dbx <[email protected]> * raise NotImplementedErrror for metadata ops Signed-off-by: varun-edachali-dbx <[email protected]> * add metadata commands Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * add metadata command unit tests Signed-off-by: varun-edachali-dbx <[email protected]> * change to valid table name Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary changes covered by #588 Signed-off-by: varun-edachali-dbx <[email protected]> * simplify test module Signed-off-by: varun-edachali-dbx <[email protected]> * logging -> debug level Signed-off-by: varun-edachali-dbx <[email protected]> * change table name in log Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary backend cahnges Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-needed GetChunksResponse Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-needed GetChunksResponse only relevant in Fetch phase Signed-off-by: varun-edachali-dbx <[email protected]> * reduce code duplication in response parsing Signed-off-by: varun-edachali-dbx <[email protected]> * reduce code duplication Signed-off-by: varun-edachali-dbx <[email protected]> * more clear docstrings Signed-off-by: varun-edachali-dbx <[email protected]> * introduce strongly typed ChunkInfo Signed-off-by: varun-edachali-dbx <[email protected]> * remove is_volume_operation from response Signed-off-by: varun-edachali-dbx <[email protected]> * add is_volume_op and more ResultData fields Signed-off-by: varun-edachali-dbx <[email protected]> * add test scripts Signed-off-by: varun-edachali-dbx <[email protected]> * Revert "Merge branch 'sea-migration' into exec-models-sea" This reverts commit 8bd12d829ea13abf8fc1507fff8cb21751001c67, reversing changes made to 030edf8df3db487b7af8d910ee51240d1339229e. * Revert "Merge branch 'exec-models-sea' into exec-phase-sea" This reverts commit be1997e0d6b6cf0f5499db2381971ec3a015a2f7, reversing changes made to 37813ba6d1fe06d7f9f10d510a059b88dc552496. * change logging level Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove _get_schema_bytes (for now) Signed-off-by: varun-edachali-dbx <[email protected]> * redundant comments S…

…bricks#622) configurable telemetry batch size, time based flush Signed-off-by: Sai Shree Pradhan <[email protected]>

* Separate Session related functionality from Connection class (#571) * decouple session class from existing Connection ensure maintenance of current APIs of Connection while delegating responsibility Signed-off-by: varun-edachali-dbx <[email protected]> * add open property to Connection to ensure maintenance of existing API Signed-off-by: varun-edachali-dbx <[email protected]> * update unit tests to address ThriftBackend through session instead of through Connection Signed-off-by: varun-edachali-dbx <[email protected]> * chore: move session specific tests from test_client to test_session Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) as in CONTRIBUTING.md Signed-off-by: varun-edachali-dbx <[email protected]> * use connection open property instead of long chain through session Signed-off-by: varun-edachali-dbx <[email protected]> * trigger integration workflow Signed-off-by: varun-edachali-dbx <[email protected]> * fix: ensure open attribute of Connection never fails in case the openSession takes long, the initialisation of the session will not complete immediately. This could make the session attribute inaccessible. If the Connection is deleted in this time, the open() check will throw because the session attribute does not exist. Thus, we default to the Connection being closed in this case. This was not an issue before because open was a direct attribute of the Connection class. Caught in the integration tests. Signed-off-by: varun-edachali-dbx <[email protected]> * fix: de-complicate earlier connection open logic earlier, one of the integration tests was failing because 'session was not an attribute of Connection'. This is likely tied to a local configuration issue related to unittest that was causing an error in the test suite itself. The tests are now passing without checking for the session attribute. https://github.com/databricks/databricks-sql-python/pull/567/commits/c676f9b0281cc3e4fe9c6d8216cc62fc75eade3b Signed-off-by: varun-edachali-dbx <[email protected]> * Revert "fix: de-complicate earlier connection open logic" This reverts commit d6b1b196c98a6e9d8e593a88c34bbde010519ef4. Signed-off-by: varun-edachali-dbx <[email protected]> * [empty commit] attempt to trigger ci e2e workflow Signed-off-by: varun-edachali-dbx <[email protected]> * Update CODEOWNERS (#562) new codeowners Signed-off-by: varun-edachali-dbx <[email protected]> * Enhance Cursor close handling and context manager exception management to prevent server side resource leaks (#554) * Enhance Cursor close handling and context manager exception management * tests * fmt * Fix Cursor.close() to properly handle CursorAlreadyClosedError * Remove specific test message from Cursor.close() error handling * Improve error handling in connection and cursor context managers to ensure proper closure during exceptions, including KeyboardInterrupt. Add tests for nested cursor management and verify operation closure on server-side errors. * add * add Signed-off-by: varun-edachali-dbx <[email protected]> * PECOBLR-86 improve logging on python driver (#556) * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <[email protected]> * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <[email protected]> * fixed format Signed-off-by: Sai Shree Pradhan <[email protected]> * used lazy logging Signed-off-by: Sai Shree Pradhan <[email protected]> * changed debug to error logs Signed-off-by: Sai Shree Pradhan <[email protected]> * used lazy logging Signed-off-by: Sai Shree Pradhan <[email protected]> --------- Signed-off-by: Sai Shree Pradhan <[email protected]> Signed-off-by: varun-edachali-dbx <[email protected]> * Revert "Merge remote-tracking branch 'upstream/sea-migration' into decouple-session" This reverts commit dbb2ec52306b91072a2ee842270c7113aece9aff, reversing changes made to 7192f117279d4f0adcbafcdf2238c18663324515. Signed-off-by: varun-edachali-dbx <[email protected]> * Reapply "Merge remote-tracking branch 'upstream/sea-migration' into decouple-session" This reverts commit bdb83817f49e1d88a01679b11da8e55e8e80b42f. Signed-off-by: varun-edachali-dbx <[email protected]> * fix: separate session opening logic from instantiation ensures correctness of self.session.open call in Connection Signed-off-by: varun-edachali-dbx <[email protected]> * fix: use is_open attribute to denote session availability Signed-off-by: varun-edachali-dbx <[email protected]> * fix: access thrift backend through session Signed-off-by: varun-edachali-dbx <[email protected]> * chore: use get_handle() instead of private session attribute in client Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * fix: remove accidentally removed assertions Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> Signed-off-by: Sai Shree Pradhan <[email protected]> Co-authored-by: Jothi Prakash <[email protected]> Co-authored-by: Madhav Sainanee <[email protected]> Co-authored-by: Sai Shree Pradhan <[email protected]> * Introduce Backend Interface (DatabricksClient) (#573) NOTE: the `test_complex_types` e2e test was not working at the time of this merge. The test must be triggered when the test is back up and running as intended. * remove excess logs, assertions, instantiations large merge artifacts Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) + remove excess log (merge artifact) Signed-off-by: varun-edachali-dbx <[email protected]> * fix typing Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary check Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary replace call Signed-off-by: varun-edachali-dbx <[email protected]> * introduce __str__ methods for CommandId and SessionId Signed-off-by: varun-edachali-dbx <[email protected]> * docstrings for DatabricksClient interface Signed-off-by: varun-edachali-dbx <[email protected]> * stronger typing of Cursor and ExecuteResponse Signed-off-by: varun-edachali-dbx <[email protected]> * remove utility functions from backend interface, fix circular import Signed-off-by: varun-edachali-dbx <[email protected]> * rename info to properties Signed-off-by: varun-edachali-dbx <[email protected]> * newline for cleanliness Signed-off-by: varun-edachali-dbx <[email protected]> * fix circular import Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * to_hex_id -> get_hex_id Signed-off-by: varun-edachali-dbx <[email protected]> * better comment on protocol version getter Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * move guid to hex id to new utils module Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * move staging allowed local path to connection props Signed-off-by: varun-edachali-dbx <[email protected]> * add strong return type for execute_command Signed-off-by: varun-edachali-dbx <[email protected]> * skip auth, error handling in databricksclient interface Signed-off-by: varun-edachali-dbx <[email protected]> * chore: docstring + line width Signed-off-by: varun-edachali-dbx <[email protected]> * get_id -> get_guid Signed-off-by: varun-edachali-dbx <[email protected]> * chore: docstring Signed-off-by: varun-edachali-dbx <[email protected]> * fix: to_hex_id -> to_hex_guid Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> * Implement ResultSet Abstraction (backend interfaces for fetch phase) (#574) * ensure backend client returns a ResultSet type in backend tests Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * newline for cleanliness Signed-off-by: varun-edachali-dbx <[email protected]> * fix circular import Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * to_hex_id -> get_hex_id Signed-off-by: varun-edachali-dbx <[email protected]> * better comment on protocol version getter Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * stricter typing for cursor Signed-off-by: varun-edachali-dbx <[email protected]> * correct typing Signed-off-by: varun-edachali-dbx <[email protected]> * correct tests and merge artifacts Signed-off-by: varun-edachali-dbx <[email protected]> * remove accidentally modified workflow files remnants of old merge Signed-off-by: varun-edachali-dbx <[email protected]> * chore: remove accidentally modified workflow files Signed-off-by: varun-edachali-dbx <[email protected]> * add back accidentally removed docstrings Signed-off-by: varun-edachali-dbx <[email protected]> * clean up docstrings Signed-off-by: varun-edachali-dbx <[email protected]> * log hex Signed-off-by: varun-edachali-dbx <[email protected]> * remove unnecessary _replace call Signed-off-by: varun-edachali-dbx <[email protected]> * add __str__ for CommandId Signed-off-by: varun-edachali-dbx <[email protected]> * take TOpenSessionResp in get_protocol_version to maintain existing interface Signed-off-by: varun-edachali-dbx <[email protected]> * active_op_handle -> active_mmand_id Signed-off-by: varun-edachali-dbx <[email protected]> * ensure None returned for close_command Signed-off-by: varun-edachali-dbx <[email protected]> * account for ResultSet return in new pydocs Signed-off-by: varun-edachali-dbx <[email protected]> * pydoc for types Signed-off-by: varun-edachali-dbx <[email protected]> * move common state to ResultSet aprent Signed-off-by: varun-edachali-dbx <[email protected]> * stronger typing in resultSet behaviour Signed-off-by: varun-edachali-dbx <[email protected]> * remove redundant patch in test Signed-off-by: varun-edachali-dbx <[email protected]> * add has_been_closed_server_side assertion Signed-off-by: varun-edachali-dbx <[email protected]> * remove redundancies in tests Signed-off-by: varun-edachali-dbx <[email protected]> * more robust close check Signed-off-by: varun-edachali-dbx <[email protected]> * use normalised state in e2e test Signed-off-by: varun-edachali-dbx <[email protected]> * simplify corrected test Signed-off-by: varun-edachali-dbx <[email protected]> * add line gaps after multi-line pydocs for consistency Signed-off-by: varun-edachali-dbx <[email protected]> * use normalised CommandState type in ExecuteResponse Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> * Introduce Sea HTTP Client and test script (#583) * introduce http client (temp) and sea test file Signed-off-by: varun-edachali-dbx <[email protected]> * reduce verbosity Signed-off-by: varun-edachali-dbx <[email protected]> * redundant comment Signed-off-by: varun-edachali-dbx <[email protected]> * reduce redundancy, params and data separate Signed-off-by: varun-edachali-dbx <[email protected]> * rename client Signed-off-by: varun-edachali-dbx <[email protected]> * fix type issues Signed-off-by: varun-edachali-dbx <[email protected]> * reduce repetition in request calls Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary elifs Signed-off-by: varun-edachali-dbx <[email protected]> * add newline at EOF Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> * Introduce `SeaDatabricksClient` (Session Implementation) (#582) * [squashed from prev branch] introduce sea client with session open and close functionality Signed-off-by: varun-edachali-dbx <[email protected]> * remove accidental changes to workflows (merge artifacts) Signed-off-by: varun-edachali-dbx <[email protected]> * pass test_input to get_protocol_version instead of session_id to maintain previous API Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black + line gaps after multi-line pydocs) Signed-off-by: varun-edachali-dbx <[email protected]> * use factory for backend instantiation Signed-off-by: varun-edachali-dbx <[email protected]> * fix type issues Signed-off-by: varun-edachali-dbx <[email protected]> * remove redundant comments Signed-off-by: varun-edachali-dbx <[email protected]> * introduce models for requests and responses Signed-off-by: varun-edachali-dbx <[email protected]> * remove http client and test script to prevent diff from showing up post http-client merge Signed-off-by: varun-edachali-dbx <[email protected]> * Introduce Sea HTTP Client and test script (#583) * introduce http client (temp) and sea test file Signed-off-by: varun-edachali-dbx <[email protected]> * reduce verbosity Signed-off-by: varun-edachali-dbx <[email protected]> * redundant comment Signed-off-by: varun-edachali-dbx <[email protected]> * reduce redundancy, params and data separate Signed-off-by: varun-edachali-dbx <[email protected]> * rename client Signed-off-by: varun-edachali-dbx <[email protected]> * fix type issues Signed-off-by: varun-edachali-dbx <[email protected]> * reduce repetition in request calls Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary elifs Signed-off-by: varun-edachali-dbx <[email protected]> * add newline at EOF Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> * CustomHttpClient -> SeaHttpClient Signed-off-by: varun-edachali-dbx <[email protected]> * redundant comment in backend client Signed-off-by: varun-edachali-dbx <[email protected]> * regex for warehouse_id instead of .split, remove excess imports and behaviour Signed-off-by: varun-edachali-dbx <[email protected]> * remove redundant attributes Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * [nit] reduce nested code Signed-off-by: varun-edachali-dbx <[email protected]> * line gap after multi-line pydoc Signed-off-by: varun-edachali-dbx <[email protected]> * redundant imports Signed-off-by: varun-edachali-dbx <[email protected]> * move sea backend and models into separate sea/ dir Signed-off-by: varun-edachali-dbx <[email protected]> * move http client into separate sea/ dir Signed-off-by: varun-edachali-dbx <[email protected]> * change commands to include ones in docs Signed-off-by: varun-edachali-dbx <[email protected]> * add link to sql-ref-parameters for session-confs Signed-off-by: varun-edachali-dbx <[email protected]> * add client side filtering for session confs, add note on warehouses over endoints Signed-off-by: varun-edachali-dbx <[email protected]> * test unimplemented methods and max_download_threads prop Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> * Normalise Execution Response (clean backend interfaces) (#587) * [squash from exec-sea] bring over execution phase changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess test Signed-off-by: varun-edachali-dbx <[email protected]> * add docstring Signed-off-by: varun-edachali-dbx <[email protected]> * remvoe exec func in sea backend Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess files Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess models Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess sea backend tests Signed-off-by: varun-edachali-dbx <[email protected]> * cleanup Signed-off-by: varun-edachali-dbx <[email protected]> * re-introduce get_schema_desc Signed-off-by: varun-edachali-dbx <[email protected]> * remove SeaResultSet Signed-off-by: varun-edachali-dbx <[email protected]> * clean imports and attributes Signed-off-by: varun-edachali-dbx <[email protected]> * pass CommandId to ExecResp Signed-off-by: varun-edachali-dbx <[email protected]> * remove changes in types Signed-off-by: varun-edachali-dbx <[email protected]> * add back essential types (ExecResponse, from_sea_state) Signed-off-by: varun-edachali-dbx <[email protected]> * fix fetch types Signed-off-by: varun-edachali-dbx <[email protected]> * excess imports Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff by maintaining logs Signed-off-by: varun-edachali-dbx <[email protected]> * fix int test types Signed-off-by: varun-edachali-dbx <[email protected]> * move guid_to_hex_id import to utils Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff in guid utils import Signed-off-by: varun-edachali-dbx <[email protected]> * move arrow_schema_bytes back into ExecuteResult Signed-off-by: varun-edachali-dbx <[email protected]> * maintain log Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary assignment Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary tuple response Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-ncessary verbose mocking Signed-off-by: varun-edachali-dbx <[email protected]> * move Queue construction to ResultSert Signed-off-by: varun-edachali-dbx <[email protected]> * move description to List[Tuple] Signed-off-by: varun-edachali-dbx <[email protected]> * frmatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff (remove explicit tuple conversion) Signed-off-by: varun-edachali-dbx <[email protected]> * remove has_more_rows from ExecuteResponse Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary has_more_rows aclc Signed-off-by: varun-edachali-dbx <[email protected]> * default has_more_rows to True Signed-off-by: varun-edachali-dbx <[email protected]> * return has_more_rows from ExecResponse conversion during GetRespMetadata Signed-off-by: varun-edachali-dbx <[email protected]> * remove unnecessary replacement Signed-off-by: varun-edachali-dbx <[email protected]> * better mocked backend naming Signed-off-by: varun-edachali-dbx <[email protected]> * remove has_more_rows test in ExecuteResponse Signed-off-by: varun-edachali-dbx <[email protected]> * introduce replacement of original has_more_rows read test Signed-off-by: varun-edachali-dbx <[email protected]> * call correct method in test_use_arrow_schema Signed-off-by: varun-edachali-dbx <[email protected]> * call correct method in test_fall_back_to_hive_schema Signed-off-by: varun-edachali-dbx <[email protected]> * re-introduce result response read test Signed-off-by: varun-edachali-dbx <[email protected]> * simplify test Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess fetch_results mocks Signed-off-by: varun-edachali-dbx <[email protected]> * more minimal changes to thrift_backend tests Signed-off-by: varun-edachali-dbx <[email protected]> * move back to old table types Signed-off-by: varun-edachali-dbx <[email protected]> * remove outdated arrow_schema_bytes return Signed-off-by: varun-edachali-dbx <[email protected]> * remove duplicate import Signed-off-by: varun-edachali-dbx <[email protected]> * rephrase model docstrings to explicitly denote that they are representations and not used over the wire Signed-off-by: varun-edachali-dbx <[email protected]> * has_more_rows -> is_direct_results Signed-off-by: varun-edachali-dbx <[email protected]> * switch docstring format to align with Connection class Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> * Introduce models for `SeaDatabricksClient` (#595) * [squash from exec-sea] bring over execution phase changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess test Signed-off-by: varun-edachali-dbx <[email protected]> * add docstring Signed-off-by: varun-edachali-dbx <[email protected]> * remvoe exec func in sea backend Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess files Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess models Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess sea backend tests Signed-off-by: varun-edachali-dbx <[email protected]> * cleanup Signed-off-by: varun-edachali-dbx <[email protected]> * re-introduce get_schema_desc Signed-off-by: varun-edachali-dbx <[email protected]> * remove SeaResultSet Signed-off-by: varun-edachali-dbx <[email protected]> * clean imports and attributes Signed-off-by: varun-edachali-dbx <[email protected]> * pass CommandId to ExecResp Signed-off-by: varun-edachali-dbx <[email protected]> * remove changes in types Signed-off-by: varun-edachali-dbx <[email protected]> * add back essential types (ExecResponse, from_sea_state) Signed-off-by: varun-edachali-dbx <[email protected]> * fix fetch types Signed-off-by: varun-edachali-dbx <[email protected]> * excess imports Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff by maintaining logs Signed-off-by: varun-edachali-dbx <[email protected]> * fix int test types Signed-off-by: varun-edachali-dbx <[email protected]> * [squashed from exec-sea] init execution func Signed-off-by: varun-edachali-dbx <[email protected]> * remove irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove ResultSetFilter functionality Signed-off-by: varun-edachali-dbx <[email protected]> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * even more irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove sea response as init option Signed-off-by: varun-edachali-dbx <[email protected]> * exec test example scripts Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * [squashed from sea-exec] merge sea stuffs Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess removed docstring Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess changes in backend Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess imports Signed-off-by: varun-edachali-dbx <[email protected]> * remove accidentally removed _get_schema_desc Signed-off-by: varun-edachali-dbx <[email protected]> * remove unnecessary init with sea_response tests Signed-off-by: varun-edachali-dbx <[email protected]> * rmeove unnecessary changes Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * improved models and filters from cloudfetch-sea branch Signed-off-by: varun-edachali-dbx <[email protected]> * filters stuff (align with JDBC) Signed-off-by: varun-edachali-dbx <[email protected]> * backend from cloudfetch-sea Signed-off-by: varun-edachali-dbx <[email protected]> * remove filtering, metadata ops Signed-off-by: varun-edachali-dbx <[email protected]> * raise NotImplementedErrror for metadata ops Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary backend cahnges Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-needed GetChunksResponse only relevant in Fetch phase Signed-off-by: varun-edachali-dbx <[email protected]> * reduce code duplication Signed-off-by: varun-edachali-dbx <[email protected]> * more clear docstrings Signed-off-by: varun-edachali-dbx <[email protected]> * introduce strongly typed ChunkInfo Signed-off-by: varun-edachali-dbx <[email protected]> * remove is_volume_operation from response Signed-off-by: varun-edachali-dbx <[email protected]> * add is_volume_op and more ResultData fields Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> * Introduce preliminary SEA Result Set (#588) * [squash from exec-sea] bring over execution phase changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess test Signed-off-by: varun-edachali-dbx <[email protected]> * add docstring Signed-off-by: varun-edachali-dbx <[email protected]> * remvoe exec func in sea backend Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess files Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess models Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess sea backend tests Signed-off-by: varun-edachali-dbx <[email protected]> * cleanup Signed-off-by: varun-edachali-dbx <[email protected]> * re-introduce get_schema_desc Signed-off-by: varun-edachali-dbx <[email protected]> * remove SeaResultSet Signed-off-by: varun-edachali-dbx <[email protected]> * clean imports and attributes Signed-off-by: varun-edachali-dbx <[email protected]> * pass CommandId to ExecResp Signed-off-by: varun-edachali-dbx <[email protected]> * remove changes in types Signed-off-by: varun-edachali-dbx <[email protected]> * add back essential types (ExecResponse, from_sea_state) Signed-off-by: varun-edachali-dbx <[email protected]> * fix fetch types Signed-off-by: varun-edachali-dbx <[email protected]> * excess imports Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff by maintaining logs Signed-off-by: varun-edachali-dbx <[email protected]> * fix int test types Signed-off-by: varun-edachali-dbx <[email protected]> * [squashed from exec-sea] init execution func Signed-off-by: varun-edachali-dbx <[email protected]> * remove irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove ResultSetFilter functionality Signed-off-by: varun-edachali-dbx <[email protected]> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * even more irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove sea response as init option Signed-off-by: varun-edachali-dbx <[email protected]> * move guid_to_hex_id import to utils Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff in guid utils import Signed-off-by: varun-edachali-dbx <[email protected]> * move arrow_schema_bytes back into ExecuteResult Signed-off-by: varun-edachali-dbx <[email protected]> * maintain log Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary assignment Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary tuple response Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-ncessary verbose mocking Signed-off-by: varun-edachali-dbx <[email protected]> * move Queue construction to ResultSert Signed-off-by: varun-edachali-dbx <[email protected]> * move description to List[Tuple] Signed-off-by: varun-edachali-dbx <[email protected]> * frmatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff (remove explicit tuple conversion) Signed-off-by: varun-edachali-dbx <[email protected]> * remove has_more_rows from ExecuteResponse Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary has_more_rows aclc Signed-off-by: varun-edachali-dbx <[email protected]> * default has_more_rows to True Signed-off-by: varun-edachali-dbx <[email protected]> * return has_more_rows from ExecResponse conversion during GetRespMetadata Signed-off-by: varun-edachali-dbx <[email protected]> * remove unnecessary replacement Signed-off-by: varun-edachali-dbx <[email protected]> * better mocked backend naming Signed-off-by: varun-edachali-dbx <[email protected]> * remove has_more_rows test in ExecuteResponse Signed-off-by: varun-edachali-dbx <[email protected]> * introduce replacement of original has_more_rows read test Signed-off-by: varun-edachali-dbx <[email protected]> * call correct method in test_use_arrow_schema Signed-off-by: varun-edachali-dbx <[email protected]> * call correct method in test_fall_back_to_hive_schema Signed-off-by: varun-edachali-dbx <[email protected]> * re-introduce result response read test Signed-off-by: varun-edachali-dbx <[email protected]> * simplify test Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess fetch_results mocks Signed-off-by: varun-edachali-dbx <[email protected]> * more minimal changes to thrift_backend tests Signed-off-by: varun-edachali-dbx <[email protected]> * move back to old table types Signed-off-by: varun-edachali-dbx <[email protected]> * remove outdated arrow_schema_bytes return Signed-off-by: varun-edachali-dbx <[email protected]> * align SeaResultSet with new structure Signed-off-by: varun-edachali-dbx <[email protected]> * correct sea res set tests Signed-off-by: varun-edachali-dbx <[email protected]> * remove duplicate import Signed-off-by: varun-edachali-dbx <[email protected]> * rephrase model docstrings to explicitly denote that they are representations and not used over the wire Signed-off-by: varun-edachali-dbx <[email protected]> * has_more_rows -> is_direct_results Signed-off-by: varun-edachali-dbx <[email protected]> * switch docstring format to align with Connection class Signed-off-by: varun-edachali-dbx <[email protected]> * has_more_rows -> is_direct_results Signed-off-by: varun-edachali-dbx <[email protected]> * fix type errors with arrow_schema_bytes Signed-off-by: varun-edachali-dbx <[email protected]> * spaces after multi line pydocs Signed-off-by: varun-edachali-dbx <[email protected]> * remove duplicate queue init (merge artifact) Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff (remove newlines) Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary changes covered by #588 anyway Signed-off-by: varun-edachali-dbx <[email protected]> * Revert "remove un-necessary changes" This reverts commit a70a6cee277db44d6951604e890f91cae9f92f32. Signed-off-by: varun-edachali-dbx <[email protected]> * b"" -> None Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> * remove invalid ExecuteResponse import Signed-off-by: varun-edachali-dbx <[email protected]> * Separate Session related functionality from Connection class (#571) * decouple session class from existing Connection ensure maintenance of current APIs of Connection while delegating responsibility Signed-off-by: varun-edachali-dbx <[email protected]> * add open property to Connection to ensure maintenance of existing API Signed-off-by: varun-edachali-dbx <[email protected]> * update unit tests to address ThriftBackend through session instead of through Connection Signed-off-by: varun-edachali-dbx <[email protected]> * chore: move session specific tests from test_client to test_session Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) as in CONTRIBUTING.md Signed-off-by: varun-edachali-dbx <[email protected]> * use connection open property instead of long chain through session Signed-off-by: varun-edachali-dbx <[email protected]> * trigger integration workflow Signed-off-by: varun-edachali-dbx <[email protected]> * fix: ensure open attribute of Connection never fails in case the openSession takes long, the initialisation of the session will not complete immediately. This could make the session attribute inaccessible. If the Connection is deleted in this time, the open() check will throw because the session attribute does not exist. Thus, we default to the Connection being closed in this case. This was not an issue before because open was a direct attribute of the Connection class. Caught in the integration tests. Signed-off-by: varun-edachali-dbx <[email protected]> * fix: de-complicate earlier connection open logic earlier, one of the integration tests was failing because 'session was not an attribute of Connection'. This is likely tied to a local configuration issue related to unittest that was causing an error in the test suite itself. The tests are now passing without checking for the session attribute. https://github.com/databricks/databricks-sql-python/pull/567/commits/c676f9b0281cc3e4fe9c6d8216cc62fc75eade3b Signed-off-by: varun-edachali-dbx <[email protected]> * Revert "fix: de-complicate earlier connection open logic" This reverts commit d6b1b196c98a6e9d8e593a88c34bbde010519ef4. Signed-off-by: varun-edachali-dbx <[email protected]> * [empty commit] attempt to trigger ci e2e workflow Signed-off-by: varun-edachali-dbx <[email protected]> * Update CODEOWNERS (#562) new codeowners Signed-off-by: varun-edachali-dbx <[email protected]> * Enhance Cursor close handling and context manager exception management to prevent server side resource leaks (#554) * Enhance Cursor close handling and context manager exception management * tests * fmt * Fix Cursor.close() to properly handle CursorAlreadyClosedError * Remove specific test message from Cursor.close() error handling * Improve error handling in connection and cursor context managers to ensure proper closure during exceptions, including KeyboardInterrupt. Add tests for nested cursor management and verify operation closure on server-side errors. * add * add Signed-off-by: varun-edachali-dbx <[email protected]> * PECOBLR-86 improve logging on python driver (#556) * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <[email protected]> * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <[email protected]> * fixed format Signed-off-by: Sai Shree Pradhan <[email protected]> * used lazy logging Signed-off-by: Sai Shree Pradhan <[email protected]> * changed debug to error logs Signed-off-by: Sai Shree Pradhan <[email protected]> * used lazy logging Signed-off-by: Sai Shree Pradhan <[email protected]> --------- Signed-off-by: Sai Shree Pradhan <[email protected]> Signed-off-by: varun-edachali-dbx <[email protected]> * Revert "Merge remote-tracking branch 'upstream/sea-migration' into decouple-session" This reverts commit dbb2ec52306b91072a2ee842270c7113aece9aff, reversing changes made to 7192f117279d4f0adcbafcdf2238c18663324515. Signed-off-by: varun-edachali-dbx <[email protected]> * Reapply "Merge remote-tracking branch 'upstream/sea-migration' into decouple-session" This reverts commit bdb83817f49e1d88a01679b11da8e55e8e80b42f. Signed-off-by: varun-edachali-dbx <[email protected]> * fix: separate session opening logic from instantiation ensures correctness of self.session.open call in Connection Signed-off-by: varun-edachali-dbx <[email protected]> * fix: use is_open attribute to denote session availability Signed-off-by: varun-edachali-dbx <[email protected]> * fix: access thrift backend through session Signed-off-by: varun-edachali-dbx <[email protected]> * chore: use get_handle() instead of private session attribute in client Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * fix: remove accidentally removed assertions Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> Signed-off-by: Sai Shree Pradhan <[email protected]> Co-authored-by: Jothi Prakash <[email protected]> Co-authored-by: Madhav Sainanee <[email protected]> Co-authored-by: Sai Shree Pradhan <[email protected]> * Introduce Backend Interface (DatabricksClient) (#573) NOTE: the `test_complex_types` e2e test was not working at the time of this merge. The test must be triggered when the test is back up and running as intended. * remove excess logs, assertions, instantiations large merge artifacts Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) + remove excess log (merge artifact) Signed-off-by: varun-edachali-dbx <[email protected]> * fix typing Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary check Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary replace call Signed-off-by: varun-edachali-dbx <[email protected]> * introduce __str__ methods for CommandId and SessionId Signed-off-by: varun-edachali-dbx <[email protected]> * docstrings for DatabricksClient interface Signed-off-by: varun-edachali-dbx <[email protected]> * stronger typing of Cursor and ExecuteResponse Signed-off-by: varun-edachali-dbx <[email protected]> * remove utility functions from backend interface, fix circular import Signed-off-by: varun-edachali-dbx <[email protected]> * rename info to properties Signed-off-by: varun-edachali-dbx <[email protected]> * newline for cleanliness Signed-off-by: varun-edachali-dbx <[email protected]> * fix circular import Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * to_hex_id -> get_hex_id Signed-off-by: varun-edachali-dbx <[email protected]> * better comment on protocol version getter Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * move guid to hex id to new utils module Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * move staging allowed local path to connection props Signed-off-by: varun-edachali-dbx <[email protected]> * add strong return type for execute_command Signed-off-by: varun-edachali-dbx <[email protected]> * skip auth, error handling in databricksclient interface Signed-off-by: varun-edachali-dbx <[email protected]> * chore: docstring + line width Signed-off-by: varun-edachali-dbx <[email protected]> * get_id -> get_guid Signed-off-by: varun-edachali-dbx <[email protected]> * chore: docstring Signed-off-by: varun-edachali-dbx <[email protected]> * fix: to_hex_id -> to_hex_guid Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> * Implement ResultSet Abstraction (backend interfaces for fetch phase) (#574) * ensure backend client returns a ResultSet type in backend tests Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * newline for cleanliness Signed-off-by: varun-edachali-dbx <[email protected]> * fix circular import Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * to_hex_id -> get_hex_id Signed-off-by: varun-edachali-dbx <[email protected]> * better comment on protocol version getter Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * stricter typing for cursor Signed-off-by: varun-edachali-dbx <[email protected]> * correct typing Signed-off-by: varun-edachali-dbx <[email protected]> * correct tests and merge artifacts Signed-off-by: varun-edachali-dbx <[email protected]> * remove accidentally modified workflow files remnants of old merge Signed-off-by: varun-edachali-dbx <[email protected]> * chore: remove accidentally modified workflow files Signed-off-by: varun-edachali-dbx <[email protected]> * add back accidentally removed docstrings Signed-off-by: varun-edachali-dbx <[email protected]> * clean up docstrings Signed-off-by: varun-edachali-dbx <[email protected]> * log hex Signed-off-by: varun-edachali-dbx <[email protected]> * remove unnecessary _replace call Signed-off-by: varun-edachali-dbx <[email protected]> * add __str__ for CommandId Signed-off-by: varun-edachali-dbx <[email protected]> * take TOpenSessionResp in get_protocol_version to maintain existing interface Signed-off-by: varun-edachali-dbx <[email protected]> * active_op_handle -> active_mmand_id Signed-off-by: varun-edachali-dbx <[email protected]> * ensure None returned for close_command Signed-off-by: varun-edachali-dbx <[email protected]> * account for ResultSet return in new pydocs Signed-off-by: varun-edachali-dbx <[email protected]> * pydoc for types Signed-off-by: varun-edachali-dbx <[email protected]> * move common state to ResultSet aprent Signed-off-by: varun-edachali-dbx <[email protected]> * stronger typing in resultSet behaviour Signed-off-by: varun-edachali-dbx <[email protected]> * remove redundant patch in test Signed-off-by: varun-edachali-dbx <[email protected]> * add has_been_closed_server_side assertion Signed-off-by: varun-edachali-dbx <[email protected]> * remove redundancies in tests Signed-off-by: varun-edachali-dbx <[email protected]> * more robust close check Signed-off-by: varun-edachali-dbx <[email protected]> * use normalised state in e2e test Signed-off-by: varun-edachali-dbx <[email protected]> * simplify corrected test Signed-off-by: varun-edachali-dbx <[email protected]> * add line gaps after multi-line pydocs for consistency Signed-off-by: varun-edachali-dbx <[email protected]> * use normalised CommandState type in ExecuteResponse Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary initialisation assertions Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary line break s Signed-off-by: varun-edachali-dbx <[email protected]> * more un-necessary line breaks Signed-off-by: varun-edachali-dbx <[email protected]> * constrain diff of test_closing_connection_closes_commands Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff of test_closing_connection_closes_commands Signed-off-by: varun-edachali-dbx <[email protected]> * use pytest-like assertions for test_closing_connection_closes_commands Signed-off-by: varun-edachali-dbx <[email protected]> * ensure command_id is not None Signed-off-by: varun-edachali-dbx <[email protected]> * line breaks after multi-line pyfocs Signed-off-by: varun-edachali-dbx <[email protected]> * ensure non null operationHandle for commandId creation Signed-off-by: varun-edachali-dbx <[email protected]> * use command_id methods instead of explicit guid_to_hex_id conversion Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary artifacts in test_session, add back assertion Signed-off-by: varun-edachali-dbx <[email protected]> * Implement SeaDatabricksClient (Complete Execution Spec) (#590) * [squash from exec-sea] bring over execution phase changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess test Signed-off-by: varun-edachali-dbx <[email protected]> * add docstring Signed-off-by: varun-edachali-dbx <[email protected]> * remvoe exec func in sea backend Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess files Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess models Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess sea backend tests Signed-off-by: varun-edachali-dbx <[email protected]> * cleanup Signed-off-by: varun-edachali-dbx <[email protected]> * re-introduce get_schema_desc Signed-off-by: varun-edachali-dbx <[email protected]> * remove SeaResultSet Signed-off-by: varun-edachali-dbx <[email protected]> * clean imports and attributes Signed-off-by: varun-edachali-dbx <[email protected]> * pass CommandId to ExecResp Signed-off-by: varun-edachali-dbx <[email protected]> * remove changes in types Signed-off-by: varun-edachali-dbx <[email protected]> * add back essential types (ExecResponse, from_sea_state) Signed-off-by: varun-edachali-dbx <[email protected]> * fix fetch types Signed-off-by: varun-edachali-dbx <[email protected]> * excess imports Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff by maintaining logs Signed-off-by: varun-edachali-dbx <[email protected]> * fix int test types Signed-off-by: varun-edachali-dbx <[email protected]> * [squashed from exec-sea] init execution func Signed-off-by: varun-edachali-dbx <[email protected]> * remove irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove ResultSetFilter functionality Signed-off-by: varun-edachali-dbx <[email protected]> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * even more irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove sea response as init option Signed-off-by: varun-edachali-dbx <[email protected]> * exec test example scripts Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * [squashed from sea-exec] merge sea stuffs Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess removed docstring Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess changes in backend Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess imports Signed-off-by: varun-edachali-dbx <[email protected]> * remove accidentally removed _get_schema_desc Signed-off-by: varun-edachali-dbx <[email protected]> * remove unnecessary init with sea_response tests Signed-off-by: varun-edachali-dbx <[email protected]> * rmeove unnecessary changes Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * improved models and filters from cloudfetch-sea branch Signed-off-by: varun-edachali-dbx <[email protected]> * filters stuff (align with JDBC) Signed-off-by: varun-edachali-dbx <[email protected]> * backend from cloudfetch-sea Signed-off-by: varun-edachali-dbx <[email protected]> * remove filtering, metadata ops Signed-off-by: varun-edachali-dbx <[email protected]> * raise NotImplementedErrror for metadata ops Signed-off-by: varun-edachali-dbx <[email protected]> * change to valid table name Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary changes covered by #588 Signed-off-by: varun-edachali-dbx <[email protected]> * simplify test module Signed-off-by: varun-edachali-dbx <[email protected]> * logging -> debug level Signed-off-by: varun-edachali-dbx <[email protected]> * change table name in log Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary backend cahnges Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-needed GetChunksResponse Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-needed GetChunksResponse only relevant in Fetch phase Signed-off-by: varun-edachali-dbx <[email protected]> * reduce code duplication in response parsing Signed-off-by: varun-edachali-dbx <[email protected]> * reduce code duplication Signed-off-by: varun-edachali-dbx <[email protected]> * more clear docstrings Signed-off-by: varun-edachali-dbx <[email protected]> * introduce strongly typed ChunkInfo Signed-off-by: varun-edachali-dbx <[email protected]> * remove is_volume_operation from response Signed-off-by: varun-edachali-dbx <[email protected]> * add is_volume_op and more ResultData fields Signed-off-by: varun-edachali-dbx <[email protected]> * add test scripts Signed-off-by: varun-edachali-dbx <[email protected]> * Revert "Merge branch 'exec-models-sea' into exec-phase-sea" This reverts commit be1997e0d6b6cf0f5499db2381971ec3a015a2f7, reversing changes made to 37813ba6d1fe06d7f9f10d510a059b88dc552496. * change logging level Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove _get_schema_bytes (for now) Signed-off-by: varun-edachali-dbx <[email protected]> * redundant comments Signed-off-by: varun-edachali-dbx <[email protected]> * remove fetch phase methods Signed-off-by: varun-edachali-dbx <[email protected]> * reduce code repetititon + introduce gaps after multi line pydocs Signed-off-by: varun-edachali-dbx <[email protected]> * remove unused imports Signed-off-by: varun-edachali-dbx <[email protected]> * move description extraction to helper func Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * add more unit tests Signed-off-by: varun-edachali-dbx <[email protected]> * streamline unit tests Signed-off-by: varun-edachali-dbx <[email protected]> * test getting the list of allowed configurations Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff Signed-off-by: varun-edachali-dbx <[email protected]> * house constants in enums for readability and immutability Signed-off-by: varun-edachali-dbx <[email protected]> * add note on hybrid disposition Signed-off-by: varun-edachali-dbx <[email protected]> * remove redundant note on arrow_schema_bytes Signed-off-by: varun-edachali-dbx <[email protected]> * remove invalid import Signed-off-by: varun-edachali-dbx <[email protected]> * add strong typing for manifest in _extract_description Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary column skipping Signed-off-by: varun-edachali-dbx <[email protected]> * remove parsing in backend Signed-off-by: varun-edachali-dbx <[email protected]> * fix: convert sea statement id to CommandId type Signed-off-by: varun-edachali-dbx <[email protected]> * make polling interval a separate constant Signed-off-by: varun-edachali-dbx <[email protected]> * align state checking with Thrift implementation Signed-off-by: varun-edachali-dbx <[email protected]> * update unit tests according to changes Signed-off-by: varun-edachali-dbx <[email protected]> * add unit tests for added methods Signed-off-by: varun-edachali-dbx <[email protected]> * add spec to description extraction docstring, add strong typing to params Signed-off-by: varun-edachali-dbx <[email protected]> * add strong typing for backend parameters arg Signed-off-by: varun-edachali-dbx <[email protected]> --------- Signed-off-by: varun-edachali-dbx <[email protected]> * add from __future__ import annotations to remove string literals around forward refs, remove some unused imports Signed-off-by: varun-edachali-dbx <[email protected]> * move docstring of DatabricksClient within class Signed-off-by: varun-edachali-dbx <[email protected]> * move ThriftResultSet import to top of file Signed-off-by: varun-edachali-dbx <[email protected]> * make backend/utils __init__ file empty Signed-off-by: varun-edachali-dbx <[email protected]> * use from __future__ import annotations to remove string literals around Cursor Signed-off-by: varun-edachali-dbx <[email protected]> * use lazy logging Signed-off-by: varun-edachali-dbx <[email protected]> * replace getters with property tag Signed-off-by: varun-edachali-dbx <[email protected]> * set active_command_id to None, not active_op_handle Signed-off-by: varun-edachali-dbx <[email protected]> * align test_session with pytest instead of unittest Signed-off-by: varun-edachali-dbx <[email protected]> * remove duplicate test, correct active_command_id attribute Signed-off-by: varun-edachali-dbx <[email protected]> * SeaDatabricksClient: Add Metadata Commands (#593) * [squash from exec-sea] bring over execution phase changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess test Signed-off-by: varun-edachali-dbx <[email protected]> * add docstring Signed-off-by: varun-edachali-dbx <[email protected]> * remvoe exec func in sea backend Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess files Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess models Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess sea backend tests Signed-off-by: varun-edachali-dbx <[email protected]> * cleanup Signed-off-by: varun-edachali-dbx <[email protected]> * re-introduce get_schema_desc Signed-off-by: varun-edachali-dbx <[email protected]> * remove SeaResultSet Signed-off-by: varun-edachali-dbx <[email protected]> * clean imports and attributes Signed-off-by: varun-edachali-dbx <[email protected]> * pass CommandId to ExecResp Signed-off-by: varun-edachali-dbx <[email protected]> * remove changes in types Signed-off-by: varun-edachali-dbx <[email protected]> * add back essential types (ExecResponse, from_sea_state) Signed-off-by: varun-edachali-dbx <[email protected]> * fix fetch types Signed-off-by: varun-edachali-dbx <[email protected]> * excess imports Signed-off-by: varun-edachali-dbx <[email protected]> * reduce diff by maintaining logs Signed-off-by: varun-edachali-dbx <[email protected]> * fix int test types Signed-off-by: varun-edachali-dbx <[email protected]> * [squashed from exec-sea] init execution func Signed-off-by: varun-edachali-dbx <[email protected]> * remove irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove ResultSetFilter functionality Signed-off-by: varun-edachali-dbx <[email protected]> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * even more irrelevant changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove sea response as init option Signed-off-by: varun-edachali-dbx <[email protected]> * exec test example scripts Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * [squashed from sea-exec] merge sea stuffs Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess removed docstring Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess changes in backend Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess imports Signed-off-by: varun-edachali-dbx <[email protected]> * remove accidentally removed _get_schema_desc Signed-off-by: varun-edachali-dbx <[email protected]> * remove unnecessary init with sea_response tests Signed-off-by: varun-edachali-dbx <[email protected]> * rmeove unnecessary changes Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * improved models and filters from cloudfetch-sea branch Signed-off-by: varun-edachali-dbx <[email protected]> * filters stuff (align with JDBC) Signed-off-by: varun-edachali-dbx <[email protected]> * backend from cloudfetch-sea Signed-off-by: varun-edachali-dbx <[email protected]> * remove filtering, metadata ops Signed-off-by: varun-edachali-dbx <[email protected]> * raise NotImplementedErrror for metadata ops Signed-off-by: varun-edachali-dbx <[email protected]> * add metadata commands Signed-off-by: varun-edachali-dbx <[email protected]> * formatting (black) Signed-off-by: varun-edachali-dbx <[email protected]> * add metadata command unit tests Signed-off-by: varun-edachali-dbx <[email protected]> * change to valid table name Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary changes covered by #588 Signed-off-by: varun-edachali-dbx <[email protected]> * simplify test module Signed-off-by: varun-edachali-dbx <[email protected]> * logging -> debug level Signed-off-by: varun-edachali-dbx <[email protected]> * change table name in log Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary backend cahnges Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-needed GetChunksResponse Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-needed GetChunksResponse only relevant in Fetch phase Signed-off-by: varun-edachali-dbx <[email protected]> * reduce code duplication in response parsing Signed-off-by: varun-edachali-dbx <[email protected]> * reduce code duplication Signed-off-by: varun-edachali-dbx <[email protected]> * more clear docstrings Signed-off-by: varun-edachali-dbx <[email protected]> * introduce strongly typed ChunkInfo Signed-off-by: varun-edachali-dbx <[email protected]> * remove is_volume_operation from response Signed-off-by: varun-edachali-dbx <[email protected]> * add is_volume_op and more ResultData fields Signed-off-by: varun-edachali-dbx <[email protected]> * add test scripts Signed-off-by: varun-edachali-dbx <[email protected]> * Revert "Merge branch 'sea-migration' into exec-models-sea" This reverts commit 8bd12d829ea13abf8fc1507fff8cb21751001c67, reversing changes made to 030edf8df3db487b7af8d910ee51240d1339229e. * Revert "Merge branch 'exec-models-sea' into exec-phase-sea" This reverts commit be1997e0d6b6cf0f5499db2381971ec3a015a2f7, reversing changes made to 37813ba6d1fe06d7f9f10d510a059b88dc552496. * change logging level Signed-off-by: varun-edachali-dbx <[email protected]> * remove un-necessary changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove excess changes Signed-off-by: varun-edachali-dbx <[email protected]> * remove _get_schema_bytes (for now) Signed-off-by: varun-edachali-dbx <[email protected]> * redundant comments …

* e2e test telemetry Signed-off-by: Sai Shree Pradhan <[email protected]> * assert session id, statement id Signed-off-by: Sai Shree Pradhan <[email protected]> * minor changes, added checks on server response Signed-off-by: Sai Shree Pradhan <[email protected]> * finally block Signed-off-by: Sai Shree Pradhan <[email protected]> * removed setup clean up Signed-off-by: Sai Shree Pradhan <[email protected]> * finally in test_complex_types Signed-off-by: Sai Shree Pradhan <[email protected]> --------- Signed-off-by: Sai Shree Pradhan <[email protected]>

Implement PEP 249-compliant transaction control with extensions for manual commit/rollback operations. This enables atomic multi-table operations with REPEATABLE_READ isolation semantics. Core API additions: - connection.autocommit property for enabling/disabling auto-commit mode - connection.commit() to commit active transactions - connection.rollback() to rollback active transactions - connection.get_transaction_isolation() returns current isolation level - connection.set_transaction_isolation() validates isolation level - TransactionError exception for transaction-specific failures Implementation details: - Added autocommit state caching in Session with optional server query - Added TRANSACTION_ISOLATION_LEVEL_REPEATABLE_READ constant - All transaction operations include proper error handling and telemetry - Supports fetch_autocommit_from_server connection parameter Testing: - Unit tests covering all transaction methods and error scenarios - e2e integration tests validating transaction behavior including multi-table atomicity, sequential transactions, and isolation semantics Documentation: - Comprehensive TRANSACTIONS.md guide with examples and best practices - Updated README.md with basic usage and reference to detailed docs Requires MST-enabled Databricks SQL warehouse and Delta tables with 'delta.feature.catalogOwned-preview' table property. --------- Signed-off-by: Jayant Singh <[email protected]>

Bump to version 4.2.0 Signed-off-by: Jayant Singh <[email protected]>

…abricks#711) Introduces a new `ignore_transactions` configuration parameter (default: True) to control transaction-related behavior in the Connection class. When ignore_transactions=True (default): - commit(): no-op, returns immediately - rollback(): raises NotSupportedError with message "Transactions are not supported on Databricks" - autocommit setter: no-op, returns immediately When ignore_transactions=False: - All transaction methods execute normally Changes: - Added ignore_transactions parameter to Connection.__init__() with default value True - Modified commit(), rollback(), and autocommit setter to check ignore_transactions flag - Updated unit tests to pass ignore_transactions=False when testing transaction functionality - Updated e2e transaction tests to pass ignore_transactions=False - Added three new unit tests to verify ignore_transactions

Signed-off-by: Vikrant Puppala <[email protected]>

This changes the default value of use_hybrid_disposition from True to False in the SEA backend, disabling hybrid disposition by default.

* Added driver connection params Signed-off-by: Nikhil Suri <[email protected]> * Added model fields for chunk/result latency Signed-off-by: Nikhil Suri <[email protected]> * fixed linting issues Signed-off-by: Nikhil Suri <[email protected]> * lint issue fixing Signed-off-by: Nikhil Suri <[email protected]> * circuit breaker changes using pybreaker Signed-off-by: Nikhil Suri <[email protected]> * Added interface layer top of http client to use circuit rbeaker Signed-off-by: Nikhil Suri <[email protected]> * Added test cases to validate ciruit breaker Signed-off-by: Nikhil Suri <[email protected]> * fixing broken tests Signed-off-by: Nikhil Suri <[email protected]> * fixed linting issues Signed-off-by: Nikhil Suri <[email protected]> * fixed failing test cases Signed-off-by: Nikhil Suri <[email protected]> * fixed urllib3 issue Signed-off-by: Nikhil Suri <[email protected]> * added more test cases for telemetry Signed-off-by: Nikhil Suri <[email protected]> * simplified CB config Signed-off-by: Nikhil Suri <[email protected]> * poetry lock Signed-off-by: Nikhil Suri <[email protected]> * fix minor issues & improvement Signed-off-by: Nikhil Suri <[email protected]> * improved circuit breaker for handling only 429/503 Signed-off-by: Nikhil Suri <[email protected]> * linting issue fixed Signed-off-by: Nikhil Suri <[email protected]> * raise CB only for 429/503 Signed-off-by: Nikhil Suri <[email protected]> * fix broken test cases Signed-off-by: Nikhil Suri <[email protected]> * fixed untyped references Signed-off-by: Nikhil Suri <[email protected]> * added more test to verify the changes Signed-off-by: Nikhil Suri <[email protected]> * description changed Signed-off-by: Nikhil Suri <[email protected]> * remove cb congig class to constants Signed-off-by: Nikhil Suri <[email protected]> * removed mocked reponse and use a new exlucded exception in CB Signed-off-by: Nikhil Suri <[email protected]> * fixed broken test Signed-off-by: Nikhil Suri <[email protected]> * added e2e test to verify circuit breaker Signed-off-by: Nikhil Suri <[email protected]> * lower log level for telemetry Signed-off-by: Nikhil Suri <[email protected]> * fixed broken test, removed tests on log assertions Signed-off-by: Nikhil Suri <[email protected]> * modified unit to reduce the noise and follow dry principle Signed-off-by: Nikhil Suri <[email protected]> --------- Signed-off-by: Nikhil Suri <[email protected]>

…ks#715) perf: Optimize telemetry latency logging to reduce overhead Optimizations implemented: 1. Eliminated extractor pattern - replaced wrapper classes with direct attribute access functions, removing object creation overhead 2. Added feature flag early exit - checks cached telemetry_enabled flag to skip heavy work when telemetry is disabled 3. Simplified code structure with early returns for better readability Signed-off-by: Samikshya Chand <[email protected]>

* basic e2e test for force telemetry verification Signed-off-by: Nikhil Suri <[email protected]> * Added more integration test scenarios Signed-off-by: Nikhil Suri <[email protected]> * default on telemetry + logs to investigate failing test Signed-off-by: Nikhil Suri <[email protected]> * fixed linting issue Signed-off-by: Nikhil Suri <[email protected]> * added more logs to identify server side flag evaluation Signed-off-by: Nikhil Suri <[email protected]> * remove unused logs Signed-off-by: Nikhil Suri <[email protected]> * fix broken test case for default enable telemetry Signed-off-by: Nikhil Suri <[email protected]> * redcude test length and made more reusable code Signed-off-by: Nikhil Suri <[email protected]> * removed telemetry e2e to daily single run Signed-off-by: Nikhil Suri <[email protected]> --------- Signed-off-by: Nikhil Suri <[email protected]>

…databricks#718) * feat: Implement host-level telemetry batching to reduce rate limiting Changes telemetry client architecture from per-session to per-host batching, matching the JDBC driver implementation. This reduces the number of HTTP requests to the telemetry endpoint and prevents rate limiting in test environments. Key changes: - Add _TelemetryClientHolder with reference counting for shared clients - Change TelemetryClientFactory to key clients by host_url instead of session_id - Add getHostUrlSafely() helper for defensive null handling - Update all callers (client.py, exc.py, latency_logger.py) to pass host_url Before: 100 connections to same host = 100 separate TelemetryClients After: 100 connections to same host = 1 shared TelemetryClient (refcount=100) This fixes rate limiting issues seen in e2e tests where 300+ parallel connections were overwhelming the telemetry endpoint with 429 errors. * chore: Change all telemetry logging to DEBUG level Reduces log noise by changing all telemetry-related log statements (info, warning, error) to debug level. Telemetry operations are background tasks and should not clutter logs with operational messages. Changes: - Circuit breaker state changes: info/warning -> debug - Telemetry send failures: error -> debug - All telemetry operations now consistently use debug level * chore: Fix remaining telemetry warning log to debug Changes remaining logger.warning in telemetry_push_client.py to debug level for consistency with other telemetry logging. * fix: Update tests to use host_url instead of session_id_hex - Update circuit breaker test to check logger.debug instead of logger.info - Replace all session_id_hex test parameters with host_url - Apply Black formatting to exc.py and telemetry_client.py This fixes test failures caused by the signature change from session_id_hex to host_url in the Error class and TelemetryClientFactory. * fix: Revert session_id_hex in tests for functions that still use it Only Error classes changed from session_id_hex to host_url. Other classes (TelemetryClient, ResultSetDownloadHandler, etc.) still use session_id_hex. Reverted: - test_telemetry.py: TelemetryClient and initialize_telemetry_client - test_downloader.py: ResultSetDownloadHandler - test_download_manager.py: ResultFileDownloadManager Kept as host_url: - test_client.py: Error class instantiation * fix: Update all Error raises and test calls to use host_url Changes: 1. client.py: Changed all error raises from session_id_hex to host_url - Connection class: session_id_hex=self.get_session_id_hex() -> host_url=self.session.host - Cursor class: session_id_hex=self.connection.get_session_id_hex() -> host_url=self.connection.session.host 2. test_telemetry.py: Updated get_telemetry_client() and close() calls - get_telemetry_client(session_id) -> get_telemetry_client(host_url) - close(session_id) -> close(host_url=host_url) 3. test_telemetry_push_client.py: Changed logger.warning to logger.debug - Updated test assertion to match debug logging level These changes complete the migration from session-level to host-level telemetry client management. * fix: Update thrift_backend.py to use host_url instead of session_id_hex Changes: 1. Added self._host attribute to store server_hostname 2. Updated all error raises to use host_url=self._host 3. Changed method signatures from session_id_hex to host_url: - _check_response_for_error - _hive_schema_to_arrow_schema - _col_to_description - _hive_schema_to_description - _check_direct_results_for_error 4. Updated all method calls to pass self._host instead of self._session_id_hex This completes the migration from session-level to host-level error reporting. * Fix Black formatting by adjusting fmt directive placement Moved the `# fmt: on` directive to the except block level instead of inside the if statement to resolve Black parsing confusion. * Fix telemetry feature flag tests to set mock session host The tests were failing because they called get_telemetry_client("test") but the mock session didn't have .host set, so the telemetry client was registered under a different key (likely None or MagicMock). This caused the factory to return NoopTelemetryClient instead of the expected client. Fixed by setting mock_session_instance.host = "test" in all three tests. * Add teardown_method to clear telemetry factory state between tests Without this cleanup, tests were sharing telemetry clients because they all used the same host key ("test"), causing test pollution. The first test would create an enabled client, and subsequent tests would reuse it even when they expected a disabled client. * Clear feature flag context cache in teardown to fix test pollution The FeatureFlagsContextFactory caches feature flag contexts per session, causing tests to share the same feature flag state. This resulted in the first test creating a context with telemetry enabled, and subsequent tests incorrectly reusing that enabled state even when they expected disabled. * fix: Access actual client from holder in flush worker The flush worker was calling _flush() on _TelemetryClientHolder objects instead of the actual TelemetryClient. Fixed by accessing holder.client before calling _flush(). Fixes AttributeError in e2e tests: '_TelemetryClientHolder' object has no attribute '_flush' * Clear telemetry client cache in e2e test teardown Added _clients.clear() to the teardown fixture to prevent telemetry clients from persisting across e2e tests, which was causing session ID pollution in test_concurrent_queries_sends_telemetry. * Pass session_id parameter to telemetry export methods With host-level telemetry batching, multiple connections share one TelemetryClient. Each client stores session_id_hex from the first connection that created it. This caused all subsequent connections' telemetry events to use the wrong session ID. Changes: - Modified telemetry export method signatures to accept optional session_id - Updated Connection.export_initial_telemetry_log() to pass session_id - Updated latency_logger.py export_latency_log() to pass session_id - Updated Error.__init__() to accept optional session_id_hex and pass it - Updated all error raises in Connection and Cursor to pass session_id_hex 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * Fix Black formatting in telemetry_client.py * Use 'test-host' instead of 'test' for mock host in telemetry tests * Replace test-session-id with test-host in test_client.py * Fix telemetry client lookup to use test-host in tests * Make session_id_hex keyword-only parameter in Error.__init__ --------- Co-authored-by: Claude <[email protected]>

* Prepare for a release with telemetry on by default Signed-off-by: samikshya-chand_data <[email protected]> * Make edits Signed-off-by: samikshya-chand_data <[email protected]> * Update version Signed-off-by: samikshya-chand_data <[email protected]> * Fix CHANGELOG formatting to match previous style Signed-off-by: samikshya-chand_data <[email protected]> * Fix telemetry e2e tests for default-enabled behavior - Update test expectations to reflect telemetry being enabled by default - Add feature flags cache cleanup in teardown to prevent state leakage between tests - This ensures each test runs with fresh feature flag state * Add wait after connection close for async telemetry submission * Remove debug logging from telemetry tests * Mark telemetry e2e tests as serial - must not run in parallel Root cause: Telemetry tests share host-level client across pytest-xdist workers, causing test isolation issues with patches. Tests pass serially but fail with -n auto. Solution: Add @pytest.mark.serial marker. CI needs to run these separately without -n auto. * Split test execution to run serial tests separately Telemetry e2e tests must run serially due to shared host-level telemetry client across pytest-xdist workers. Running with -n auto causes test isolation issues where futures aren't properly captured. Changes: - Run parallel tests with -m 'not serial' -n auto - Run serial tests with -m 'serial' without parallelization - Use --cov-append for serial tests to combine coverage - Mark telemetry e2e tests with @pytest.mark.serial - Update test expectations for default telemetry behavior - Add feature flags cache cleanup in test teardown * Mark telemetry e2e tests as serial - must not run in parallel The concurrent telemetry e2e test globally patches telemetry methods to capture events. When run in parallel with other tests via pytest-xdist, it captures telemetry events from other concurrent tests, causing assertion failures (expected 60 events, got 88). All telemetry e2e tests must run serially to avoid cross-test interference with the shared host-level telemetry client. --------- Signed-off-by: samikshya-chand_data <[email protected]>

* added pandas 2.3.3 support and tests for py 3.14 Signed-off-by: Sreekanth Vadigi <[email protected]> * generated poetry.lock Signed-off-by: Sreekanth Vadigi <[email protected]> * lz4 version update for py 3.14 Signed-off-by: Sreekanth Vadigi <[email protected]> * dependency selection based on py version Signed-off-by: Sreekanth Vadigi <[email protected]> * pyarrow version update for py 3.14 Signed-off-by: Sreekanth Vadigi <[email protected]> * poetry.lock with latest poetry version Signed-off-by: Sreekanth Vadigi <[email protected]> --------- Signed-off-by: Sreekanth Vadigi <[email protected]>

* pandas 2.3.3 support for py < 3.14 Signed-off-by: Sreekanth Vadigi <[email protected]> * poetry lock Signed-off-by: Sreekanth Vadigi <[email protected]> --------- Signed-off-by: Sreekanth Vadigi <[email protected]>

…icks#723) Fixed the exception handler calls close() on _TelemetryClientHolder objects instead of accessing the client inside them.

…ks#724) * created util method to normalise http protocol in http path Signed-off-by: Nikhil Suri <[email protected]> * Added impacted files using util method Signed-off-by: Nikhil Suri <[email protected]> * Fixed linting issues Signed-off-by: Nikhil Suri <[email protected]> * fixed broken test with mock host string Signed-off-by: Nikhil Suri <[email protected]> * mocked http client Signed-off-by: Nikhil Suri <[email protected]> * made case sensitive check in url utils Signed-off-by: Nikhil Suri <[email protected]> * linting issue resolved Signed-off-by: Nikhil Suri <[email protected]> * removed unnecessary md files Signed-off-by: Nikhil Suri <[email protected]> * made test readbale Signed-off-by: Nikhil Suri <[email protected]> * changes done in auth util as well as sea http Signed-off-by: Nikhil Suri <[email protected]> --------- Signed-off-by: Nikhil Suri <[email protected]>

New minor version release

* query tags telemetry Signed-off-by: Sreekanth Vadigi <[email protected]> * code linting fix Signed-off-by: Sreekanth Vadigi <[email protected]> --------- Signed-off-by: Sreekanth Vadigi <[email protected]>

…heck failures in the repo (databricks#735) * Fix 60 seconds delay in gov cloud connections * keep it simple :) * Add fix for krb error * pin poetry * Pin for publish flow too * Fix failing tests * Edit order for pypi * One last fix : pls work

…ycle management (databricks#734) * Fix databricks#729 and databricks#731: Telemetry lifecycle management Signed-off-by: Madhavendra Rathore <[email protected]> * Address review comments: revert timeout and telemetry_enabled changes Per reviewer feedback on PR databricks#734: 1. Revert timeout from 30s back to 900s (line 299) - Reviewer noted that with wait=False, timeout is not critical - The async nature and wait=False handle the exit speed 2. Revert telemetry_enabled parameter back to True (line 734) - Reviewer noted this is redundant given the early return - If enable_telemetry=False, we return early (line 729) - Line 734 only executes when enable_telemetry=True - Therefore using the parameter here is unnecessary These changes address the reviewer's valid technical concerns while keeping the core fixes intact: - wait=False for non-blocking shutdown (critical for Issue databricks#729) - Early return when enable_telemetry=False (critical for Issue databricks#729) - All Issue databricks#731 fixes (null-safety, __del__, documentation) Signed-off-by: Madhavendra Rathore <[email protected]> * Fix Black formatting violations Apply Black formatting to files modified in previous commits: - src/databricks/sql/common/unified_http_client.py - src/databricks/sql/telemetry/telemetry_client.py Changes are purely cosmetic (quote style consistency). Signed-off-by: Madhavendra Rathore <[email protected]> * Fix CI test failure: Prevent parallel execution of telemetry tests Add @pytest.mark.xdist_group to telemetry test classes to ensure they run sequentially on the same worker when using pytest-xdist (-n auto). Root cause: Tests marked @pytest.mark.serial were still being parallelized in CI because pytest-xdist doesn't respect custom markers by default. With host-level telemetry batching (PR databricks#718), tests running in parallel would share the same TelemetryClient and interfere with each other's event counting, causing test_concurrent_queries_sends_telemetry to see 88 events instead of the expected 60. The xdist_group marker ensures all tests in the "serial_telemetry" group run on the same worker sequentially, preventing state interference. Signed-off-by: Claude Sonnet 4.5 <[email protected]> * Fix telemetry test fixtures: Clean up state before AND after tests Modified telemetry_setup_teardown fixtures to clean up TelemetryClientFactory state both BEFORE and AFTER each test, not just after. This prevents leftover state from previous tests (pending events, active executors) from interfering with the current test. Root cause: In CI with sequential execution on the same worker, if a previous test left pending telemetry events in the executor, those events could be captured by the next test's mock, causing inflated event counts (88 instead of 60). Now ensures complete isolation between tests by resetting all shared state before each test starts. Signed-off-by: Claude Sonnet 4.5 <[email protected]> * Fix CI test failure: Clear _flush_event between tests The _flush_event threading.Event was never cleared after stopping the flush thread, remaining in "set" state. This caused timing issues in subsequent tests where the Event was already signaled, triggering unexpected flush behavior and causing extra telemetry events to be captured (88 instead of 60). Now explicitly clear the _flush_event flag in both setup (before test) and teardown (after test) to ensure clean state isolation between tests. This explains why CI consistently got 88 events - the flush_event from previous tests triggered additional flushes during test execution. Signed-off-by: Claude Sonnet 4.5 <[email protected]> * Add debug workflow and output to diagnose CI test failure 1. Created new workflow 'test-telemetry-only.yml' that runs only the failing telemetry test with -n auto, mimicking real CI but much faster 2. Added debug output to test showing: - Client-side captured events - Number of futures/batches - Number of server responses - Server-reported successful events This will help identify why CI gets 88 events vs local 60 events. Signed-off-by: Claude Sonnet 4.5 <[email protected]> * Fix workflow: Add krb5 system dependency The workflow was failing during poetry install due to missing krb5 system libraries needed for kerberos dependencies. Signed-off-by: Claude Sonnet 4.5 <[email protected]> * Fix xdist_group: Add --dist=loadgroup to pytest commands The @pytest.mark.xdist_group markers were being ignored because pytest-xdist uses --dist=load by default, which doesn't respect groups. With --dist=loadgroup, tests in the same xdist_group run sequentially on the same worker, preventing telemetry state interference between tests. This is the ROOT CAUSE of the 88 vs 60 events issue - tests were running in parallel across workers instead of sequentially on one worker as intended. Signed-off-by: Claude Sonnet 4.5 <[email protected]> * Add aggressive flush before test to prevent event interference CI shows 72 events instead of 60. Debug output reveals: - Client captured: 60 events (correct) - Server received: 72 events across 2 batches The 12 extra events accumulate in the timing window between fixture cleanup and mock setup. Other tests (like circuit breaker tests not in our xdist_group) may be sending telemetry concurrently. Solution: Add an explicit flush+shutdown RIGHT BEFORE setting up the mock to ensure a completely clean slate with zero buffered events. Signed-off-by: Claude Sonnet 4.5 <[email protected]> * Split workflow: Isolate telemetry tests in separate job To prevent interference from other e2e tests, split into two jobs: Job 1 (run-non-telemetry-tests): - Runs all e2e tests EXCEPT telemetry tests - Uses -n auto for parallel execution Job 2 (run-telemetry-tests): - Runs ONLY telemetry tests - Depends on Job 1 completing (needs: run-non-telemetry-tests) - Fresh Python process = complete isolation - No ambient telemetry from other tests This eliminates the 68 vs 60 event discrepancy by ensuring telemetry tests run in a clean environment with zero interference. Signed-off-by: Claude Sonnet 4.5 <[email protected]> * Fix workflows: Add krb5 deps and cleanup debug code Changes across multiple workflows: 1. integration.yml: - Add krb5 system dependency to telemetry job - Fixes: krb5-config command not found error during poetry install 2. code-coverage.yml: - Add krb5 system dependency - Split telemetry tests into separate step for isolation - Maintains coverage accumulation with --cov-append 3. publish-test.yml: - Add krb5 system dependency for consistent builds 4. test_concurrent_telemetry.py: - Remove debug print statements 5. Delete test-telemetry-only.yml: - Remove temporary debug workflow All workflows now have proper telemetry test isolation and required system dependencies for kerberos packages. Signed-off-by: Claude Sonnet 4.5 <[email protected]> * Fix publish-test.yml: Update Python 3.9 -> 3.10 Poetry 2.3.2 installation fails with Python 3.9: Installing Poetry (2.3.2): An error occurred. Other workflows use Python 3.10 and work fine. Updating to match ensures consistency and avoids Poetry installation issues. Signed-off-by: Claude Sonnet 4.5 <[email protected]> * Fix integration workflow: Remove --dist=loadgroup from non-telemetry tests - Remove --dist=loadgroup from non-telemetry job (only needed for telemetry) - Remove test_telemetry_e2e.py from telemetry job (was skipped before) - This should fix test_uc_volume_life_cycle failure caused by changed test distribution * Fix code-coverage workflow: Remove test_telemetry_e2e.py from coverage tests - Only run test_concurrent_telemetry.py in isolated telemetry step - test_telemetry_e2e.py was excluded in original workflow, keep it excluded * Fix publish-test workflow: Remove cache conditional - Always run poetry install (not just on cache miss) - Ensures fresh install with system dependencies (krb5) - Matches pattern used in integration.yml * Fix publish-test.yml: Remove duplicate krb5 install, restore cache conditional - Remove duplicate system dependencies step - Restore cache conditional to match main branch - Keep Python 3.10 (our change from 3.9) * Fix code-coverage: Remove serial tests step - All serial tests are telemetry tests (test_concurrent_telemetry.py and test_telemetry_e2e.py) - They're already run in the isolated telemetry step - Running -m serial with --ignore on both files results in 0 tests (exit code 5) --------- Signed-off-by: Madhavendra Rathore <[email protected]> Signed-off-by: Claude Sonnet 4.5 <[email protected]>

Signed-off-by: Jayant Singh <[email protected]>

* Add statement level query tag support by introducing it as a parameter on execute* methods Signed-off-by: Jiabin Hu <[email protected]> * Add query_tags support to executemany method - Added query_tags parameter to executemany() method - Query tags are applied to all queries in the batch - Updated example to demonstrate executemany usage with query_tags - All tests pass (122/122 client tests) Signed-off-by: Jiabin Hu <[email protected]> * add example that doesn't have tag Signed-off-by: Jiabin Hu <[email protected]> * fix presubmit errors Signed-off-by: Jiabin Hu <[email protected]> * another lint Signed-off-by: Jiabin Hu <[email protected]> * address review comments Signed-off-by: Jiabin Hu <[email protected]> --------- Signed-off-by: Jiabin Hu <[email protected]>

…tion (databricks#749) * Allow specifiying query tags as a dict upon connection creation Signed-off-by: Jiabin Hu <[email protected]> * fix comment Signed-off-by: Jiabin Hu <[email protected]> --------- Signed-off-by: Jiabin Hu <[email protected]>

databricks#742) * Fix float inference to use DoubleParameter (64-bit) instead of FloatParameter (32-bit) Signed-off-by: Shubhambhusate <[email protected]> * Add DoubleParameter with Primitive.DOUBLE to test_inference coverage --------- Signed-off-by: Shubhambhusate <[email protected]>

… default (databricks#751) Updated pyarrow-concat

* Add statement-level query_tags support for SEA backend Signed-off-by: Sreekanth Vadigi <[email protected]> * Simplify None handling in query_tags serialization Signed-off-by: Sreekanth Vadigi <[email protected]> --------- Signed-off-by: Sreekanth Vadigi <[email protected]>

Adds 42 new E2E tests and 3 unit tests covering gaps identified in MST + xDBC metadata audit: E2E tests (test_transactions.py): - Metadata operations (cursor.columns/tables/schemas/catalogs) in txn - MSTCheckRule-blocked SQL (SHOW/DESCRIBE/information_schema) in MST - Allowed operations (USE CATALOG/SCHEMA) in MST - DML variants: executemany, parameterized, DELETE, mixed DML in txn - Concurrent transactions: write conflict, write skew, repeatable reads - Multiple cursors in single transaction - Connection close with pending transaction (implicit rollback) - DDL behavior in transactions (CREATE/DROP/ALTER TABLE) - Edge cases: empty txn, read-only, atomicity, rollback recovery, MERGE Unit tests (test_client.py): - executemany error propagation in transaction context - Cursor close does not affect transaction state - Autocommit toggle consistency Co-authored-by: Isaac Signed-off-by: Vikrant Puppala <[email protected]>

Co-authored-by: Isaac Signed-off-by: Vikrant Puppala <[email protected]>

vikrantpuppala and others added 30 commits May 9, 2025 23:02

Cleanup fields as they might be deprecated/removed/change in the futu…

bdb5154

…re (databricks#553) * Clean thrift files Signed-off-by: Vikrant Puppala <[email protected]>

[PECOBLR-361] convert column table to arrow if arrow present (databri…

7b51c6e

…cks#551)

Update CODEOWNERS (databricks#562)

f9936d7

new codeowners

Update github actions run conditions (databricks#569)

eaa542d

More conditions to run github actions

New Complex type test table + Github Action changes (databricks#575)

0947b9a

* Added new table * Some more fixes

Bump version to 4.0.4 (databricks#599)

0271b28

* Added change log * Update CHANGELOG.md Co-authored-by: Vikrant Puppala <[email protected]> --------- Co-authored-by: Vikrant Puppala <[email protected]>

Updated the changelog format (databricks#600)

c123af3

nit

Updated tests (databricks#614)

8b841c7

Add test to check thrift field IDs (databricks#602)

4cebc36

* Add test to check thrift field IDs --------- Signed-off-by: Vikrant Puppala <[email protected]>

Revert "Enhance Cursor close handling and context manager exception m… (

b3a6f55

databricks#613) * Revert "Enhance Cursor close handling and context manager exception management to prevent server side resource leaks (databricks#554)" This reverts commit edfb283. * revert e2e

Bump version to 4.0.5 (databricks#615)

7696316

* Release version 4.0.5: Reverted cursor close handling changes to fix user errors. Updated version numbers in pyproject.toml and __init__.py. * Update CHANGELOG.md to include reference to issue databricks#613 for cursor close handling fix.

Removed Codeowners (databricks#623)

90f0ac1

nit

[PECOBLR-587] Azure Service Principal Credential Provider (databricks…

e50e86d

…#621) * basic setup * Nit * working * moved pyjwt to code dependency * nit * nit * nit * nit * nit * nit * testing sdk * Refractor * logging * nit * nit * nit * nit * nit

Arrow performance optimizations (databricks#638)

e0ca049

* Minor fix * Perf update * more * test fix

added logs for cloud fetch speed (databricks#654)

59d28b0

Make telemetry batch size configurable and add time-based flush (data…

a0d7cd1

…bricks#622) configurable telemetry batch size, time based flush Signed-off-by: Sai Shree Pradhan <[email protected]>

jayantsing-db and others added 27 commits November 14, 2025 02:45

Bump to version 4.2.0 (databricks#707)

cca421b

Bump to version 4.2.0 Signed-off-by: Jayant Singh <[email protected]>

Ready for 4.2.1 release (databricks#713)

ad227ca

Signed-off-by: Vikrant Puppala <[email protected]>

Change default use_hybrid_disposition to False (databricks#714)

b8494ff

This changes the default value of use_hybrid_disposition from True to False in the SEA backend, disabling hybrid disposition by default.

pandas 2.3.3 support for py < 3.14 (databricks#721)

f7822fd

* pandas 2.3.3 support for py < 3.14 Signed-off-by: Sreekanth Vadigi <[email protected]> * poetry lock Signed-off-by: Sreekanth Vadigi <[email protected]> --------- Signed-off-by: Sreekanth Vadigi <[email protected]>

New minor release (databricks#722)

ce55e7b

Fixed the exception handler close() on _TelemetryClientHolder (databr…

9b4e577

…icks#723) Fixed the exception handler calls close() on _TelemetryClientHolder objects instead of accessing the client inside them.

New minor version release 4.2.4 (databricks#725)

03eb369

New minor version release

[PECOBLR-1168] query tags telemetry (databricks#716)

4b7df5b

* query tags telemetry Signed-off-by: Sreekanth Vadigi <[email protected]> * code linting fix Signed-off-by: Sreekanth Vadigi <[email protected]> --------- Signed-off-by: Sreekanth Vadigi <[email protected]>

Bump to version 4.2.5 (databricks#737)

9fe7356

Signed-off-by: Jayant Singh <[email protected]>

Updated the PyArrow concatenation of tables to use promote_options as…

36fb376

… default (databricks#751) Updated pyarrow-concat

Rename test table to avoid collisions with JDBC driver tests

cf92d96

Co-authored-by: Isaac Signed-off-by: Vikrant Puppala <[email protected]>

vikrantpuppala had a problem deploying to azure-prod April 2, 2026 06:50 — with GitHub Actions Failure

vikrantpuppala closed this Apr 2, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[PECOBLR-2086] Add comprehensive MST transaction metadata and edge case tests#1

[PECOBLR-2086] Add comprehensive MST transaction metadata and edge case tests#1
vikrantpuppala wants to merge 81 commits intomainfrom
add-mst-transaction-tests

vikrantpuppala commented Apr 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

13 participants

Conversation

vikrantpuppala commented Apr 2, 2026

Summary

Test plan

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

13 participants