Skip to content

[pull] main from tobymao:main#830

Open
pull[bot] wants to merge 7878 commits into
MarqueIV:mainfrom
tobymao:main
Open

[pull] main from tobymao:main#830
pull[bot] wants to merge 7878 commits into
MarqueIV:mainfrom
tobymao:main

Conversation

@pull

@pull pull Bot commented Nov 26, 2025

Copy link
Copy Markdown

See Commits and Changes for more details.


Created by pull[bot] (v2.0.0-alpha.4)

Can you help keep this open source service alive? 💖 Please sponsor : )

@pull pull Bot locked and limited conversation to collaborators Nov 26, 2025
@pull pull Bot added ⤵️ pull merge-conflict Resolve conflicts manually labels Nov 26, 2025
@georgesittas georgesittas force-pushed the main branch 4 times, most recently from 5882a13 to 7a70164 Compare December 10, 2025 13:00
@georgesittas georgesittas force-pushed the main branch 3 times, most recently from 32ef521 to bbeb881 Compare January 13, 2026 11:27
@georgesittas georgesittas force-pushed the main branch 4 times, most recently from 5068bc7 to 14f9e93 Compare January 16, 2026 09:54
@georgesittas georgesittas force-pushed the main branch 6 times, most recently from 2f19d1d to f1c4fa0 Compare January 26, 2026 10:25
@georgesittas georgesittas force-pushed the main branch 3 times, most recently from 84af655 to 04002ae Compare February 3, 2026 12:55
@georgesittas georgesittas force-pushed the main branch 2 times, most recently from ff7370c to aa14297 Compare February 11, 2026 10:31
@georgesittas georgesittas force-pushed the main branch 2 times, most recently from cc48178 to e18a24e Compare February 27, 2026 13:46
geooo109 and others added 30 commits July 10, 2026 18:47
Build the TokenizerCore fresh in Tokenizer.__init__ instead of caching it
in a process-lifetime thread-local. The cache stored a stateful core keyed
by tokenizer type; building fresh keeps the core scoped to its Tokenizer
instance and removes the shared mutable thread-local.
Signed-off-by: c-tonneslan <cst0520@gmail.com>
…res (#7855)

* fix(bigquery): transpile single projection GENERATE_SERIES from postgres

* ref

* ref impl

* ref 2

* ref 3

* ref f
Co-authored-by: Giorgos Michas <geomichas96@gmail.com>
Co-authored-by: Giorgos Michas <geomichas96@gmail.com>
…REGR_SYY (#7851)

* feat(typing): add databricks type inference for REGR_SXX, REGR_SXY, REGR_SYY

* fix(typing): correct DISTINCT arg semantics for REGR_SXX and REGR_SYY in Databricks

* refactor(typing): extract shared _regr_sql helper in DatabricksGenerator

* fix: update databricks generator to have entries in transform
…iter, preserve escapes in raw strings (#7870)

* Fix(tokenizer)!: handle escaped quote before closing multi-char delimiter, preserve escapes in raw strings

* Sync w/ integration tests
* feat(typing): add databricks type inference for RINT, ROUND

* fix(optimizer)!: revert Round pass-through annotator for databricks [CLAUDE]
* fix(spark): parse month, day without leading zeros

* fix(spark): also fix generation of `exp.StrToTime`

* chore(spark): narrow type to fix mypy attr-defined

* fix(spark): parse strict month/day via distinct canonical format

Spark 3+ parses MM/dd strictly (single-digit months/days don't parse),
unlike the lax %m/%d most dialects produce. Map Spark's MM/dd to a distinct
canonical token (%mstrict/%dstrict) only when parsing, so the strict format
roundtrips while lax %m/%d still becomes the lenient M/d. Formatting keeps the
padded %m/%d -> MM/dd. The internal tokens degrade to %m/%d for every other
dialect via the metaclass inverse fallback and the generic strtotime_sql.

Assisted-by: Claude Opus 4.8 <noreply@anthropic.com>

* refactor(spark): narrow generator dialect type via TYPE_CHECKING

Replace the per-call local import and `assert isinstance(self.dialect, Spark)`
in SparkGenerator.format_time with a TYPE_CHECKING-guarded `dialect: Spark`
class annotation, removing the runtime overhead while keeping mypy happy.

Assisted-by: Claude Opus 4.8 <noreply@anthropic.com>

* fix(bigquery): degrade strict time token in FORMAT clause

The strict->lax canonical fallback was applied only to INVERSE_TIME_MAPPING,
so BigQuery's `FORMAT '...'` clause (which uses INVERSE_FORMAT_MAPPING) leaked
the internal token, e.g. Spark `TO_DATE(x, 'MM/dd/yyyy')` -> BigQuery
`... FORMAT 'MMstrict/DDstrict/YYYY'`. Apply the same fallback to
INVERSE_FORMAT_MAPPING via a shared helper so it degrades to 'MM/DD/YYYY'.

Assisted-by: Claude Opus 4.8 <noreply@anthropic.com>

* refactor(spark): share parse-time expression set, clarify strtotime CLAUDE

Reuse dialect.STRICT_PARSE_TIME_EXPRESSIONS in SparkGenerator.format_time
instead of a duplicate LENIENT_TIME_EXPRESSIONS tuple, removing the cross-module
constant that had to be kept in sync by hand. Also document why the base
strtotime_sql degrades only the strict tokens rather than routing the whole
format through self.format_time().

Assisted-by: Claude Opus 4.8 <noreply@anthropic.com>

* refactor(spark): host lenient time mapping on the base Dialect CLAUDE

Declare LENIENT_INVERSE_TIME_MAPPING/_TRIE on the base Dialect (mirroring
STRICT_TIME_MAPPING and INVERSE_TIME_MAPPING) and build the trie in the
metaclass. This drops the Spark-only attribute that forced a `dialect: Spark`
type-narrowing annotation in SparkGenerator plus its TYPE_CHECKING import, and
lets spark.py stop hand-building the trie. No behavior change.

Assisted-by: Claude Opus 4.8 <noreply@anthropic.com>

* refactor(spark): use `or` fallback in format_time, matching the base CLAUDE

Replace the `inverse_time_mapping is None and ...` guard with the same
`inverse_time_mapping or ...` fallback idiom the base Generator.format_time
uses. Same semantics (an explicitly-passed mapping still wins), but consistent
with the base and without the extra clause. No behavior change.

Assisted-by: Claude Opus 4.8 <noreply@anthropic.com>

* docs(spark): trim over-explaining comments on lenient time mapping CLAUDE

Drop the "used by the generator's format_time override / trie built by the
metaclass" note on Spark.LENIENT_INVERSE_TIME_MAPPING (no other mapping documents
where it's consumed or that the metaclass builds its trie), and reword the base
docstring's awkward "Lets e.g. Spark" to the parenthetical "(e.g. Spark)" form
already used by STRICT_TIME_MAPPING.

Assisted-by: Claude Opus 4.8 <noreply@anthropic.com>

* refactor(hive, spark): model strict MM/dd parsing via TIME_MAPPING + a flag CLAUDE

Applies the maintainer's feedback on #7773 and extends strict date parsing to the
whole Hive family.

Instead of patch-1's STRICT_TIME_MAPPING + LENIENT_INVERSE_TIME_MAPPING (and the
`strict` flag threaded through Dialect.format_time / build_formatted_time), fold
MM -> %mstrict into the regular TIME_MAPPING and decide the parse/format role in the
generator. The _Dialect metaclass keeps degrading %mstrict -> %m for non-strict
dialects.

Strictness (modern Hive, Spark 3+, Databricks are strict; Spark 2 is lenient) is a
single dialect flag STRICT_TIME_PARSING; a lenient canonical %m is rendered as the
non-padded M for parse expressions in HiveGenerator.format_time, so single-digit
sources stay parseable while %mstrict round-trips to MM.

Assisted-by: Claude Opus 4.8 <noreply@anthropic.com>

* refactor(dialects): derive strict-time inverse in the metaclass CLAUDE

Hive, Spark, and Databricks each restated the same
`INVERSE_TIME_MAPPING = {"%m": "MM", "%d": "dd"}` to restore the padded
formatting inverse that MM/dd -> %mstrict/%dstrict displaces. Fold that into
`_with_strict_time_fallback`: for a strict dialect (whose auto-derived inverse
already contains %mstrict), default the lax %m/%d to the strict form's inverse.

This is the maintainer's suggestion on #7773 - set the strict entries once in
the _Dialect metaclass instead of repeating them per dialect.

Assisted-by: Claude Opus 4.8 <noreply@anthropic.com>

* refactor(dialects): drive strict-time tokens from STRICT_TIME_PARSING CLAUDE

Strictness was declared twice per dialect: the STRICT_TIME_PARSING flag and the
hand-written MM/dd -> %mstrict/%dstrict entries in TIME_MAPPING, which had to be
kept in sync. Make the flag the single source of truth and derive the tokens in
the _Dialect metaclass: promote lax %m/%d to the strict tokens for strict
dialects, strip them back otherwise (needed when a strict base is inherited,
e.g. Spark 2 <- Hive).

Dialects now declare only the flag - Spark and Spark2 drop their TIME_MAPPING
overrides entirely, and the %mstrict/%dstrict tokens no longer appear in any
dialect file. Rename the inverse helper to _with_strict_time_inverse so it
pairs with the new _with_strict_time_mapping.

Assisted-by: Claude Opus 4.8 <noreply@anthropic.com>

* refactor(hive): render parse formats in one pass via a derived mapping CLAUDE

The strict-dialect format_time override ran two passes: first rewriting the
lax canonical %m/%d to %-m/%-d (with identity entries just to keep the trie
from partial-matching the %m prefix inside %mstrict), then converting through
INVERSE_TIME_MAPPING. Instead, derive a PARSE_INVERSE_TIME_MAPPING in the
_Dialect metaclass - INVERSE_TIME_MAPPING with %m/%d overridden to render
non-padded - and pass it to the base format_time in a single call.

This drops the LENIENT_PARSE_TIME_MAPPING identity-entry hack, halves the
format traversals, and makes the parse-side rendering an explicit named
mapping. Lenient dialects alias the plain inverse mapping, so no extra tries
are built for them.

Assisted-by: Claude Fable 5 <noreply@anthropic.com>

* fix(hive, spark): extend lenient parse formats to StrToUnix CLAUDE

UNIX_TIMESTAMP parses its string input, but exp.StrToUnix was missing from
PARSE_TIME_EXPRESSIONS, so a lax foreign %m/%d still rendered as the strict
MM/dd - e.g. STR_TO_UNIX(x, '%Y-%m-%d') became UNIX_TIMESTAMP(x, 'yyyy-MM-dd'),
which silently returns NULL for single-digit months/days in Spark 3+ and
modern Hive. Include it so lax sources render as M/d, matching the
StrToTime/StrToDate/TsOrDsToDate handling; Spark's own strict MM/dd still
roundtrips via %mstrict/%dstrict.

Assisted-by: Claude Fable 5 <noreply@anthropic.com>
* fix for projection collision

* updates from PR feedback

* merge issues

* add schema, update merged_sources, search deeper into group by

* fix(optimizer): revert unrelated qualify_columns star-dedup change (CLAUDE)

This half of the previous commit was the query-star-collision fix and
belongs on optimizer/bug-query-star-collision instead. Keeps this
branch scoped to the merge_subqueries.py literal GROUP BY fix.

* added test and handling for empty schema
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

⤵️ pull merge-conflict Resolve conflicts manually

Projects

None yet

Development

Successfully merging this pull request may close these issues.