Skip to content

Support an immutable pipeline timezone for NOW() #6710

Description

@Mrhs121

Problem

Feldera's `NOW()` currently returns a `TIMESTAMP WITHOUT TIME ZONE`
containing the current UTC time.

Our source data contains timezone-naive local timestamps in
`Asia/Shanghai`. As a result, temporal filters such as:

```sql
WHERE event_time >= NOW() - INTERVAL 1 DAY

compare Shanghai wall-clock timestamps with UTC wall-clock timestamps and
produce an 8-hour offset.

Changing the host or container timezone does not affect NOW().

The current workaround is:

CONVERT_TIMEZONE('UTC', 'Asia/Shanghai', NOW())

This works, but it is cumbersome to apply consistently across a large
number of queries, especially temporal filters.

Determinism requirements

I agree that Feldera programs should remain deterministic. The timezone should not be inferred from the host, container, JVM, or environment.

The configured timezone should:

  • default to UTC for backward compatibility;
  • be explicitly stored as part of the program or pipeline configuration;
  • produce identical results on different hosts;
  • be immutable after the pipeline starts;
  • be preserved and validated when restoring from a checkpoint;
  • not be changeable while replaying existing data.

Acceptable solutions

Any of the following would address our use case.

1. Immutable pipeline property

Add a pipeline property such as:

sql_timezone: Asia/Shanghai

or:

clock_timezone: +08:00

The clock connector would use this timezone when producing the
TIMESTAMP WITHOUT TIME ZONE value consumed by NOW().

The property should be fixed for the lifetime of the pipeline. An attempt to restart or restore a pipeline with a different timezone should be rejected.

2. Compile-time SQL setting

Support a declaration such as:

SET TIME ZONE 'Asia/Shanghai';

This would be part of the SQL program and program hash, rather than mutable session state.

Editing the SQL program before starting the pipeline would be acceptable for our use case.

3. Fixed-offset configuration as an initial implementation

If IANA timezones introduce problems related to daylight-saving transitions and monotonicity, supporting an immutable fixed offset first would also solve our use case:

sql_timezone: +08:00

Asia/Shanghai currently uses a fixed UTC+8 offset, so this would be
sufficient for us.

Expected behavior

With:

sql_timezone: Asia/Shanghai

or:

sql_timezone: +08:00

the following query:

SELECT NOW();

should return Shanghai local wall-clock time.

A pipeline with this temporal filter:

WHERE event_time >= NOW() - INTERVAL 1 DAY

should work correctly when event_time contains timezone-naive Shanghai timestamps.

When no timezone is configured, NOW() should retain its current UTC behavior.

Metadata

Metadata

Assignees

Labels

Pipeline managerPipeline manager (API, API server, runner, compiler server)

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions