Skip to content

Query/job is not cancelled when interrupted by KeyboardInterrupt #15694

@OlegWock

Description

@OlegWock

Looking at current code, in _wait_or_cancel we cancel query if any of the exceptions happens. However, since we're intercepting Exception, this means that KeyboardInterrupt is not handled (as it inherits from BaseException, not Exception) and bubbles up (usually resulting in program termination). If query was already submitted at that moment, it will continue running, which is not desirable behavior (e.g. I realized query wasn't right and pressed Ctrl+C to terminate the program)

Environment details

Not relevant

Steps to reproduce

  1. Submit any long-running query using python-bigquery and wait for ~10s to make sure query was submitted
  2. Press Ctrl+C to raise KeyboardInterrupt
  3. Program exits, but query is still running on the BigQuery

Code example

from google.cloud import bigquery

client = bigquery.Client()

query = """
WITH a AS (
  SELECT x AS i
  FROM UNNEST(GENERATE_ARRAY(1, 10000)) AS x
),
b AS (
  SELECT y AS j
  FROM UNNEST(GENERATE_ARRAY(1, 10000)) AS y
),
pairs AS (
  SELECT
    i,
    j,
    SIN(i * j) AS s,
    COS(i + j) AS c
  FROM a
  CROSS JOIN b
),
heavy AS (
  SELECT
    i,
    j,
    -- intentionally nasty CPU-heavy expression
    EXP(
      SIN(s) * COS(c)
      + LOG(ABS(s) + 1.000001)
      + LOG(ABS(c) + 1.000001)
    )
    + EXP(
      SIN(i) * COS(j)
      + LOG(i + 1.000001)
      + LOG(j + 1.000001)
    ) AS heavy_val
  FROM pairs
)
SELECT
  COUNT(*) AS row_count,
  SUM(heavy_val) AS total_heavy_val
FROM heavy;
"""

rows = client.query_and_wait(query)

for row in rows:
    print(f"row_count={row['row_count']}, total_heavy_val={row['total_heavy_val']}")

Metadata

Metadata

Assignees

No one assigned

    Labels

    api: bigqueryIssues related to the BigQuery API.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions