Skip to content

BUG: fix Table.group_by().groups.aggregate() for multidimensional columns - #20410

Open
bhuvan-somisetty wants to merge 1 commit into
astropy:mainfrom
bhuvan-somisetty:fix-groups-aggregate-multidim-20409
Open

bhuvan-somisetty wants to merge 1 commit into
astropy:mainfrom
bhuvan-somisetty:fix-groups-aggregate-multidim-20409

Conversation

@bhuvan-somisetty

@bhuvan-somisetty bhuvan-somisetty commented Sep 15, 2026

Copy link
Copy Markdown

Description

ColumnGroups.aggregate() (used by Table.group_by(...).groups.aggregate()) has a fast path for np.mean that does:

vals = np.add.reduceat(par_col, i0s) / np.diff(self.indices)

np.diff(self.indices) is a 1-D array of per-group row counts. For a multidimensional column, np.add.reduceat(par_col, i0s) has shape (n_groups, *trailing_dims), and dividing it by the 1-D counts array broadcasts against the last axis instead of the row axis. This means:

  • if n_groups happens to equal a trailing dimension size, the division silently succeeds but produces wrong numbers, with no error or warning at all;
  • otherwise it raises a broadcasting error, which gets caught and turned into a warning, and the column is silently dropped from the aggregated table.

This fixes it by reshaping the per-group counts so the division only broadcasts against the row axis, preserving the trailing dimensions. It's the same underlying issue as #13225, which was already fixed for TimeSeries.aggregate_downsample() — this applies the same fix to the Table.group_by().groups.aggregate() code path.

Added a regression test covering both failure modes described above (wrong values when shapes coincide, dropped column otherwise), and a changelog fragment.

Fixes #20409

AI Disclosure

If AI tools were used to develop this pull request, describe the tools including specific model and version, how they were used, and what content is AI generated. Otherwise enter "N/A".

I used Claude Code (Claude Sonnet 5) as an assistant throughout this contribution:

  • It helped search the codebase and identify the root cause in astropy/table/groups.py.
  • I reviewed and ran the reproduction scripts myself to confirm the bug before filing the issue.
  • The issue body, PR description text,
  • I certify that I am human and that I take full responsibility for this pull request including all interactions with reviewers.

Merge method

  • By checking this box, the PR author has requested that maintainers do NOT use the "Squash and Merge" button. Maintainers should respect this when possible; however, the final decision is at the discretion of the maintainer that merges the PR.

…umns

ColumnGroups.aggregate() computed per-group means for multidimensional
columns as np.add.reduceat(par_col, i0s) / np.diff(self.indices), where
the count array only accounts for the leading (row) axis. Dividing a
multidimensional reduceat result by this 1-D count array either
broadcasts silently against the wrong axis (giving wrong numbers with
no error) whenever the number of groups happens to equal a trailing
dimension size, or raises a broadcasting error that gets caught and
turned into a warning, silently dropping the column from the
aggregated table.

Reshape the per-group counts to broadcast only against the row axis,
matching the fix already applied to aggregate_downsample() for the
same underlying issue (astropy#13225).

Fixes astropy#20409
@github-actions

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Astropy! 🌌 This checklist is meant to remind the package maintainers who will review this pull request of some common things to look for.

  • Do the proposed changes actually accomplish desired goals?
  • Do the proposed changes follow the Astropy coding guidelines?
  • Are tests added/updated as required? If so, do they follow the Astropy testing guidelines?
  • Are docs added/updated as required? If so, do they follow the Astropy documentation guidelines?
  • Is rebase and/or squash necessary? If so, please provide the author with appropriate instructions. Also see instructions for rebase and squash.
  • Did the CI pass? If no, are the failures related? If you need to run daily and weekly cron jobs as part of the PR, please apply the "Extra CI" label. Codestyle issues can be fixed by the bot.
  • Is a change log needed? If yes, did the change log check pass? If no, add the "no-changelog-entry-needed" label. If this is a manual backport, use the "skip-changelog-checks" label unless special changelog handling is necessary.
  • Is this a big PR that makes a "What's new?" entry worthwhile and if so, is (1) a "what's new" entry included in this PR and (2) the "whatsnew-needed" label applied?
  • At the time of adding the milestone, if the milestone set requires a backport to release branch(es), apply the appropriate "backport-X.Y.x" label(s) before merge.

@pllim

This comment was marked as resolved.

@pllim pllim added this to the v8.1.0 milestone Sep 15, 2026
@pllim pllim added the Bug label Sep 15, 2026
@bhuvan-somisetty

Copy link
Copy Markdown
Author

CircleCI, docs build, and pre-commit are all passing. Looks like the GH Actions workflows are stuck waiting on approval since this is my first PR here - could a maintainer approve them so the rest of CI can run? Thanks!

@bhuvan-somisetty

Copy link
Copy Markdown
Author

Good catch, sorry for missing that - added an AI usage disclosure section to the PR description. Let me know if you'd like more detail.

@pllim

This comment was marked as resolved.

@bhuvan-somisetty

Copy link
Copy Markdown
Author

Filled out the AI Disclosure section in the PR description - I used Claude Code (Claude Sonnet 5) to help investigate, write the fix/tests, and draft the PR/issue text, and I've reviewed and tested everything before it was posted. Checked the certification box too. Thanks for flagging it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Table.group_by().groups.aggregate() gives wrong results (or silently drops the column) for multidimensional columns

2 participants