New features

• Weights by subsamples (#294): per-subsample weights, inclusive and by-category, with full variation (up/down) support. Subsample-specific weight variations are correctly isolated per subsample in the histogram variation axis.
• Shape variations by subsamples (#330): subsample-specific shape (calibrator) variations. Each subsample can carry a different set of shape variations; histogram axes are built per-subsample so the variation labels are correctly isolated.
• Fix histogram filling for subsample weight and shape variations (#491): several bugs in HistManager were fixed — wrong variation axis used in the fill loop when subsamples are configured, subsample-specific shape variations silently skipped during filling, and subsample nominal weights not applied during shape-variation passes. Comprehensive integration tests added.
• Variations-aware skimming (#467): the skim step now retains events that pass at least one preselection variation (not just the nominal), ensuring shape-variation passes do not lose events skimmed away by the nominal selection.
• MET Type-1 correction calibrator (#472): new calibrator for MET Type-1 propagation, including unclustered energy systematics. Extended to Run2.
• Multi-dimensional variables (#374): HistConf now supports multi-dimensional delayed-evaluation variables.
• Jet pT regression (#461): jet pT regression support in the jet calibrator, with merge_collections_for_variations and collection_name_alias options to merge regressed and standard jet collections.
• JetID from correctionlib (#421, #395): JetID delivered via correctionlib, replacing hardcoded values.
• msoftdrop correction (#420): soft-drop mass correction for AK8 jets.
• Default JEC via correctionlib (#401, #418): JEC/JER now use correctionlib as the default backend.
• Read unpublished remote datasets over XRootD (#461): dataset builder can now discover and read files from unpublished XRootD endpoints.
• merge_columns CLI command (#482): new pocket-coffea merge-columns sub-command to merge columnar outputs.
• CCsorted function (#456): utility to sort collections by a given field with stable ordering.

Scale factors and corrections

• Muon promptMVA scale factors, including ISO SF for 2024 (#470)
• Electron promptMVA scale factors with promptMVA ID (#474)
• Updated BTV scale factors for 2024 (#473)
• Lepton (electron and muon) scale factors for 2024 (#466)
• Electron trigger scale factor added to common weights for Run3 (#476); sf_ele_trigger renamed for Run2 consistency (#481)
• Updated electron scale factors and electron SS energy scale/smearing (#444, #414, #419)
• Muon calibrator update (#432)
• Rochester muon momentum corrections (#475): new MuonsRochesterCalibrator for Run 2 Ultra-Legacy muon pT corrections using the RoccoR library; correction files included for 2016 pre/post-VFP, 2017, and 2018.
• Muon SF lower pT bound for Run3 set to 26 GeV; electron to 25 GeV (#476)
• Jets for NanoAODv12 (#355)
• JetVetoMap fix (#430)
• FatJet ID fix and 2024 AK8 support (#429, #440)
• Updated JEC data tags for 2022/2023 (#489); JEC configuration for 2023 preBPix MC
• Lepton mvaTTH discriminant added to lepton objects (#455)
• UParT tagger scores added to jet_taggers_hists() (#460)
• Updated luminosity values for 2023 and 2024; golden JSON updated for 2022, 2023, 2024 and added for 2025 (#441, #490)
• Updated correctionlib tags (#415)

Plotter and output improvements

• Plotter: correct handling of subsamples and weight/shape variations (#442)
• Plotter: fix systematic band rendering (#408)
• Save shape variation arrays to output (#368)
• Cutflow: major rework with subsample support and various fixes (#369); fix for cutflow with subsamples in plot scripts; fix handling of variations in sum-of-weights
• Datacard output support (#407)
• New analysis metadata stored in output (#409)

LAW integration

• Updated LAW task parameters and skipbadfiles option (#464)
• LAW tasks for plotting (#393)
• Dask scheduler support in runner (#403, #411)
• Updated dataset-definition parameter handling (#464)

Bug fixes

• Fixed priority list not being passed to dataset builder (#485)
• NanoAOD v15 compatibility for Run 2 UL (#457): rho field access, PU jet ID, and JER now use the correct NanoAOD-version-aware branch paths (Rho.fixedGridRhoFastjetAll for v12+, fixedGridRhoFastjetAll for <=v9); PUid selection correctly disabled for NanoAOD v15.
• Fixed jets variations for AK8Puppi collection
• Fixed jets variations and config when using collection_name_alias
• Fixed legacy calibrator raw-factor NaN (#428)
• Fixed filesets configuration exception message with filter details (#448)
• Fixed histogram filling for only_variations histograms (#392)
• Fixed calibrators variations handling (#389)
• Fixed muon SF for Run3 (#397)
• Fixed dump_ak_array for XRootD mkdir (#448)
• Improved hadd_skimmed_files, check_jobs, and merge_output scripts

New Contributors

• @vslokenb made their first contribution in #400
• @NinaHerfort made their first contribution in #408
• @piosifid made their first contribution in #432
• @Copilot made their first contribution in #448
• @Pa-Ker made their first contribution in #456
• @brkronheim made their first contribution in #374

Full Changelog: 0.9.10...0.9.11

Many improvements on different fronts:

• First version of the datacard export system
• improved handling of big analyses on condor@lxplus executor: automatic resubmission and update of the filelist for failing remote reading
• New sites supported by new executors
• Improvements in the plotting scripts

What's Changed

• Added MET rescaling after JEC by @valsdav in #307
• Add option to set x axis to log scale by @yihui-lai in #331
• Update/plotting logscale by @felixzinn in #332
• Update recipes.md to add onnx executor how-to by @Emma-223 in #338
• Fix type error by @sethcooper in #336
• Fix #327: fix incorrect throughput by @chihchun1011 in #340
• change condor lxplus executor to set up the virtualenv by @matteomalucchi in #341
• Update/dataset query by @felixzinn in #342
• Implementing PNet jet energy regression by @andreypz in #345
• Priority sorting in build-dataset and dataset-discovery-cli by @thierryharte in #349
• Fixing DESY NAF executor by @leyva-daina in #353
• Filter invalid files in a dataset by @andreypz in #352
• Fix possible bad skim save path issue by @thierryharte in #357
• Prevent save_skimmed_files from failing with --test by @thierryharte in #356
• add datacard class and utilities by @felixzinn in #236
• Update paramers for DESY executor by @andreypz in #359
• condor@lxplus: Automatic selection of failed jobs while resubmitting; backup xrootd site for xrootd errors by @mondalspandan in #358
• Fix an empty variations check by @andreypz in #354
• Add split-by-category feature for handling large outputs with histograms from a large number of categories by @mondalspandan in #362
• check-jobs: Replace files with xrootd errors; implement blacklist for sites with repeated failures by @mondalspandan in #363

New Contributors

• @yihui-lai made their first contribution in #331
• @Emma-223 made their first contribution in #338
• @sethcooper made their first contribution in #336
• @thierryharte made their first contribution in #349
• @leyva-daina made their first contribution in #353

Full Changelog: 0.9.9...0.9.10

Minor version update with many improvements, new executors and few new features.

What's Changed

• Fixing loading of shape variations by @valsdav in #280
• Improvements for manual condor jobs and skim execution by @valsdav in #279
• Define tagger scores based on used tagger only once when selecting jets, and prevent repeated codes by @mondalspandan in #276
• Blind ratio plots when blinding data by @mondalspandan in #283
• Adding executor for Oscar cluster @ Brown University (slurm based) by @mondalspandan in #282
• fix: btagSF for JES variations was not properly loaded by @valsdav in #286
• Update JEC txt files by @gbonomel in #288
• Minor developments to various scripts by @andreypz in #264
• fix: hadd for skimmed files by @valsdav in #290
• Parton shower and QCD scale weights by @mmarchegiani in #298
• FIX bug - don't look for x509_path when ignore-grid-certificate is true by @matteomalucchi in #301
• Improve docs for installation by @andreypz in #299
• Update/law tasks by @felixzinn in #269
• Add sorting by b tagging score AND fix naming convention of 2023 eras in jet veto map config by @mondalspandan in #303
• fix: automatically remove _Disk from sitename when filtering by @valsdav in #308
• Minor-update on plotting and law tasks by @felixzinn in #311
• update documentation and docstring by @felixzinn in #313
• add missing default for ylim_log systematics by @felixzinn in #317
• Improved plotting and skim export bugfix by @mmarchegiani in #304
• dask executor on CLAIX by @chihchun1011 in #319
• Kinematic_fit_for_VHcc by @ValVau in #321
• Minor improvements in plotting by @mmarchegiani in #320
• Minor changes to executor parameters, rucio script and requirements by @andreypz in #314
• Modify the make-plot script & Add UMD_T3 executor by @De-Cristo in #323
• Implement photon selection and scale factors by @elham-khazaie in #328
• Add weights to Data samples by @valsdav in #312

New Contributors

• @gbonomel made their first contribution in #288
• @chihchun1011 made their first contribution in #319
• @De-Cristo made their first contribution in #323
• @elham-khazaie made their first contribution in #328

Full Changelog: 0.9.8...0.9.9

What's Changed

• Make gen_dijet work for GenJets by @andreypz in #247
• Update analysis_example.md by @valsdav in #248
• Improvements in plotting + various by @mmarchegiani in #246
• Hotfix for jet_taggers_hists() by @mmarchegiani in #251
• Support for dask at Brown brux20 cluster by @mondalspandan in #253
• Run grouping by sample by @mmarchegiani in #257
• load plotting style defaults from yaml file by @felixzinn in #256
• Improved scripts by @mmarchegiani in #261
• Bump codecov/codecov-action from 4.0.1 to 4.6.0 by @dependabot in #245
• Bump pypa/gh-action-pypi-publish from 1.10.0 to 1.10.3 by @dependabot in #244
• Preliminary implementation of nominal ele scale smearing by @phnattla in #252
• Ele smearing rnd seed fix by @phnattla in #266
• Implemented MET-xy corrections for Run2 UL by @guanfacin24 in #268
• More testing on skimmed NanoAOD sumgenweight propagation by @valsdav in #265
• Better sum of flow bins by @mmarchegiani in #271
• Move apptainer images to Python 3.11 and newer dask by @valsdav in #275
• update electron trigger scale factors by @felixzinn in #212
• Analysis execution with condor job submission - no dask by @valsdav in #277

New Contributors

• @mondalspandan made their first contribution in #253
• @guanfacin24 made their first contribution in #268

Full Changelog: 0.9.7...0.9.8

What's Changed

• improving dataset_query to add several metadata information by @uttiyasarkar in #188
• Basic methods to deal with GenParticles and GenJets, and more tagger options for Jets by @andreypz in #233
• Introducing dask for RWTH executor by @andreypz in #234
• Weights cache to local variable to avoid dask issues by @valsdav in #240
• Assign provenance to b-quarks in ttHbb dileptonic decays by @matintorkian in #239
• Update/law tasks by @felixzinn in #242
• Save rescaling of genWeight to make skimmed NanoAOD coherent by @valsdav in #241

New Contributors

• @matintorkian made their first contribution in #239

Full Changelog: 0.9.6...0.9.7

What's Changed

• Correct under/overflow bins by @mmarchegiani in #227
• Bump pypa/gh-action-pypi-publish from 1.8.11 to 1.10.0 by @dependabot in #226
• Fix the flow for compare option by @andreypz in #229
• Update executors_infn_af.py to better handle already configured Dask cluster by @ttedeschi in #231

New Contributors

• @ttedeschi made their first contribution in #231

Full Changelog: 0.9.5...0.9.6

Summary

• Generalized handling of common Weights and user-defined Weights
• Added first integration of LAW tasks to stear an analysis running PocketCoffea workflows
• Many improvements in the plotting scripts
• Added Dataset discovery cli to dynamically query for dataset and build the dataset definition file
• Cleaning up of the default skimming function (Breaking changes! see below)
• Added CDCI tests of utils and full configuration tests
• New parameters exploration CLI
• Added more executors
• Tested the Swan AF and INFN AF

Thanks to all the contributors!! :)

Breaking changes

Checkout the list of breaking changes in the docs page: https://pocketcoffea.readthedocs.io/en/stable/changelog.html#breaking-changes

What's Changed

• Small corrections to JEC code for Run3 by @phnattla in #183
• Adding JEC for Run3 and apply JES on data by @valsdav in #182
• Enhancement/plotting by @felixzinn in #187
• Creating new docker image based on el9 by @valsdav in #190
• Preparing executor for CERN Swan by @valsdav in #189
• Removed application of sumgenweight rescaling by default by @valsdav in #191
• Further VHcc changes. Important variables to be stored by @ValVau in #192
• Added clearing of query results in the CLI by @valsdav in #194
• Separated init and loading in the Configurator by @valsdav in #195
• New parameters exploration tool by @valsdav in #196
• Added new command to create a template configuration by @valsdav in #197
• fix check of genWeight in weights config by @felixzinn in #199
• Assignment of colors to samples by @felixzinn in #200
• Various improvements by @mmarchegiani in #207
• Fixed wildcard weights variation implementation by @valsdav in #208
• Enhancement/plotting: 10-color scheme added by @felixzinn in #206
• Removed default skim cuts from base processor by @valsdav in #193
• Improving weights handling by @valsdav in #147
• update plotting of systematic shifts by @felixzinn in #201
• Feature/law tasks by @felixzinn in #204
• Adding benchmarking histograms and throughput printout by @valsdav in #90
• Changes for CAT weights CDCI by @valsdav in #202
• Added test stage in CDCI by @valsdav in #209
• Add INFN AnalysisFacility executor by @valsdav in #210
• Enable blinding data for certain histograms by @andreypz in #211
• feat: plot separate histogram for signal samples, print category name by @andreypz in #214
• Bug fix for signal samples plotting by @andreypz in #215
• Update plotting by @felixzinn in #216
• Fix import with custom executor setup by @mmarchegiani in #217
• Color aliases by @mmarchegiani in #218
• Adding more full configurations tests by @valsdav in #213
• Update/law tasks by @felixzinn in #219
• feat: compare option for plotting, allowing shape comparison instead of data/mc by @andreypz in #221
• fix/plotting by @felixzinn in #220

Full Changelog: 0.9.4...0.9.5

Full Changelog: 0.9.3...0.9.4

What's Changed

• Implementing JECs for 2022 and 2023 eras by @andreypz in #174
• Enhancement/plotting by @felixzinn in #178
• Allow INFN redirector to be used when no sites are found with -ws option by @andreypz in #176

Full Changelog: 0.9.2...0.9.3

PocketCoffea v0.9.0

• New executor plugin setup to support multiple sites and analysis facilities
• New defaults for Run3 corrections
• Improve CLI interface and unified scripts under pocket-coffea command
• Added dataset-discovery-cli to build dataset_definition files

What's Changed

• New parameters setup and Configurator improvements by @valsdav in #73
• Working on improving sumgenweights and subsamples handling by @valsdav in #79
• Update leptons.py by @ryanm124 in #82
• Fixed to pileup scale factor input branch by @valsdav in #83
• Adapted plotting script to update output format by @valsdav in #84
• Enhancing plotting code by @mmarchegiani in #98
• Define new metrics for matching by @mmarchegiani in #99
• B-tag weights in YAML scheme by @mmarchegiani in #100
• Interactive plotting by @mmarchegiani in #101
• Workflow management with law by @mmarchegiani in #102
• Update columns_manager.py to allow for branches using event parameters by @ryanm124 in #103
• Modifications need for workflows to work at RWTH by @andreypz in #97
• Bugfix: Decorrelate JERC between AK4 and AK8 jets by @mmarchegiani in #107
• Improving plotting scripts by @andreypz in #106
• Fix crash due to missing rescale_sample parameter in plotting by @andreypz in #112
• Adding jet C-tagging, improving Parsl runner, logger, and minor bugfixes by @andreypz in #118
• Improved plotting by @mmarchegiani in #115
• feat: Improved executors interface and generalized runner script by @valsdav in #121
• Make it run on 2022 and 2023 datasets; and sign(genWeight) scaling by @andreypz in #125
• Improve the CLI using click by @valsdav in #122
• Add coffea.casa executor by @oshadura in #135
• Change executor from T3_PSI_CH to T3_CH_PSI by @matteomalucchi in #139
• JET PU Id for 2016 and naf executor by @phnattla in #142
• Executors based on parsl for DESY NAF by @valsdav in #129
• Fixed errors in make_plots.py related to click by @phnattla in #143
• Improved b-tagging parameters by @mmarchegiani in #144
• Improved plotting by @mmarchegiani in #148
• Naf executor fixes for new runner by @phnattla in #149
• Added functionality to run on RWTH parsl/condor by @uttiyasarkar in #150
• c-tagging shape SFs by @andreypz in #134
• Run3 sf defaults additions by @phnattla in #154
• Adding Run2 sf_ele default mappings by @phnattla in #161
• Additional ctag SF norm corrections by @andreypz in #158
• Add Purdue Analysis Facility executor (Dask Gateway) by @kondratyevd in #163
• Update Mu SFs for latest convention from POG by @andreypz in #164
• Run3 sf file name defaults by @phnattla in #166
• Added back the dataset discovery cli in PocketCoffea by @valsdav in #170
• Moving to pyproject.toml setup by @valsdav in #172

New Contributors

• @ryanm124 made their first contribution in #82
• @andreypz made their first contribution in #97
• @oshadura made their first contribution in #135
• @matteomalucchi made their first contribution in #139
• @phnattla made their first contribution in #142
• @uttiyasarkar made their first contribution in #150
• @kondratyevd made their first contribution in #163

Full Changelog: 0.1...0.9.0