Skip to content

fix: correct modified Huber loss gradient in ml/incr/binary-classification - #15317

Open
Abhist17 wants to merge 1 commit into
stdlib-js:developfrom
Abhist17:fix/modified-huber-loss-gradient-factor
Open

Abhist17 wants to merge 1 commit into
stdlib-js:developfrom
Abhist17:fix/modified-huber-loss-gradient-factor

Conversation

@Abhist17

@Abhist17 Abhist17 commented Sep 18, 2026

Copy link
Copy Markdown

Description

Fixes #13111. The else branch of _modifiedHuberLoss in ml/incr/binary-classification's Model was missing a factor of 2 in the "quadratically smoothed" region, and was being applied unconditionally for d = y*f(x) < 1 — including the d >= 1 region, where the modified Huber loss (and therefore its gradient) is exactly zero.

This goes slightly beyond what #13111 literally flagged (just the missing 2): I independently re-derived the three-region gradient against the cited references before changing anything, since applying only the factor-of-2 fix without also bounding the branch at d < 1.0 would still be wrong for d >= 1.

Per the modified Huber loss definition (Wikipedia, "Variant for classification"; scikit-learn's _sgd_fast.pyx.tp ModifiedHuber.dloss), the gradient of the loss w.r.t. the decision value p = y*f(x) is:

  • p >= 1: 0
  • -1 <= p < 1: -2*y*(1-p)
  • p < -1: -4*y

Since the weight update here is w += eta * (-dloss/dp) * x, the middle region needs 2*eta*(y - p*y) (not eta*(y - p*y)), and the p >= 1 region needs no update at all. The p < -1 branch (4.0*eta*y) was already correct and is unchanged.

Testing

Added three new test cases to test/test.model.js covering each region (p < -1, -1 <= p < 1, p >= 1), confirming:

  • the factor-of-2 fix in the quadratic region,
  • the model weight vector is left unchanged when p >= 1,
  • the existing p < -1 branch is unaffected.
$ node test/test.model.js
TAP version 13
# main export is a function
ok 1 ...
ok 2 main export is a function
# the modified Huber loss update includes a `2` factor when `-1 <= y*f(x) < 1` (see stdlib-js/stdlib#13111)
ok 3 includes the `2` factor in the update
# the modified Huber loss does not update the model weight vector when `y*f(x) >= 1`
ok 4 leaves the model weight vector unchanged
# the modified Huber loss update matches the existing `-4*eta*y` branch when `y*f(x) < -1`
ok 5 applies the unchanged `d < -1` branch

1..5
# tests 5
# pass  5

eslint on both touched files is clean (aside from a pre-existing, unrelated spelling warning on "pegasos").

Related Issues

Fixes #13111

Checklist

…cation`

The `else` branch of `_modifiedHuberLoss` was missing a factor of `2`
and was applied unconditionally for `d = y*f(x) < 1`, including the
`d >= 1` region where the loss (and its gradient) is exactly zero.

Per the modified Huber loss definition (Wikipedia, "Variant for
classification"; scikit-learn's `_sgd_fast.pyx.tp` `ModifiedHuber`),
the gradient of the loss w.r.t. the decision value `p = y*f(x)` is:

- `p >= 1`:  `0`
- `-1 <= p < 1`:  `-2*y*(1-p)`
- `p < -1`:  `-4*y`

The weight update is `w += eta * (-dloss/dp) * x`, so the middle
region needs `2*eta*(y - p*y)`, not `eta*(y - p*y)`, and the
`p >= 1` region needs no update at all.

Fixes stdlib-js#13111
@Abhist17
Abhist17 requested a review from a team September 18, 2026 06:48
@stdlib-bot stdlib-bot added First-time Contributor A pull request from a contributor who has never previously committed to the project repository. Needs Review A pull request which needs code review. labels Sep 18, 2026
@stdlib-bot

Copy link
Copy Markdown
Contributor

Hello! Thank you for your contribution to stdlib.

We noticed that the contributing guidelines acknowledgment is missing from your pull request. Here's what you need to do:

  1. Please read our contributing guidelines.

  2. Update your pull request description to include this checked box:

    - [x] Read, understood, and followed the [contributing guidelines](https://github.com/stdlib-js/stdlib/blob/develop/CONTRIBUTING.md)

This acknowledgment confirms that you've read the guidelines, which include:

  • The developer's certificate of origin
  • Your agreement to license your contributions under the project's terms

We can't review or accept contributions without this acknowledgment.

Thank you for your understanding and cooperation. We look forward to reviewing your contribution!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

First-time Contributor A pull request from a contributor who has never previously committed to the project repository. Needs Review A pull request which needs code review.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG]: missing multiplication in _modifiedHuberLoss implementation in ml/incr/binary-classification

2 participants