github-starred/cs249r_book
2
0
Fork 0
mirror of https://github.com/harvard-edge/cs249r_book.git synced 2026-07-17 08:28:07 -05:00
Code Issues 276 Packages Projects Releases 26 Wiki Activity

[PR #1874] [CLOSED] fix(quantization): constant-tensor zero_point clamp silently corrupts values outside [-127,127] #36493

New Issue
Closed
opened 2026-07-16 00:22:30 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-07-16 00:22:30 -05:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/harvard-edge/cs249r_book/pull/1874
Author: @Shashank-Tripathi-07
Created: 6/16/2026
Status: ❌ Closed

Base: main ← Head: fix/quantization-constant-tensor-scale

📝 Commits (1)

  • a75a96f fix quantization of constant tensors when the value is outside INT8 range

📊 Changes

1 file changed (+16 additions, -7 deletions)

View changed files

📝 tinytorch/src/15_quantization/15_quantization.py (+16 -7)

📄 Description

What breaks

quantize_int8() handles constant tensors (all elements equal to c) with:

scale = 1.0
zero_point = int(np.clip(np.round(-c), INT8_MIN_VALUE, INT8_MAX_VALUE))

The dequantization identity is (q - zero_point) * scale = original_value. With q=0 for all elements, this requires zero_point = -c. But np.clip silently caps zero_point at [-128, 127], so for any |c| > 127:

c = 500 => zero_point = clip(-500, -128, 127) = -128
dequantize: (0 - (-128)) * 1.0 = 128  ≠  500

No error, no warning -- just silently wrong values.

Fix

Three cases based on |c|:

  • c == 0: scale=1.0, zero_point=0
  • |c| <= 127: scale=1.0, zero_point=-round(c) (original logic, always in range)
  • |c| > 127: set zero_point = -1 (positive c) or +1 (negative c), compute scale = |c| so that (0 - zero_point) * scale = c exactly -- no clamping needed

Test plan

  • pytest tinytorch/tests/15_quantization/ passes
  • dequantize(quantize(Tensor([[500.0, 500.0]]))) returns values close to 500.0
  • dequantize(quantize(Tensor([[2.0, 2.0]]))) still returns values close to 2.0

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/harvard-edge/cs249r_book/pull/1874 **Author:** [@Shashank-Tripathi-07](https://github.com/Shashank-Tripathi-07) **Created:** 6/16/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `fix/quantization-constant-tensor-scale` --- ### 📝 Commits (1) - [`a75a96f`](https://github.com/harvard-edge/cs249r_book/commit/a75a96fa7c03f225f255b3284eeed65679c913d8) fix quantization of constant tensors when the value is outside INT8 range ### 📊 Changes **1 file changed** (+16 additions, -7 deletions) <details> <summary>View changed files</summary> 📝 `tinytorch/src/15_quantization/15_quantization.py` (+16 -7) </details> ### 📄 Description ## What breaks `quantize_int8()` handles constant tensors (all elements equal to `c`) with: ```python scale = 1.0 zero_point = int(np.clip(np.round(-c), INT8_MIN_VALUE, INT8_MAX_VALUE)) ``` The dequantization identity is `(q - zero_point) * scale = original_value`. With `q=0` for all elements, this requires `zero_point = -c`. But `np.clip` silently caps `zero_point` at `[-128, 127]`, so for any `|c| > 127`: ``` c = 500 => zero_point = clip(-500, -128, 127) = -128 dequantize: (0 - (-128)) * 1.0 = 128 ≠ 500 ``` No error, no warning -- just silently wrong values. ## Fix Three cases based on `|c|`: - `c == 0`: `scale=1.0, zero_point=0` - `|c| <= 127`: `scale=1.0, zero_point=-round(c)` (original logic, always in range) - `|c| > 127`: set `zero_point = -1` (positive c) or `+1` (negative c), compute `scale = |c|` so that `(0 - zero_point) * scale = c` exactly -- no clamping needed ## Test plan - [ ] `pytest tinytorch/tests/15_quantization/` passes - [ ] `dequantize(quantize(Tensor([[500.0, 500.0]])))` returns values close to 500.0 - [ ] `dequantize(quantize(Tensor([[2.0, 2.0]])))` still returns values close to 2.0 --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
GiteaMirror added the pull-request label 2026-07-16 00:22:30 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
area: book
area: collabs
area: kits
area: labs
area: socratiq
area: staffml
area: tinytorch
area: tools
area: website
bug
dependencies
format: epub
format: pdf
javascript
link-health
link-rot
priority-high
pull-request
Mirrored from GitHub Pull Request
 staffml
type: bug
type: citation
type: code
type: errata
type: improvement
type: new
type: question
vault-sli
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/cs249r_book#36493
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?