Originally created by @asgalon on GitHub (Mar 9, 2026).
This showed up when running the test_unit_sigmoid cell:
That is a bit of a mixed message there. Looking at the test, the assertion holds for the Tensor([-1000,1000]), so the warning for overflow is probably expected. Perhaps it would be a good idea to add a comment saying so?
Same happens in the module integration test below, and in the Aha moment output
Originally created by @asgalon on GitHub (Mar 9, 2026).
This showed up when running the test_unit_sigmoid cell:
<img width="1760" height="893" alt="Image" src="https://github.com/user-attachments/assets/b079b924-97d2-4b8a-b395-2e86bcca5ce8" />
That is a bit of a mixed message there. Looking at the test, the assertion holds for the Tensor([-1000,1000]), so the warning for overflow is probably expected. Perhaps it would be a good idea to add a comment saying so?
Same happens in the module integration test below, and in the Aha moment output
I faced this issue too, I think the np.exp() function with float32 dtype is not able to handle larger numbers. I tried fixing this.
Below was the claude response:
Based on the below, 88.72 is max else it will overflow.
float32: np.exp(x) overflows when x > 88.72
float64: np.exp(x) overflows when x > 709.78
Workaround for the Sigmoid forward function:
exp_neg_abs = np.exp(-np.abs(x.data)) # always safe, always tiny
result = np.where(
x.data >= 0,
1.0 / (1.0 + exp_neg_abs), # positive x → standard formula
exp_neg_abs / (1.0 + exp_neg_abs) # negative x → rearranged formula
)
return Tensor(result)
@adityamulik commented on GitHub (Mar 9, 2026):
I faced this issue too, I think the np.exp() function with float32 dtype is not able to handle larger numbers. I tried fixing this.
Below was the claude response:
Based on the below, 88.72 is max else it will overflow.
```
float32: np.exp(x) overflows when x > 88.72
float64: np.exp(x) overflows when x > 709.78
```
Workaround for the Sigmoid forward function:
```
exp_neg_abs = np.exp(-np.abs(x.data)) # always safe, always tiny
result = np.where(
x.data >= 0,
1.0 / (1.0 + exp_neg_abs), # positive x → standard formula
exp_neg_abs / (1.0 + exp_neg_abs) # negative x → rearranged formula
)
return Tensor(result)
```
I think that the overflow is expected here. The problem is merely that the warning is a bit confusing because it looks like something that should be fixed, while the test explicitly checks that the result is within the expected range when exp overflows. So either this warning should be suppressed, or explicitly advertised as expected. The red color does not really help here. The assertion passes and looks useable for the expected use cases of Sigmoid, so the test result for tensor values that cause exp to overflow is correct. There is no need to complicate the algorithm, just to get rid of the confusing warning, I'd say.
@asgalon commented on GitHub (Mar 10, 2026):
I think that the overflow is expected here. The problem is merely that the warning is a bit confusing because it looks like something that should be fixed, while the test explicitly checks that the result is within the expected range when exp overflows. So either this warning should be suppressed, or explicitly advertised as expected. The red color does not really help here. The assertion passes and looks useable for the expected use cases of Sigmoid, so the test result for tensor values that cause exp to overflow is correct. There is no need to complicate the algorithm, just to get rid of the confusing warning, I'd say.
just not sure where is the best place. On the other hand, this hides the fact that an overflow happens, so from an educational standpoint it may be preferable to just put a note in before it happens that this warning is expected. That is more a question of style, perhaps.
@asgalon commented on GitHub (Mar 10, 2026):
I think the correct way to deal with this is to set
```
np.warnings.filterwarnings('ignore', category=RuntimeWarning)
```
just not sure where is the best place. On the other hand, this hides the fact that an overflow happens, so from an educational standpoint it may be preferable to just put a note in before it happens that this warning is expected. That is more a question of style, perhaps.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @asgalon on GitHub (Mar 9, 2026).
This showed up when running the test_unit_sigmoid cell:
That is a bit of a mixed message there. Looking at the test, the assertion holds for the Tensor([-1000,1000]), so the warning for overflow is probably expected. Perhaps it would be a good idea to add a comment saying so?
Same happens in the module integration test below, and in the Aha moment output
@adityamulik commented on GitHub (Mar 9, 2026):
I faced this issue too, I think the np.exp() function with float32 dtype is not able to handle larger numbers. I tried fixing this.
Below was the claude response:
Based on the below, 88.72 is max else it will overflow.
Workaround for the Sigmoid forward function:
@asgalon commented on GitHub (Mar 10, 2026):
I think that the overflow is expected here. The problem is merely that the warning is a bit confusing because it looks like something that should be fixed, while the test explicitly checks that the result is within the expected range when exp overflows. So either this warning should be suppressed, or explicitly advertised as expected. The red color does not really help here. The assertion passes and looks useable for the expected use cases of Sigmoid, so the test result for tensor values that cause exp to overflow is correct. There is no need to complicate the algorithm, just to get rid of the confusing warning, I'd say.
@asgalon commented on GitHub (Mar 10, 2026):
I think the correct way to deal with this is to set
just not sure where is the best place. On the other hand, this hides the fact that an overflow happens, so from an educational standpoint it may be preferable to just put a note in before it happens that this warning is expected. That is more a question of style, perhaps.