github-starred/Made-With-ML
2
0
Fork 0
mirror of https://github.com/GokuMohandas/Made-With-ML.git synced 2026-03-09 07:12:37 -05:00
Code Issues 21 Packages Projects Releases Wiki Activity

Foundations --> Logistic regression feedback/errors #28

New Issue
Closed
opened 2025-11-02 00:01:50 -05:00 by GiteaMirror · 1 comment
GiteaMirror commented 2025-11-02 00:01:50 -05:00
Owner
Copy Link

Originally created by @gitgithan on GitHub (Oct 17, 2021).

  1. Webpage says W dimension is Dx1 but notebook says DxC. Prefer webpage to also show DxC to expose people to the more general multi-class W

  2. Two errors causing notebook to not run top-down
    a. Extra single quote behind k: plt.scatter(X[:, 0], X[:, 1], c=[colors[_y] for _y in y], s=25, edgecolors="k"')
    b. SyntaxError: Double quotes to index dictionary early closing double quotes for f-string (happens in 2 cells) print (f"m:b = {class_counts["malignant"]/class_counts["benign"]:.2f}")

  3. Hope the matrix calculus section had more explanation, feels to me like for people who understand it, they won't need the formulas, but for people who don't understand, it doesn't help much.
    Some questions I had going through that section.

    1. What's the physical meaning of y and j indexes in loss formula? Why does W have y subscript in numerator and sum across j in denominator? Why does denominator's W have no subscript. Seems to me like both y, j refer to one of the classes in a set of unique classes.
    2. In gradients formula, what is the physical meaning of Wy Wj, and why are we differentiating wrt to them?
    3. Why did i disappear from subscript of X in gradients section
    4. Why do some W have subscripts y/j while some W don't have any subscript
    5. Linking to some derivations like these (https://towardsdatascience.com/derivative-of-the-softmax-function-and-the-categorical-cross-entropy-loss-ffceefc081d1) would be very helpful

  4. How did db = np.sum(dscores, axis=0, keepdims=True) implementation come about? Was expecting a formula version describing gradient wrt bias but previous it's mentioned We'll leave the bias weights out for now to avoid complicating the backpropagation calculation

  5. W_{unscaled} includes sum in formula which it shouldn't?

Originally created by @gitgithan on GitHub (Oct 17, 2021). 1. Webpage says W dimension is Dx1 but notebook says DxC. Prefer webpage to also show DxC to expose people to the more general multi-class W 2. Two errors causing notebook to not run top-down a. Extra single quote behind k: `plt.scatter(X[:, 0], X[:, 1], c=[colors[_y] for _y in y], s=25, edgecolors="k"')` b. SyntaxError: Double quotes to index dictionary early closing double quotes for f-string (happens in 2 cells) `print (f"m:b = {class_counts["malignant"]/class_counts["benign"]:.2f}")` 3. Hope the matrix calculus section had more explanation, feels to me like for people who understand it, they won't need the formulas, but for people who don't understand, it doesn't help much. Some questions I had going through that section. 1. What's the physical meaning of y and j indexes in loss formula? Why does W have y subscript in numerator and sum across j in denominator? Why does denominator's W have no subscript. Seems to me like both y, j refer to one of the classes in a set of unique classes. 2. In gradients formula, what is the physical meaning of Wy Wj, and why are we differentiating wrt to them? 3. Why did i disappear from subscript of X in gradients section 4. Why do some W have subscripts y/j while some W don't have any subscript 5. Linking to some derivations like these (https://towardsdatascience.com/derivative-of-the-softmax-function-and-the-categorical-cross-entropy-loss-ffceefc081d1) would be very helpful 4. How did `db = np.sum(dscores, axis=0, keepdims=True)` implementation come about? Was expecting a formula version describing gradient wrt bias but previous it's mentioned `We'll leave the bias weights out for now to avoid complicating the backpropagation calculation` 5. `W_{unscaled}` includes sum in formula which it shouldn't?
GiteaMirror commented 2025-11-02 00:01:51 -05:00
Author
Owner
Copy Link

@GokuMohandas commented on GitHub (Oct 18, 2021):

  1. good idea. By the way, the notebooks are no longer up to date and I'm going to edit all of them this winter. For now, let's use the webpage as the up-to-date source and I'll just port over all the code from the webpage to the notebooks later this year.
  2. Good catch, must've happened when I tried to use double quotes everywhere and did an erroneous replace.
  3. I'll definitely clean these up a lot more when I sit down to polish all of these. I haven't touched these specific lessons in almost 6 years now (just moved them to here) so I'll go over them this winter.
  4. I'll include this in the update too. But I left out bias in the math (to avoid clutter) but not in the code.
  5. Correct, for same reasoning we discussed last week. I'll just make the change to the webpage now but will update notebooks later when I polish them.

I really appreciate these @gitgithan!

@GokuMohandas commented on GitHub (Oct 18, 2021): 1. good idea. By the way, the notebooks are no longer up to date and I'm going to edit all of them this winter. For now, let's use the webpage as the up-to-date source and I'll just port over all the code from the webpage to the notebooks later this year. 2. Good catch, must've happened when I tried to use double quotes everywhere and did an erroneous replace. 3. I'll definitely clean these up a lot more when I sit down to polish all of these. I haven't touched these specific lessons in almost 6 years now (just moved them to here) so I'll go over them this winter. 4. I'll include this in the update too. But I left out bias in the math (to avoid clutter) but not in the code. 5. Correct, for same reasoning we discussed last week. I'll just make the change to the webpage now but will update notebooks later when I polish them. I really appreciate these @gitgithan!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
pull-request
Mirrored from GitHub Pull Request
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/Made-With-ML#28
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?