Foundations --> Embeddings #32

New Issue

GiteaMirror · 2025-11-02T00:01:56-05:00

GiteaMirror commented

2025-11-02 00:01:56 -05:00

Originally created by @gitgithan on GitHub (Oct 19, 2021).

Typo under Model section: 3. We'll apply convolution via filters (filter_size, vocab_size, num_filters) should be embedding_dim to replace vocab_size?
Typo under Experiments: first have to decice
Typo under Interpretability padding our inputs before convolution to result is outputs is should be in
Could there be a general explanation of moving models/data across devices? My current understanding is that they have to be both on the same place (cpu/gpu). If on gpu, just stay on gpu through the whole train/eval/predict session. I couldn't understand why under Inference device = torch.device("cpu") moves things back to cpu.
interpretable_trainer.predict_step(dataloader) breaks with AttributeError: 'list' object has no attribute 'dim'. The precise step is F.softmax(z), where for interpretable_model, z is a list of 3 items and it was trying to softmax a list instead of a tensor.

Originally created by @gitgithan on GitHub (Oct 19, 2021). 1. Typo under Model section: `3. We'll apply convolution via filters (filter_size, vocab_size, num_filters)` should be `embedding_dim` to replace `vocab_size`? 2. Typo under Experiments: `first have to decice` 3. Typo under Interpretability `padding our inputs before convolution to result is outputs` `is` should be `in` 4. Could there be a general explanation of moving models/data across devices? My current understanding is that they have to be both on the same place (cpu/gpu). If on gpu, just stay on gpu through the whole train/eval/predict session. I couldn't understand why under Inference `device = torch.device("cpu")` moves things back to cpu. 5. `interpretable_trainer.predict_step(dataloader)` breaks with `AttributeError: 'list' object has no attribute 'dim'`. The precise step is `F.softmax(z)`, where for interpretable_model, z is a list of 3 items and it was trying to softmax a list instead of a tensor.

GiteaMirror closed this issue

2025-11-02 00:01:56 -05:00

GiteaMirror commented

2025-11-02 00:01:57 -05:00

@GokuMohandas commented on GitHub (Oct 19, 2021):

1-3: thanks for these.
4. You usually want to do inference on CPU/TPU (optimized for forward pass) so you don't waste a machine with a GPU (which is usually reserved for training with backprop).
5. good catch, I've updated the code on the webpage to look like this now:

# Get conv outputs
interpretable_model.eval()
conv_outputs = []
with torch.inference_mode():
    for i, batch in enumerate(dataloader):

        # Forward pass w/ inputs
        inputs, targets = batch[:-1], batch[-1]
        z = interpretable_model(inputs)

        # Store conv outputs
        conv_outputs.extend(z)

conv_outputs = np.vstack(conv_outputs)
print (conv_outputs.shape) # (len(filter_sizes), num_filters, max_seq_len)

@GokuMohandas commented on GitHub (Oct 19, 2021): 1-3: thanks for these. 4. You usually want to do inference on CPU/TPU (optimized for forward pass) so you don't waste a machine with a GPU (which is usually reserved for training with backprop). 5. good catch, I've updated the code on the webpage to look like this now: ```python # Get conv outputs interpretable_model.eval() conv_outputs = [] with torch.inference_mode(): for i, batch in enumerate(dataloader): # Forward pass w/ inputs inputs, targets = batch[:-1], batch[-1] z = interpretable_model(inputs) # Store conv outputs conv_outputs.extend(z) conv_outputs = np.vstack(conv_outputs) print (conv_outputs.shape) # (len(filter_sizes), num_filters, max_seq_len) ```

GiteaMirror commented

2025-11-02 00:01:57 -05:00

@JellenRoberts commented on GitHub (Oct 20, 2021):

Hi,

I have been part of this thread and have no idea how to end these emails.

Any idea?

Thanks!

On Tue, 19 Oct 2021 at 19:25, Goku Mohandas @.***>
wrote:

1-3: thanks for these.
4. You usually want to do inference on CPU/TPU (optimized for forward
pass) so you don't waste a machine with a GPU (which is usually reserved
for training with backprop).
5. good catch, I've updated the code on the webpage to look like this now:

Get conv outputsinterpretable_model.eval()conv_outputs = []with torch.inference_mode():
for i, batch in enumerate(dataloader):

    # Forward pass w/ inputs
    inputs, targets = batch[:-1], batch[-1]
    z = interpretable_model(inputs)

    # Store conv outputs
    conv_outputs.extend(z)
conv_outputs = np.vstack(conv_outputs)print (conv_outputs.shape) # (len(filter_sizes), num_filters, max_seq_len)

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/GokuMohandas/MadeWithML/issues/203#issuecomment-946939380,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ANXUXQ2SU6CV2M3SHLLZZEDUHWSYRANCNFSM5GIT5CAQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

@JellenRoberts commented on GitHub (Oct 20, 2021): Hi, I have been part of this thread and have no idea how to end these emails. Any idea? Thanks! On Tue, 19 Oct 2021 at 19:25, Goku Mohandas ***@***.***> wrote: > 1-3: thanks for these. > 4. You usually want to do inference on CPU/TPU (optimized for forward > pass) so you don't waste a machine with a GPU (which is usually reserved > for training with backprop). > 5. good catch, I've updated the code on the webpage to look like this now: > > # Get conv outputsinterpretable_model.eval()conv_outputs = []with torch.inference_mode(): > for i, batch in enumerate(dataloader): > > # Forward pass w/ inputs > inputs, targets = batch[:-1], batch[-1] > z = interpretable_model(inputs) > > # Store conv outputs > conv_outputs.extend(z) > conv_outputs = np.vstack(conv_outputs)print (conv_outputs.shape) # (len(filter_sizes), num_filters, max_seq_len) > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub > <https://github.com/GokuMohandas/MadeWithML/issues/203#issuecomment-946939380>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/ANXUXQ2SU6CV2M3SHLLZZEDUHWSYRANCNFSM5GIT5CAQ> > . > Triage notifications on the go with GitHub Mobile for iOS > <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> > or Android > <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. > >

GiteaMirror commented

2025-11-02 00:01:58 -05:00

@GokuMohandas commented on GitHub (Oct 20, 2021):

hey @JellenRoberts , I'm sorry but I'm not entirely sure either but I agree that we don't have to open issues this frequently. @gitgithan I sent you a LinkedIn request so let's chat on there for all future minor issues and clarifications because I think more than a thousand people are watching this repo so they are actively getting emails for every single conversation. We can share any large implications here for the whole community as we come across them.

@GokuMohandas commented on GitHub (Oct 20, 2021): hey @JellenRoberts , I'm sorry but I'm not entirely sure either but I agree that we don't have to open issues this frequently. @gitgithan I sent you a LinkedIn request so let's chat on there for all future minor issues and clarifications because I think more than a thousand people are watching this repo so they are actively getting emails for every single conversation. We can share any large implications here for the whole community as we come across them.

GiteaMirror referenced this issue

2025-11-02 00:04:23 -05:00

[PR #32] [MERGED] finished basics of computer vision #106

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/Made-With-ML#32