Foundations --> Transformers #36

New Issue

GiteaMirror · 2025-11-02T00:02:04-05:00

GiteaMirror commented

2025-11-02 00:02:04 -05:00

Originally created by @shashankvasisht on GitHub (Jun 14, 2022).

Hi Goku... I am really thankful for all your amazing tutorials.

I however was facing some issues in the Transformers lecture. There are a few minor bugs here with missing variables and imports; which was not an issue.

The training code however is missing the block:

# Train
best_model = trainer.train(
    num_epochs, patience, train_dataloader, val_dataloader)

Also when i wrote this and ran it, I got an error:

/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:14: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  
/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:15: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  from ipykernel import kernelapp as app
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
[<ipython-input-68-8d0f0dee99db>](https://localhost:8080/#) in <module>()
      1 # Train
      2 best_model = trainer.train(
----> 3     num_epochs, patience, train_dataloader, val_dataloader)

6 frames
[/usr/local/lib/python3.7/dist-packages/torch/nn/functional.py](https://localhost:8080/#) in dropout(input, p, training, inplace)
   1277     if p < 0.0 or p > 1.0:
   1278         raise ValueError("dropout probability has to be between 0 and 1, " "but got {}".format(p))
-> 1279     return _VF.dropout_(input, p, training) if inplace else _VF.dropout(input, p, training)
   1280 
   1281 

TypeError: dropout(): argument 'input' (position 1) must be Tensor, not str

Apparently, the issue comes from the line :

seq, pool = self.transformer(input_ids=ids, attention_mask=masks)

wherein the "pool" returned is of class string.
Upon printing the type and the value of it i get the following :

<class 'str'>
pooler_output

Can you please have a look into this.
Thanks in Advance!!

Originally created by @shashankvasisht on GitHub (Jun 14, 2022). Hi Goku... I am really thankful for all your amazing tutorials. I however was facing some issues in the Transformers lecture. There are a few minor bugs here with missing variables and imports; which was not an issue. The training code however is missing the block: ``` # Train best_model = trainer.train( num_epochs, patience, train_dataloader, val_dataloader) ``` Also when i wrote this and ran it, I got an error: ``` /usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:14: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor). /usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:15: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor). from ipykernel import kernelapp as app --------------------------------------------------------------------------- TypeError Traceback (most recent call last) [<ipython-input-68-8d0f0dee99db>](https://localhost:8080/#) in <module>() 1 # Train 2 best_model = trainer.train( ----> 3 num_epochs, patience, train_dataloader, val_dataloader) 6 frames [/usr/local/lib/python3.7/dist-packages/torch/nn/functional.py](https://localhost:8080/#) in dropout(input, p, training, inplace) 1277 if p < 0.0 or p > 1.0: 1278 raise ValueError("dropout probability has to be between 0 and 1, " "but got {}".format(p)) -> 1279 return _VF.dropout_(input, p, training) if inplace else _VF.dropout(input, p, training) 1280 1281 TypeError: dropout(): argument 'input' (position 1) must be Tensor, not str ``` Apparently, the issue comes from the line : ``` seq, pool = self.transformer(input_ids=ids, attention_mask=masks) ``` wherein the "pool" returned is of class string. Upon printing the type and the value of it i get the following : ``` <class 'str'> pooler_output ``` Can you please have a look into this. Thanks in Advance!!

GiteaMirror closed this issue

2025-11-02 00:02:04 -05:00

GiteaMirror commented

2025-11-02 00:02:05 -05:00

@GokuMohandas commented on GitHub (Jun 14, 2022):

HI @shashankvasisht, based on the error messages, it looks like you're running this on a local notebook instead of running on Google colab directly? If not, make sure you have the exact version of the transformers (3.0.2) package (their future releases might be not be backwards compatible) and other packages as well (use pip freeze inside the notebook to check). I just reran the notebook and it ran successfully to completion.

@GokuMohandas commented on GitHub (Jun 14, 2022): HI @shashankvasisht, based on the error messages, it looks like you're running this on a local notebook instead of running on [Google colab](https://colab.research.google.com/github/GokuMohandas/MadeWithML/blob/main/notebooks/15_Transformers.ipynb) directly? If not, make sure you have the exact version of the transformers (3.0.2) package (their future releases might be not be backwards compatible) and other packages as well (use `pip freeze` inside the notebook to check). I just reran the notebook and it ran successfully to completion.

GiteaMirror commented

2025-11-02 00:02:05 -05:00

@knosing commented on GitHub (Dec 8, 2022):

@shashankvasisht
Hi, I just got the same error and found the issue.
This error pops up if you are using the newer 4.0 ver of transformers.
So one solution is what @GokuMohandas recommended and the other would be to add the argument return_dict=False when you pass inputs to your model:
seq, pool = self.transformer(input_ids=ids, attention_mask=masks, return_dict=False)

Rest all works fine, cheers!

@knosing commented on GitHub (Dec 8, 2022): @shashankvasisht Hi, I just got the same error and found the issue. This error pops up if you are using the newer 4.0 ver of transformers. So one solution is what @GokuMohandas recommended and the other would be to add the argument `return_dict=False` when you pass inputs to your model: `seq, pool = self.transformer(input_ids=ids, attention_mask=masks, return_dict=False)` Rest all works fine, cheers!

GiteaMirror referenced this issue

2025-11-02 00:04:32 -05:00

[PR #36] [MERGED] added binder #110

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/Made-With-ML#36