cuda/Indexing.cu:1289: indexSelectLargeIndex: block: [19,0,0], thread: [51,0,0] Assertion `srcIndex

Question

how to fix this error

Exception has occurred: RuntimeError
CUDA error: device-side assert triggered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.
  File "/mnt/k/MajorbotData/TraningCommentsModelPara/gpt2.py", line 89, in training_step
    loss = self(input_ids, attention_mask, labels)
  File "/mnt/k/MajorbotData/TraningCommentsModelPara/gpt2.py", line 73, in forward
    outputs = self.model(input_ids, attention_mask=attention_mask, labels=labels)
RuntimeError: CUDA error: device-side assert triggered

mrmucha · Answer

In my case that was just changing special token from tokenizer.add_special_tokens({'pad_token': '[PAD]'}) to tokenizer.pad_token = tokenizer.eos_token

Forum

cuda/Indexing.cu:1289: indexSelectLargeIndex: block: [19,0,0], thread: [51,0,0] Assertion `srcIndex < srcSelectDimSize` failed.