Many thanks for the kind sharing!!
When reproducing training results, the docker file specifies torch==1.8.1. However, torch at 1.8.1 had not introduced torch.concat yet, which will cause errors on this line of code:
https://github.com/EleutherAI/gpt-neox/blob/71df4d5017f9f4919566a11454fe3a507ffdc632/megatron/model/transformer.py#L432
Simply changing torch.concat to torch.cat can make it compatible.
Many thanks for the kind sharing!!
When reproducing training results, the docker file specifies
torch==1.8.1. However, torch at 1.8.1 had not introducedtorch.concatyet, which will cause errors on this line of code:https://github.com/EleutherAI/gpt-neox/blob/71df4d5017f9f4919566a11454fe3a507ffdc632/megatron/model/transformer.py#L432
Simply changing
torch.concattotorch.catcan make it compatible.