Results dont match paper

[DominikVincent]

Hello,

I now trained as described and confirmed by you, see #20.

However, the results don't match what you report in the paper. Most of the metrics are slightly worse; sometimes, the metrics are off by a big margin e.g. motion catptioning R3 precision 0.83 vs 0.78:
T2M:

M2T:

Can you upload the exact training config or explain what the issue is on why the results don't quite match?

Thanks a lot for your help :)

[anon123x]

@DominikVincent
I re-trained the model stage-I again, but the videos from validate_motion still don’t make sense. How do the visual results look on your side?
@BoifZ

[anon123x]

@DominikVincent Did you solve the problem? How about the visual results?

[BoifZ]

@DominikVincent We have update the config files. Maybe try OPTIM.params.lr:1e-4 and OPTIM.params_diff.lr=2e-4 for Stage1.

Here is our validation metrics on HumanML3D dataset, after Stage1 training (REPLICATION_TIMES: 2).
T2M:

M2T:

[anon123x]

Please refer to the blow validation video of Stage 1

Epoch 29

M001816.mp4

Epoch 49

M004294.mp4

[DominikVincent]

@anon123x i get similar visual results. it looks like it's not working at all