[chessing234]

Summary

MaskedSoftmaxCELoss.forward used .mean(dim=1) over the sequence dimension, dividing each sample's loss by num_steps even though padding positions already have weight 0.

Root cause

train_seq2seq accumulates l.sum() and reports metric[0] / Y_valid_len.sum(). That expects per-sample loss to be the sum over valid tokens. With .mean, loss is scaled down by num_steps, so the final metric is off by roughly num_steps / valid_len and training under-penalizes EOS (sequences stay long).

Fix

Use .sum(dim=1) so only valid-token cross-entropy contributes; downstream normalization by total valid tokens is then correct.

Fixes #2076

Made with Cursor