[BUG] --rope-scaling-factor is silently ignored: GPT models always use the default factor of 8.0

[Alvorecer721]

Describe the bug

--rope-scaling-factor is parsed but never forwarded to the model. gpt_config_from_args sets rope_scaling from --use-rope-scaling but not the factor, so GPTModelConfig.rope_scaling_factor keeps its dataclass default of 8.0 regardless of the command line. The same gap exists in gpt_builders.py (used by train_rl.py and the checkpoint converter tools) and in megatron/post_training/model_builder.py.

Training with the Llama 3.1 factor of 8.0 works by coincidence of defaults. Any other value, e.g. 32.0 for Llama 3.2 style scaling, silently trains with 8.0. The heterogeneous checkpoint path even validates args.rope_scaling_factor against the HF config's rope_scaling["factor"], then drops the validated value anyway.

#2902 fixed this on the old file layout but was closed without being merged, and the training migration (#4741) carried the gap into the new gpt_config_from_args path.

Steps/Code to reproduce bug

import sys
from megatron.training.arguments import parse_args, validate_args
from megatron.training.argument_utils import gpt_config_from_args

sys.argv = [
    'pretrain_gpt.py',
    '--num-layers', '2', '--hidden-size', '64', '--num-attention-heads', '4',
    '--seq-length', '128', '--max-position-embeddings', '131072',
    '--micro-batch-size', '1', '--vocab-size', '128256',
    '--position-embedding-type', 'rope', '--no-rope-fusion',
    '--use-rope-scaling', '--rope-scaling-factor', '32.0',
]
args = parse_args()
validate_args(args)
cfg = gpt_config_from_args(args)
print('args.rope_scaling_factor =', args.rope_scaling_factor)
print('cfg.rope_scaling         =', cfg.rope_scaling)
print('cfg.rope_scaling_factor  =', cfg.rope_scaling_factor)

Output on current main (3aee84c):

args.rope_scaling_factor = 32.0
cfg.rope_scaling         = True
cfg.rope_scaling_factor  = 8.0

Expected behavior

cfg.rope_scaling_factor == 32.0, so the configured factor reaches RotaryEmbedding.

Additional context

pretrain_vlm.py and examples/multimodal/model.py have the same issue on the LLaVA path: they pass language_rope_scaling but never language_rope_scaling_factor. That is a separate call surface and not covered by the linked PR.