Unexpected PPL Results with 4:8 Sparsity on LLaMA-2-7B

[forlight123]

Hi, I'm trying to reproduce the results from the paper using the provided command for STBLLM on LLaMA-2-7B with the Wikitext-2 dataset.

When running without sparsity, the command:

python3 run.py /home/models/llama2-7b wikitext2 braq \
  --blocksize 128 \
  --salient_metric hessian \
  --prune_method si_structure \
  --reconstruction \
  --sparsity_type 4:8

produces a reasonable perplexity (PPL) of 10.1.

However, when I enable structured sparsity with --sparsity_ratio 0.5:

python3 run.py /home/models/llama2-7b wikitext2 braq \
  --blocksize 128 \
  --salient_metric hessian \
  --prune_method si_structure \
  --reconstruction \
  --sparsity_ratio 0.5 \
  --sparsity_type 4:8

the PPL jumps dramatically to 178.6, which seems inconsistent with the results reported in the paper :

Could you help clarify:

Whether this behavior is expected?
If there are additional hyperparameters or steps needed when applying 4:8 sparsity with BRAGPTQ?
Or if there might be a bug or version mismatch causing this discrepancy?
Thanks in advance for your help!

[pprp]

Thanks for your attention. This might be the version mismatch problem as there are several branches beding developed at the same time. I will check it and answer you a few days later.