Train issues - decoder

[jesusbft]

Hey,

I testing the train party command and have a couple issues.

1 - the encoder in the party code don't match at all with the new party_european_langs.safetensors

2- I change the file model.py to represent the new encoder:

class RecognitionModel(L.LightningModule):
    def __init__(self,
                 (...)
                 encoder: str = 'swin_base_patch4_window12_384.ms_in22k',
                 (...)
        super().__init__()

3- However, I still have errors to train using the party_european_langs.safetensors:

The problem:

Model expects: encoder.layers_0.blocks.0... (underscore)
Checkpoint has: encoder.layers.0.blocks.0... (dot)

And for the adapter:

Model expects: adapter.adapter.0...
Checkpoint has: adapter.0...

I can't find how to fix this issue, maybe you know

party -d cuda:0 --workers 4  train -t dataset-party/train.lst -e dataset-party/val.lst -o models/Portuguese --load dataset-party/party_european_langs.safetensors
Using bfloat16 Automatic Mixed Precision (AMP)
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
HPU available: False, using: 0 HPUs
`Trainer(val_check_interval=1.0)` was configured so validation will run at the end of the training epoch..
Loading from checkpoint dataset-party/party_european_langs.safetensors.
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /home/jesus/venv/bin/party:8 in <module>                                                         │
│                                                                                                  │
│   5 from party.cli import cli                                                                    │
│   6 if __name__ == '__main__':                                                                   │
│   7 │   sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])                         │
│ ❱ 8 │   sys.exit(cli())                                                                          │
│   9                                                                                              │
│                                                                                                  │
│ /home/jesus/venv/lib/python3.12/site-packages/click/core.py:1161 in __call__                     │
│                                                                                                  │
│ /home/jesus/venv/lib/python3.12/site-packages/click/core.py:1082 in main                         │
│                                                                                                  │
│ /home/jesus/venv/lib/python3.12/site-packages/click/core.py:1697 in invoke                       │
│                                                                                                  │
│ /home/jesus/venv/lib/python3.12/site-packages/click/core.py:1443 in invoke                       │
│                                                                                                  │
│ /home/jesus/venv/lib/python3.12/site-packages/click/core.py:788 in invoke                        │
│                                                                                                  │
│ /home/jesus/venv/lib/python3.12/site-packages/click/decorators.py:33 in new_func                 │
│                                                                                                  │
│ /home/jesus/venv/party/party/cli/train.py:298 in train                                           │
│                                                                                                  │
│   295 │   │   elif load:                                                                         │
│   296 │   │   │   message(f'Loading from checkpoint {load}.')                                    │
│   297 │   │   │   if load.endswith('safetensors'):                                               │
│ ❱ 298 │   │   │   │   model = RecognitionModel.load_from_safetensors(load, **params)             │
│   299 │   │   │   elif load.endswith('ckpt'):                                                    │
│   300 │   │   │   │   model = RecognitionModel.load_from_checkpoint(load, **params)              │
│   301 │   │   │   else:                                                                          │
│                                                                                                  │
│ /home/jesus/venv/party/party/model.py:204 in load_from_safetensors                               │
│                                                                                                  │
│   201 │   │   from safetensors.torch import load_file                                            │
│   202 │   │                                                                                      │
│   203 │   │   module = cls(*args, **kwargs, pretrained=False)                                    │
│ ❱ 204 │   │   module.model.load_state_dict(load_file(file))                                      │
│   205 │   │   module.model.train()                                                               │
│   206 │   │   return module                                                                      │
│   207                                                                                            │
│                                                                                                  │
│ /home/jesus/venv/lib/python3.12/site-packages/torch/nn/modules/module.py:2629 in load_state_dict │
│                                                                                                  │
│   2626 │   │   │   │   )                                                                         │
│   2627 │   │                                                                                     │
│   2628 │   │   if len(error_msgs) > 0:                                                           │
│ ❱ 2629 │   │   │   raise RuntimeError(                                                           │
│   2630 │   │   │   │   "Error(s) in loading state_dict for {}:\n\t{}".format(                    │
│   2631 │   │   │   │   │   self.__class__.__name__, "\n\t".join(error_msgs)                      │
│   2632 │   │   │   │   )                                                                         │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
RuntimeError: Error(s) in loading state_dict for PartyModel:
        Missing key(s) in state_dict: "encoder.layers_0.blocks.0.norm1.weight", "encoder.layers_0.blocks.0.norm1.bias", "encoder.layers_0.blocks.0.attn.relative_position_bias_table",
"encoder.layers_0.blocks.0.attn.qkv.weight", "encoder.layers_0.blocks.0.attn.qkv.bias", "encoder.layers_0.blocks.0.attn.proj.weight", "encoder.layers_0.blocks.0.attn.proj.bias",
"encoder.layers_0.blocks.0.norm2.weight", "encoder.layers_0.blocks.0.norm2.bias", "encoder.layers_0.blocks.0.mlp.fc1.weight", "encoder.layers_0.blocks.0.mlp.fc1.bias",
"encoder.layers_0.blocks.0.mlp.fc2.weight", "encoder.layers_0.blocks.0.mlp.fc2.bias", "encoder.layers_0.blocks.1.norm1.weight", "encoder.layers_0.blocks.1.norm1.bias",
"encoder.layers_0.blocks.1.attn.relative_position_bias_table", "encoder.layers_0.blocks.1.attn.qkv.weight", "encoder.layers_0.blocks.1.attn.qkv.bias", "encoder.layers_0.blocks.1.attn.proj.weight",
"encoder.layers_0.blocks.1.attn.proj.bias", "encoder.layers_0.blocks.1.norm2.weight", "encoder.layers_0.blocks.1.norm2.bias", "encoder.layers_0.blocks.1.mlp.fc1.weight",
"encoder.layers_0.blocks.1.mlp.fc1.bias", "encoder.layers_0.blocks.1.mlp.fc2.weight", "encoder.layers_0.blocks.1.mlp.fc2.bias", "encoder.layers_1.downsample.norm.weight",
"encoder.layers_1.downsample.norm.bias", "encoder.layers_1.downsample.reduction.weight", "encoder.layers_1.blocks.0.norm1.weight", "encoder.layers_1.blocks.0.norm1.bias",
"encoder.layers_1.blocks.0.attn.relative_position_bias_table", "encoder.layers_1.blocks.0.attn.qkv.weight", "encoder.layers_1.blocks.0.attn.qkv.bias", "encoder.layers_1.blocks.0.attn.proj.weight",
"encoder.layers_1.blocks.0.attn.proj.bias", "encoder.layers_1.blocks.0.norm2.weight", "encoder.layers_1.blocks.0.norm2.bias", "encoder.layers_1.blocks.0.mlp.fc1.weight",
"encoder.layers_1.blocks.0.mlp.fc1.bias", "encoder.layers_1.blocks.0.mlp.fc2.weight", "encoder.layers_1.blocks.0.mlp.fc2.bias", "encoder.layers_1.blocks.1.norm1.weight",
"encoder.layers_1.blocks.1.norm1.bias", "encoder.layers_1.blocks.1.attn.relative_position_bias_table", "encoder.layers_1.blocks.1.attn.qkv.weight", "encoder.layers_1.blocks.1.attn.qkv.bias",
"encoder.layers_1.blocks.1.attn.proj.weight", "encoder.layers_1.blocks.1.attn.proj.bias", "encoder.layers_1.blocks.1.norm2.weight", "encoder.layers_1.blocks.1.norm2.bias",
"encoder.layers_1.blocks.1.mlp.fc1.weight", "encoder.layers_1.blocks.1.mlp.fc1.bias", "encoder.layers_1.blocks.1.mlp.fc2.weight", "encoder.layers_1.blocks.1.mlp.fc2.bias",
"encoder.layers_2.downsample.norm.weight", "encoder.layers_2.downsample.norm.bias", "encoder.layers_2.downsample.reduction.weight", "encoder.layers_2.blocks.0.norm1.weight",
"encoder.layers_2.blocks.0.norm1.bias", "encoder.layers_2.blocks.0.attn.relative_position_bias_table", "encoder.layers_2.blocks.0.attn.qkv.weight", "encoder.layers_2.blocks.0.attn.qkv.bias",
"encoder.layers_2.blocks.0.attn.proj.weight", "encoder.layers_2.blocks.0.attn.proj.bias", "encoder.layers_2.blocks.0.norm2.weight", "encoder.layers_2.blocks.0.norm2.bias",
"encoder.layers_2.blocks.0.mlp.fc1.weight", "encoder.layers_2.blocks.0.mlp.fc1.bias", "encoder.layers_2.blocks.0.mlp.fc2.weight", "encoder.layers_2.blocks.0.mlp.fc2.bias",
"encoder.layers_2.blocks.1.norm1.weight", "encoder.layers_2.blocks.1.norm1.bias", "encoder.layers_2.blocks.1.attn.relative_position_bias_table", "encoder.layers_2.blocks.1.attn.qkv.weight",
"encoder.layers_2.blocks.1.attn.qkv.bias", "encoder.layers_2.blocks.1.attn.proj.weight", "encoder.layers_2.blocks.1.attn.proj.bias", "encoder.layers_2.blocks.1.norm2.weight",
"encoder.layers_2.blocks.1.norm2.bias", "encoder.layers_2.blocks.1.mlp.fc1.weight", "encoder.layers_2.blocks.1.mlp.fc1.bias", "encoder.layers_2.blocks.1.mlp.fc2.weight",
"encoder.layers_2.blocks.1.mlp.fc2.bias", "encoder.layers_2.blocks.2.norm1.weight", "encoder.layers_2.blocks.2.norm1.bias", "encoder.layers_2.blocks.2.attn.relative_position_bias_table",
"encoder.layers_2.blocks.2.attn.qkv.weight", "encoder.layers_2.blocks.2.attn.qkv.bias", "encoder.layers_2.blocks.2.attn.proj.weight", "encoder.layers_2.blocks.2.attn.proj.bias",
"encoder.layers_2.blocks.2.norm2.weight", "encoder.layers_2.blocks.2.norm2.bias", "encoder.layers_2.blocks.2.mlp.fc1.weight", "encoder.layers_2.blocks.2.mlp.fc1.bias",
"encoder.layers_2.blocks.2.mlp.fc2.weight", "encoder.layers_2.blocks.2.mlp.fc2.bias", "encoder.layers_2.blocks.3.norm1.weight", "encoder.layers_2.blocks.3.norm1.bias",
"encoder.layers_2.blocks.3.attn.relative_position_bias_table", "encoder.layers_2.blocks.3.attn.qkv.weight", "encoder.layers_2.blocks.3.attn.qkv.bias", "encoder.layers_2.blocks.3.attn.proj.weight",
"encoder.layers_2.blocks.3.attn.proj.bias", "encoder.layers_2.blocks.3.norm2.weight", "encoder.layers_2.blocks.3.norm2.bias", "encoder.layers_2.blocks.3.mlp.fc1.weight",
"encoder.layers_2.blocks.3.mlp.fc1.bias", "encoder.layers_2.blocks.3.mlp.fc2.weight", "encoder.layers_2.blocks.3.mlp.fc2.bias", "encoder.layers_2.blocks.4.norm1.weight",
"encoder.layers_2.blocks.4.norm1.bias", "encoder.layers_2.blocks.4.attn.relative_position_bias_table", "encoder.layers_2.blocks.4.attn.qkv.weight", "encoder.layers_2.blocks.4.attn.qkv.bias",
"encoder.layers_2.blocks.4.attn.proj.weight", "encoder.layers_2.blocks.4.attn.proj.bias", "encoder.layers_2.blocks.4.norm2.weight", "encoder.layers_2.blocks.4.norm2.bias",
"encoder.layers_2.blocks.4.mlp.fc1.weight", "encoder.layers_2.blocks.4.mlp.fc1.bias", "encoder.layers_2.blocks.4.mlp.fc2.weight", "encoder.layers_2.blocks.4.mlp.fc2.bias",
"encoder.layers_2.blocks.5.norm1.weight", "encoder.layers_2.blocks.5.norm1.bias", "encoder.layers_2.blocks.5.attn.relative_position_bias_table", "encoder.layers_2.blocks.5.attn.qkv.weight",
"encoder.layers_2.blocks.5.attn.qkv.bias", "encoder.layers_2.blocks.5.attn.proj.weight", "encoder.layers_2.blocks.5.attn.proj.bias", "encoder.layers_2.blocks.5.norm2.weight",
"encoder.layers_2.blocks.5.norm2.bias", "encoder.layers_2.blocks.5.mlp.fc1.weight", "encoder.layers_2.blocks.5.mlp.fc1.bias", "encoder.layers_2.blocks.5.mlp.fc2.weight",
"encoder.layers_2.blocks.5.mlp.fc2.bias", "encoder.layers_2.blocks.6.norm1.weight", "encoder.layers_2.blocks.6.norm1.bias", "encoder.layers_2.blocks.6.attn.relative_position_bias_table",
"encoder.layers_2.blocks.6.attn.qkv.weight", "encoder.layers_2.blocks.6.attn.qkv.bias", "encoder.layers_2.blocks.6.attn.proj.weight", "encoder.layers_2.blocks.6.attn.proj.bias",
"encoder.layers_2.blocks.6.norm2.weight", "encoder.layers_2.blocks.6.norm2.bias", "encoder.layers_2.blocks.6.mlp.fc1.weight", "encoder.layers_2.blocks.6.mlp.fc1.bias",
"encoder.layers_2.blocks.6.mlp.fc2.weight", "encoder.layers_2.blocks.6.mlp.fc2.bias", "encoder.layers_2.blocks.7.norm1.weight", "encoder.layers_2.blocks.7.norm1.bias",
"encoder.layers_2.blocks.7.attn.relative_position_bias_table", "encoder.layers_2.blocks.7.attn.qkv.weight", "encoder.layers_2.blocks.7.attn.qkv.bias", "encoder.layers_2.blocks.7.attn.proj.weight",
"encoder.layers_2.blocks.7.attn.proj.bias", "encoder.layers_2.blocks.7.norm2.weight", "encoder.layers_2.blocks.7.norm2.bias", "encoder.layers_2.blocks.7.mlp.fc1.weight",
"encoder.layers_2.blocks.7.mlp.fc1.bias", "encoder.layers_2.blocks.7.mlp.fc2.weight", "encoder.layers_2.blocks.7.mlp.fc2.bias", "encoder.layers_2.blocks.8.norm1.weight",
"encoder.layers_2.blocks.8.norm1.bias", "encoder.layers_2.blocks.8.attn.relative_position_bias_table", "encoder.layers_2.blocks.8.attn.qkv.weight", "encoder.layers_2.blocks.8.attn.qkv.bias",
"encoder.layers_2.blocks.8.attn.proj.weight", "encoder.layers_2.blocks.8.attn.proj.bias", "encoder.layers_2.blocks.8.norm2.weight", "encoder.layers_2.blocks.8.norm2.bias",
"encoder.layers_2.blocks.8.mlp.fc1.weight", "encoder.layers_2.blocks.8.mlp.fc1.bias", "encoder.layers_2.blocks.8.mlp.fc2.weight", "encoder.layers_2.blocks.8.mlp.fc2.bias",
"encoder.layers_2.blocks.9.norm1.weight", "encoder.layers_2.blocks.9.norm1.bias", "encoder.layers_2.blocks.9.attn.relative_position_bias_table", "encoder.layers_2.blocks.9.attn.qkv.weight",
"encoder.layers_2.blocks.9.attn.qkv.bias", "encoder.layers_2.blocks.9.attn.proj.weight", "encoder.layers_2.blocks.9.attn.proj.bias", "encoder.layers_2.blocks.9.norm2.weight",
"encoder.layers_2.blocks.9.norm2.bias", "encoder.layers_2.blocks.9.mlp.fc1.weight", "encoder.layers_2.blocks.9.mlp.fc1.bias", "encoder.layers_2.blocks.9.mlp.fc2.weight",
"encoder.layers_2.blocks.9.mlp.fc2.bias", "encoder.layers_2.blocks.10.norm1.weight", "encoder.layers_2.blocks.10.norm1.bias", "encoder.layers_2.blocks.10.attn.relative_position_bias_table",
"encoder.layers_2.blocks.10.attn.qkv.weight", "encoder.layers_2.blocks.10.attn.qkv.bias", "encoder.layers_2.blocks.10.attn.proj.weight", "encoder.layers_2.blocks.10.attn.proj.bias",
"encoder.layers_2.blocks.10.norm2.weight", "encoder.layers_2.blocks.10.norm2.bias", "encoder.layers_2.blocks.10.mlp.fc1.weight", "encoder.layers_2.blocks.10.mlp.fc1.bias",
"encoder.layers_2.blocks.10.mlp.fc2.weight", "encoder.layers_2.blocks.10.mlp.fc2.bias", "encoder.layers_2.blocks.11.norm1.weight", "encoder.layers_2.blocks.11.norm1.bias",
"encoder.layers_2.blocks.11.attn.relative_position_bias_table", "encoder.layers_2.blocks.11.attn.qkv.weight", "encoder.layers_2.blocks.11.attn.qkv.bias", "encoder.layers_2.blocks.11.attn.proj.weight",
"encoder.layers_2.blocks.11.attn.proj.bias", "encoder.layers_2.blocks.11.norm2.weight", "encoder.layers_2.blocks.11.norm2.bias", "encoder.layers_2.blocks.11.mlp.fc1.weight",
"encoder.layers_2.blocks.11.mlp.fc1.bias", "encoder.layers_2.blocks.11.mlp.fc2.weight", "encoder.layers_2.blocks.11.mlp.fc2.bias", "encoder.layers_2.blocks.12.norm1.weight",
"encoder.layers_2.blocks.12.norm1.bias", "encoder.layers_2.blocks.12.attn.relative_position_bias_table", "encoder.layers_2.blocks.12.attn.qkv.weight", "encoder.layers_2.blocks.12.attn.qkv.bias",
"encoder.layers_2.blocks.12.attn.proj.weight", "encoder.layers_2.blocks.12.attn.proj.bias", "encoder.layers_2.blocks.12.norm2.weight", "encoder.layers_2.blocks.12.norm2.bias",
"encoder.layers_2.blocks.12.mlp.fc1.weight", "encoder.layers_2.blocks.12.mlp.fc1.bias", "encoder.layers_2.blocks.12.mlp.fc2.weight", "encoder.layers_2.blocks.12.mlp.fc2.bias",
"encoder.layers_2.blocks.13.norm1.weight", "encoder.layers_2.blocks.13.norm1.bias", "encoder.layers_2.blocks.13.attn.relative_position_bias_table", "encoder.layers_2.blocks.13.attn.qkv.weight",
"encoder.layers_2.blocks.13.attn.qkv.bias", "encoder.layers_2.blocks.13.attn.proj.weight", "encoder.layers_2.blocks.13.attn.proj.bias", "encoder.layers_2.blocks.13.norm2.weight",
"encoder.layers_2.blocks.13.norm2.bias", "encoder.layers_2.blocks.13.mlp.fc1.weight", "encoder.layers_2.blocks.13.mlp.fc1.bias", "encoder.layers_2.blocks.13.mlp.fc2.weight",
"encoder.layers_2.blocks.13.mlp.fc2.bias", "encoder.layers_2.blocks.14.norm1.weight", "encoder.layers_2.blocks.14.norm1.bias", "encoder.layers_2.blocks.14.attn.relative_position_bias_table",
"encoder.layers_2.blocks.14.attn.qkv.weight", "encoder.layers_2.blocks.14.attn.qkv.bias", "encoder.layers_2.blocks.14.attn.proj.weight", "encoder.layers_2.blocks.14.attn.proj.bias",
"encoder.layers_2.blocks.14.norm2.weight", "encoder.layers_2.blocks.14.norm2.bias", "encoder.layers_2.blocks.14.mlp.fc1.weight", "encoder.layers_2.blocks.14.mlp.fc1.bias",
"encoder.layers_2.blocks.14.mlp.fc2.weight", "encoder.layers_2.blocks.14.mlp.fc2.bias", "encoder.layers_2.blocks.15.norm1.weight", "encoder.layers_2.blocks.15.norm1.bias",
"encoder.layers_2.blocks.15.attn.relative_position_bias_table", "encoder.layers_2.blocks.15.attn.qkv.weight", "encoder.layers_2.blocks.15.attn.qkv.bias", "encoder.layers_2.blocks.15.attn.proj.weight",
"encoder.layers_2.blocks.15.attn.proj.bias", "encoder.layers_2.blocks.15.norm2.weight", "encoder.layers_2.blocks.15.norm2.bias", "encoder.layers_2.blocks.15.mlp.fc1.weight",
"encoder.layers_2.blocks.15.mlp.fc1.bias", "encoder.layers_2.blocks.15.mlp.fc2.weight", "encoder.layers_2.blocks.15.mlp.fc2.bias", "encoder.layers_2.blocks.16.norm1.weight",
"encoder.layers_2.blocks.16.norm1.bias", "encoder.layers_2.blocks.16.attn.relative_position_bias_table", "encoder.layers_2.blocks.16.attn.qkv.weight", "encoder.layers_2.blocks.16.attn.qkv.bias",
"encoder.layers_2.blocks.16.attn.proj.weight", "encoder.layers_2.blocks.16.attn.proj.bias", "encoder.layers_2.blocks.16.norm2.weight", "encoder.layers_2.blocks.16.norm2.bias",
"encoder.layers_2.blocks.16.mlp.fc1.weight", "encoder.layers_2.blocks.16.mlp.fc1.bias", "encoder.layers_2.blocks.16.mlp.fc2.weight", "encoder.layers_2.blocks.16.mlp.fc2.bias",
"encoder.layers_2.blocks.17.norm1.weight", "encoder.layers_2.blocks.17.norm1.bias", "encoder.layers_2.blocks.17.attn.relative_position_bias_table", "encoder.layers_2.blocks.17.attn.qkv.weight",
"encoder.layers_2.blocks.17.attn.qkv.bias", "encoder.layers_2.blocks.17.attn.proj.weight", "encoder.layers_2.blocks.17.attn.proj.bias", "encoder.layers_2.blocks.17.norm2.weight",
"encoder.layers_2.blocks.17.norm2.bias", "encoder.layers_2.blocks.17.mlp.fc1.weight", "encoder.layers_2.blocks.17.mlp.fc1.bias", "encoder.layers_2.blocks.17.mlp.fc2.weight",
"encoder.layers_2.blocks.17.mlp.fc2.bias", "adapter.adapter.0.attn.q_proj.weight", "adapter.adapter.0.attn.k_proj.weight", "adapter.adapter.0.attn.v_proj.weight",
"adapter.adapter.0.attn.output_proj.weight", "adapter.adapter.0.mlp.w1.weight", "adapter.adapter.0.mlp.w1.bias", "adapter.adapter.0.mlp.w2.weight", "adapter.adapter.0.mlp.w2.bias",
"adapter.adapter.0.sa_norm.scale", "adapter.adapter.0.mlp_norm.scale", "adapter.adapter.0.sa_scale.scale", "adapter.adapter.0.mlp_scale.scale", "adapter.adapter.1.attn.q_proj.weight",
"adapter.adapter.1.attn.k_proj.weight", "adapter.adapter.1.attn.v_proj.weight", "adapter.adapter.1.attn.output_proj.weight", "adapter.adapter.1.mlp.w1.weight", "adapter.adapter.1.mlp.w1.bias",
"adapter.adapter.1.mlp.w2.weight", "adapter.adapter.1.mlp.w2.bias", "adapter.adapter.1.sa_norm.scale", "adapter.adapter.1.mlp_norm.scale", "adapter.adapter.1.sa_scale.scale",
"adapter.adapter.1.mlp_scale.scale", "adapter.adapter.2.attn.q_proj.weight", "adapter.adapter.2.attn.k_proj.weight", "adapter.adapter.2.attn.v_proj.weight", "adapter.adapter.2.attn.output_proj.weight",
"adapter.adapter.2.mlp.w1.weight", "adapter.adapter.2.mlp.w1.bias", "adapter.adapter.2.mlp.w2.weight", "adapter.adapter.2.mlp.w2.bias", "adapter.adapter.2.sa_norm.scale", "adapter.adapter.2.mlp_norm.scale",
"adapter.adapter.2.sa_scale.scale", "adapter.adapter.2.mlp_scale.scale", "adapter.adapter.3.attn.q_proj.weight", "adapter.adapter.3.attn.k_proj.weight", "adapter.adapter.3.attn.v_proj.weight",
"adapter.adapter.3.attn.output_proj.weight", "adapter.adapter.3.mlp.w1.weight", "adapter.adapter.3.mlp.w1.bias", "adapter.adapter.3.mlp.w2.weight", "adapter.adapter.3.mlp.w2.bias",
"adapter.adapter.3.sa_norm.scale", "adapter.adapter.3.mlp_norm.scale", "adapter.adapter.3.sa_scale.scale", "adapter.adapter.3.mlp_scale.scale", "adapter.adapter.4.weight", "adapter.adapter.4.bias".
        Unexpected key(s) in state_dict: "encoder.layers.0.blocks.0.attn.proj.bias", "encoder.layers.0.blocks.0.attn.proj.weight", "encoder.layers.0.blocks.0.attn.qkv.bias",
"encoder.layers.0.blocks.0.attn.qkv.weight", "encoder.layers.0.blocks.0.attn.relative_position_bias_table", "encoder.layers.0.blocks.0.mlp.fc1.bias", "encoder.layers.0.blocks.0.mlp.fc1.weight",
"encoder.layers.0.blocks.0.mlp.fc2.bias", "encoder.layers.0.blocks.0.mlp.fc2.weight", "encoder.layers.0.blocks.0.norm1.bias", "encoder.layers.0.blocks.0.norm1.weight",
"encoder.layers.0.blocks.0.norm2.bias", "encoder.layers.0.blocks.0.norm2.weight", "encoder.layers.0.blocks.1.attn.proj.bias", "encoder.layers.0.blocks.1.attn.proj.weight",
"encoder.layers.0.blocks.1.attn.qkv.bias", "encoder.layers.0.blocks.1.attn.qkv.weight", "encoder.layers.0.blocks.1.attn.relative_position_bias_table", "encoder.layers.0.blocks.1.mlp.fc1.bias",
"encoder.layers.0.blocks.1.mlp.fc1.weight", "encoder.layers.0.blocks.1.mlp.fc2.bias", "encoder.layers.0.blocks.1.mlp.fc2.weight", "encoder.layers.0.blocks.1.norm1.bias",
"encoder.layers.0.blocks.1.norm1.weight", "encoder.layers.0.blocks.1.norm2.bias", "encoder.layers.0.blocks.1.norm2.weight", "encoder.layers.1.blocks.0.attn.proj.bias",
"encoder.layers.1.blocks.0.attn.proj.weight", "encoder.layers.1.blocks.0.attn.qkv.bias", "encoder.layers.1.blocks.0.attn.qkv.weight", "encoder.layers.1.blocks.0.attn.relative_position_bias_table",
"encoder.layers.1.blocks.0.mlp.fc1.bias", "encoder.layers.1.blocks.0.mlp.fc1.weight", "encoder.layers.1.blocks.0.mlp.fc2.bias", "encoder.layers.1.blocks.0.mlp.fc2.weight",
"encoder.layers.1.blocks.0.norm1.bias", "encoder.layers.1.blocks.0.norm1.weight", "encoder.layers.1.blocks.0.norm2.bias", "encoder.layers.1.blocks.0.norm2.weight",
"encoder.layers.1.blocks.1.attn.proj.bias", "encoder.layers.1.blocks.1.attn.proj.weight", "encoder.layers.1.blocks.1.attn.qkv.bias", "encoder.layers.1.blocks.1.attn.qkv.weight",
"encoder.layers.1.blocks.1.attn.relative_position_bias_table", "encoder.layers.1.blocks.1.mlp.fc1.bias", "encoder.layers.1.blocks.1.mlp.fc1.weight", "encoder.layers.1.blocks.1.mlp.fc2.bias",
"encoder.layers.1.blocks.1.mlp.fc2.weight", "encoder.layers.1.blocks.1.norm1.bias", "encoder.layers.1.blocks.1.norm1.weight", "encoder.layers.1.blocks.1.norm2.bias",
"encoder.layers.1.blocks.1.norm2.weight", "encoder.layers.1.downsample.norm.bias", "encoder.layers.1.downsample.norm.weight", "encoder.layers.1.downsample.reduction.weight",
"encoder.layers.2.blocks.0.attn.proj.bias", "encoder.layers.2.blocks.0.attn.proj.weight", "encoder.layers.2.blocks.0.attn.qkv.bias", "encoder.layers.2.blocks.0.attn.qkv.weight",
"encoder.layers.2.blocks.0.attn.relative_position_bias_table", "encoder.layers.2.blocks.0.mlp.fc1.bias", "encoder.layers.2.blocks.0.mlp.fc1.weight", "encoder.layers.2.blocks.0.mlp.fc2.bias",
"encoder.layers.2.blocks.0.mlp.fc2.weight", "encoder.layers.2.blocks.0.norm1.bias", "encoder.layers.2.blocks.0.norm1.weight", "encoder.layers.2.blocks.0.norm2.bias",
"encoder.layers.2.blocks.0.norm2.weight", "encoder.layers.2.blocks.1.attn.proj.bias", "encoder.layers.2.blocks.1.attn.proj.weight", "encoder.layers.2.blocks.1.attn.qkv.bias",
"encoder.layers.2.blocks.1.attn.qkv.weight", "encoder.layers.2.blocks.1.attn.relative_position_bias_table", "encoder.layers.2.blocks.1.mlp.fc1.bias", "encoder.layers.2.blocks.1.mlp.fc1.weight",
"encoder.layers.2.blocks.1.mlp.fc2.bias", "encoder.layers.2.blocks.1.mlp.fc2.weight", "encoder.layers.2.blocks.1.norm1.bias", "encoder.layers.2.blocks.1.norm1.weight",
"encoder.layers.2.blocks.1.norm2.bias", "encoder.layers.2.blocks.1.norm2.weight", "encoder.layers.2.blocks.10.attn.proj.bias", "encoder.layers.2.blocks.10.attn.proj.weight",
"encoder.layers.2.blocks.10.attn.qkv.bias", "encoder.layers.2.blocks.10.attn.qkv.weight", "encoder.layers.2.blocks.10.attn.relative_position_bias_table", "encoder.layers.2.blocks.10.mlp.fc1.bias",
"encoder.layers.2.blocks.10.mlp.fc1.weight", "encoder.layers.2.blocks.10.mlp.fc2.bias", "encoder.layers.2.blocks.10.mlp.fc2.weight", "encoder.layers.2.blocks.10.norm1.bias",
"encoder.layers.2.blocks.10.norm1.weight", "encoder.layers.2.blocks.10.norm2.bias", "encoder.layers.2.blocks.10.norm2.weight", "encoder.layers.2.blocks.11.attn.proj.bias",
"encoder.layers.2.blocks.11.attn.proj.weight", "encoder.layers.2.blocks.11.attn.qkv.bias", "encoder.layers.2.blocks.11.attn.qkv.weight", "encoder.layers.2.blocks.11.attn.relative_position_bias_table",
"encoder.layers.2.blocks.11.mlp.fc1.bias", "encoder.layers.2.blocks.11.mlp.fc1.weight", "encoder.layers.2.blocks.11.mlp.fc2.bias", "encoder.layers.2.blocks.11.mlp.fc2.weight",
"encoder.layers.2.blocks.11.norm1.bias", "encoder.layers.2.blocks.11.norm1.weight", "encoder.layers.2.blocks.11.norm2.bias", "encoder.layers.2.blocks.11.norm2.weight",
"encoder.layers.2.blocks.12.attn.proj.bias", "encoder.layers.2.blocks.12.attn.proj.weight", "encoder.layers.2.blocks.12.attn.qkv.bias", "encoder.layers.2.blocks.12.attn.qkv.weight",
"encoder.layers.2.blocks.12.attn.relative_position_bias_table", "encoder.layers.2.blocks.12.mlp.fc1.bias", "encoder.layers.2.blocks.12.mlp.fc1.weight", "encoder.layers.2.blocks.12.mlp.fc2.bias",
"encoder.layers.2.blocks.12.mlp.fc2.weight", "encoder.layers.2.blocks.12.norm1.bias", "encoder.layers.2.blocks.12.norm1.weight", "encoder.layers.2.blocks.12.norm2.bias",
"encoder.layers.2.blocks.12.norm2.weight", "encoder.layers.2.blocks.13.attn.proj.bias", "encoder.layers.2.blocks.13.attn.proj.weight", "encoder.layers.2.blocks.13.attn.qkv.bias",
"encoder.layers.2.blocks.13.attn.qkv.weight", "encoder.layers.2.blocks.13.attn.relative_position_bias_table", "encoder.layers.2.blocks.13.mlp.fc1.bias", "encoder.layers.2.blocks.13.mlp.fc1.weight",
"encoder.layers.2.blocks.13.mlp.fc2.bias", "encoder.layers.2.blocks.13.mlp.fc2.weight", "encoder.layers.2.blocks.13.norm1.bias", "encoder.layers.2.blocks.13.norm1.weight",
"encoder.layers.2.blocks.13.norm2.bias", "encoder.layers.2.blocks.13.norm2.weight", "encoder.layers.2.blocks.14.attn.proj.bias", "encoder.layers.2.blocks.14.attn.proj.weight",
"encoder.layers.2.blocks.14.attn.qkv.bias", "encoder.layers.2.blocks.14.attn.qkv.weight", "encoder.layers.2.blocks.14.attn.relative_position_bias_table", "encoder.layers.2.blocks.14.mlp.fc1.bias",
"encoder.layers.2.blocks.14.mlp.fc1.weight", "encoder.layers.2.blocks.14.mlp.fc2.bias", "encoder.layers.2.blocks.14.mlp.fc2.weight", "encoder.layers.2.blocks.14.norm1.bias",
"encoder.layers.2.blocks.14.norm1.weight", "encoder.layers.2.blocks.14.norm2.bias", "encoder.layers.2.blocks.14.norm2.weight", "encoder.layers.2.blocks.15.attn.proj.bias",
"encoder.layers.2.blocks.15.attn.proj.weight", "encoder.layers.2.blocks.15.attn.qkv.bias", "encoder.layers.2.blocks.15.attn.qkv.weight", "encoder.layers.2.blocks.15.attn.relative_position_bias_table",
"encoder.layers.2.blocks.15.mlp.fc1.bias", "encoder.layers.2.blocks.15.mlp.fc1.weight", "encoder.layers.2.blocks.15.mlp.fc2.bias", "encoder.layers.2.blocks.15.mlp.fc2.weight",
"encoder.layers.2.blocks.15.norm1.bias", "encoder.layers.2.blocks.15.norm1.weight", "encoder.layers.2.blocks.15.norm2.bias", "encoder.layers.2.blocks.15.norm2.weight",
"encoder.layers.2.blocks.16.attn.proj.bias", "encoder.layers.2.blocks.16.attn.proj.weight", "encoder.layers.2.blocks.16.attn.qkv.bias", "encoder.layers.2.blocks.16.attn.qkv.weight",
"encoder.layers.2.blocks.16.attn.relative_position_bias_table", "encoder.layers.2.blocks.16.mlp.fc1.bias", "encoder.layers.2.blocks.16.mlp.fc1.weight", "encoder.layers.2.blocks.16.mlp.fc2.bias",
"encoder.layers.2.blocks.16.mlp.fc2.weight", "encoder.layers.2.blocks.16.norm1.bias", "encoder.layers.2.blocks.16.norm1.weight", "encoder.layers.2.blocks.16.norm2.bias",
"encoder.layers.2.blocks.16.norm2.weight", "encoder.layers.2.blocks.17.attn.proj.bias", "encoder.layers.2.blocks.17.attn.proj.weight", "encoder.layers.2.blocks.17.attn.qkv.bias",
"encoder.layers.2.blocks.17.attn.qkv.weight", "encoder.layers.2.blocks.17.attn.relative_position_bias_table", "encoder.layers.2.blocks.17.mlp.fc1.bias", "encoder.layers.2.blocks.17.mlp.fc1.weight",
"encoder.layers.2.blocks.17.mlp.fc2.bias", "encoder.layers.2.blocks.17.mlp.fc2.weight", "encoder.layers.2.blocks.17.norm1.bias", "encoder.layers.2.blocks.17.norm1.weight",
"encoder.layers.2.blocks.17.norm2.bias", "encoder.layers.2.blocks.17.norm2.weight", "encoder.layers.2.blocks.2.attn.proj.bias", "encoder.layers.2.blocks.2.attn.proj.weight",
"encoder.layers.2.blocks.2.attn.qkv.bias", "encoder.layers.2.blocks.2.attn.qkv.weight", "encoder.layers.2.blocks.2.attn.relative_position_bias_table", "encoder.layers.2.blocks.2.mlp.fc1.bias",
"encoder.layers.2.blocks.2.mlp.fc1.weight", "encoder.layers.2.blocks.2.mlp.fc2.bias", "encoder.layers.2.blocks.2.mlp.fc2.weight", "encoder.layers.2.blocks.2.norm1.bias",
"encoder.layers.2.blocks.2.norm1.weight", "encoder.layers.2.blocks.2.norm2.bias", "encoder.layers.2.blocks.2.norm2.weight", "encoder.layers.2.blocks.3.attn.proj.bias",
"encoder.layers.2.blocks.3.attn.proj.weight", "encoder.layers.2.blocks.3.attn.qkv.bias", "encoder.layers.2.blocks.3.attn.qkv.weight", "encoder.layers.2.blocks.3.attn.relative_position_bias_table",
"encoder.layers.2.blocks.3.mlp.fc1.bias", "encoder.layers.2.blocks.3.mlp.fc1.weight", "encoder.layers.2.blocks.3.mlp.fc2.bias", "encoder.layers.2.blocks.3.mlp.fc2.weight",
"encoder.layers.2.blocks.3.norm1.bias", "encoder.layers.2.blocks.3.norm1.weight", "encoder.layers.2.blocks.3.norm2.bias", "encoder.layers.2.blocks.3.norm2.weight",
"encoder.layers.2.blocks.4.attn.proj.bias", "encoder.layers.2.blocks.4.attn.proj.weight", "encoder.layers.2.blocks.4.attn.qkv.bias", "encoder.layers.2.blocks.4.attn.qkv.weight",
"encoder.layers.2.blocks.4.attn.relative_position_bias_table", "encoder.layers.2.blocks.4.mlp.fc1.bias", "encoder.layers.2.blocks.4.mlp.fc1.weight", "encoder.layers.2.blocks.4.mlp.fc2.bias",
"encoder.layers.2.blocks.4.mlp.fc2.weight", "encoder.layers.2.blocks.4.norm1.bias", "encoder.layers.2.blocks.4.norm1.weight", "encoder.layers.2.blocks.4.norm2.bias",
"encoder.layers.2.blocks.4.norm2.weight", "encoder.layers.2.blocks.5.attn.proj.bias", "encoder.layers.2.blocks.5.attn.proj.weight", "encoder.layers.2.blocks.5.attn.qkv.bias",
"encoder.layers.2.blocks.5.attn.qkv.weight", "encoder.layers.2.blocks.5.attn.relative_position_bias_table", "encoder.layers.2.blocks.5.mlp.fc1.bias", "encoder.layers.2.blocks.5.mlp.fc1.weight",
"encoder.layers.2.blocks.5.mlp.fc2.bias", "encoder.layers.2.blocks.5.mlp.fc2.weight", "encoder.layers.2.blocks.5.norm1.bias", "encoder.layers.2.blocks.5.norm1.weight",
"encoder.layers.2.blocks.5.norm2.bias", "encoder.layers.2.blocks.5.norm2.weight", "encoder.layers.2.blocks.6.attn.proj.bias", "encoder.layers.2.blocks.6.attn.proj.weight",
"encoder.layers.2.blocks.6.attn.qkv.bias", "encoder.layers.2.blocks.6.attn.qkv.weight", "encoder.layers.2.blocks.6.attn.relative_position_bias_table", "encoder.layers.2.blocks.6.mlp.fc1.bias",
"encoder.layers.2.blocks.6.mlp.fc1.weight", "encoder.layers.2.blocks.6.mlp.fc2.bias", "encoder.layers.2.blocks.6.mlp.fc2.weight", "encoder.layers.2.blocks.6.norm1.bias",
"encoder.layers.2.blocks.6.norm1.weight", "encoder.layers.2.blocks.6.norm2.bias", "encoder.layers.2.blocks.6.norm2.weight", "encoder.layers.2.blocks.7.attn.proj.bias",
"encoder.layers.2.blocks.7.attn.proj.weight", "encoder.layers.2.blocks.7.attn.qkv.bias", "encoder.layers.2.blocks.7.attn.qkv.weight", "encoder.layers.2.blocks.7.attn.relative_position_bias_table",
"encoder.layers.2.blocks.7.mlp.fc1.bias", "encoder.layers.2.blocks.7.mlp.fc1.weight", "encoder.layers.2.blocks.7.mlp.fc2.bias", "encoder.layers.2.blocks.7.mlp.fc2.weight",
"encoder.layers.2.blocks.7.norm1.bias", "encoder.layers.2.blocks.7.norm1.weight", "encoder.layers.2.blocks.7.norm2.bias", "encoder.layers.2.blocks.7.norm2.weight",
"encoder.layers.2.blocks.8.attn.proj.bias", "encoder.layers.2.blocks.8.attn.proj.weight", "encoder.layers.2.blocks.8.attn.qkv.bias", "encoder.layers.2.blocks.8.attn.qkv.weight",
"encoder.layers.2.blocks.8.attn.relative_position_bias_table", "encoder.layers.2.blocks.8.mlp.fc1.bias", "encoder.layers.2.blocks.8.mlp.fc1.weight", "encoder.layers.2.blocks.8.mlp.fc2.bias",
"encoder.layers.2.blocks.8.mlp.fc2.weight", "encoder.layers.2.blocks.8.norm1.bias", "encoder.layers.2.blocks.8.norm1.weight", "encoder.layers.2.blocks.8.norm2.bias",
"encoder.layers.2.blocks.8.norm2.weight", "encoder.layers.2.blocks.9.attn.proj.bias", "encoder.layers.2.blocks.9.attn.proj.weight", "encoder.layers.2.blocks.9.attn.qkv.bias",
"encoder.layers.2.blocks.9.attn.qkv.weight", "encoder.layers.2.blocks.9.attn.relative_position_bias_table", "encoder.layers.2.blocks.9.mlp.fc1.bias", "encoder.layers.2.blocks.9.mlp.fc1.weight",
"encoder.layers.2.blocks.9.mlp.fc2.bias", "encoder.layers.2.blocks.9.mlp.fc2.weight", "encoder.layers.2.blocks.9.norm1.bias", "encoder.layers.2.blocks.9.norm1.weight",
"encoder.layers.2.blocks.9.norm2.bias", "encoder.layers.2.blocks.9.norm2.weight", "encoder.layers.2.downsample.norm.bias", "encoder.layers.2.downsample.norm.weight",
"encoder.layers.2.downsample.reduction.weight", "adapter.0.attn.k_proj.weight", "adapter.0.attn.output_proj.weight", "adapter.0.attn.q_proj.weight", "adapter.0.attn.v_proj.weight", "adapter.0.mlp.w1.bias",
"adapter.0.mlp.w1.weight", "adapter.0.mlp.w2.bias", "adapter.0.mlp.w2.weight", "adapter.0.mlp_norm.scale", "adapter.0.mlp_scale.scale", "adapter.0.sa_norm.scale", "adapter.0.sa_scale.scale",
"adapter.1.attn.k_proj.weight", "adapter.1.attn.output_proj.weight", "adapter.1.attn.q_proj.weight", "adapter.1.attn.v_proj.weight", "adapter.1.mlp.w1.bias", "adapter.1.mlp.w1.weight",
"adapter.1.mlp.w2.bias", "adapter.1.mlp.w2.weight", "adapter.1.mlp_norm.scale", "adapter.1.mlp_scale.scale", "adapter.1.sa_norm.scale", "adapter.1.sa_scale.scale", "adapter.2.attn.k_proj.weight",
"adapter.2.attn.output_proj.weight", "adapter.2.attn.q_proj.weight", "adapter.2.attn.v_proj.weight", "adapter.2.mlp.w1.bias", "adapter.2.mlp.w1.weight", "adapter.2.mlp.w2.bias", "adapter.2.mlp.w2.weight",
"adapter.2.mlp_norm.scale", "adapter.2.mlp_scale.scale", "adapter.2.sa_norm.scale", "adapter.2.sa_scale.scale", "adapter.3.attn.k_proj.weight", "adapter.3.attn.output_proj.weight",
"adapter.3.attn.q_proj.weight", "adapter.3.attn.v_proj.weight", "adapter.3.mlp.w1.bias", "adapter.3.mlp.w1.weight", "adapter.3.mlp.w2.bias", "adapter.3.mlp.w2.weight", "adapter.3.mlp_norm.scale",
"adapter.3.mlp_scale.scale", "adapter.3.sa_norm.scale", "adapter.3.sa_scale.scale", "adapter.4.bias", "adapter.4.weight".

[mittagessen]

I've been training a new base model using a convnextv2 visual encoder + a smaller 40m parameter language decoder that offers the same accuracy as the old base model with significantly less computational needs. There's also a mostly finished kraken integration using the new plugin system which requires a slightly different weights format as well. I'll push the checkpoint in the next couple of days and will finish the integration sometime in early December which basically makes everything release-ready. In the meanwhile, if you want to use the existing checkpoint, commit 271d6a7 should do the trick.

…

On 25/11/24 05:14AM, Weslley Oliveira wrote: jesusbft created an issue (mittagessen/party#21) Hey, I testing the train party command and have a couple issues. 1 - the encoder in the party code don't match at all with the new party_european_langs.safetensors 2- I change the file model.py to represent the new encoder: ``` class RecognitionModel(L.LightningModule): def __init__(self, (...) encoder: str = 'swin_base_patch4_window12_384.ms_in22k', (...) super().__init__() ``` 3- However, I still have errors to train using the party_european_langs.safetensors: The problem: Model expects: encoder.layers_0.blocks.0... (underscore) Checkpoint has: encoder.layers.0.blocks.0... (dot) And for the adapter: Model expects: adapter.adapter.0... Checkpoint has: adapter.0... I can't find how to fix this issue, maybe you know ``` party -d cuda:0 --workers 4 train -t dataset-party/train.lst -e dataset-party/val.lst -o models/Portuguese --load dataset-party/party_european_langs.safetensors Using bfloat16 Automatic Mixed Precision (AMP) GPU available: True (cuda), used: True TPU available: False, using: 0 TPU cores HPU available: False, using: 0 HPUs `Trainer(val_check_interval=1.0)` was configured so validation will run at the end of the training epoch.. Loading from checkpoint dataset-party/party_european_langs.safetensors. ╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮ │ /home/jesus/venv/bin/party:8 in <module> │ │ │ │ 5 from party.cli import cli │ │ 6 if __name__ == '__main__': │ │ 7 │ sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0]) │ │ ❱ 8 │ sys.exit(cli()) │ │ 9 │ │ │ │ /home/jesus/venv/lib/python3.12/site-packages/click/core.py:1161 in __call__ │ │ │ │ /home/jesus/venv/lib/python3.12/site-packages/click/core.py:1082 in main │ │ │ │ /home/jesus/venv/lib/python3.12/site-packages/click/core.py:1697 in invoke │ │ │ │ /home/jesus/venv/lib/python3.12/site-packages/click/core.py:1443 in invoke │ │ │ │ /home/jesus/venv/lib/python3.12/site-packages/click/core.py:788 in invoke │ │ │ │ /home/jesus/venv/lib/python3.12/site-packages/click/decorators.py:33 in new_func │ │ │ │ /home/jesus/venv/party/party/cli/train.py:298 in train │ │ │ │ 295 │ │ elif load: │ │ 296 │ │ │ message(f'Loading from checkpoint {load}.') │ │ 297 │ │ │ if load.endswith('safetensors'): │ │ ❱ 298 │ │ │ │ model = RecognitionModel.load_from_safetensors(load, **params) │ │ 299 │ │ │ elif load.endswith('ckpt'): │ │ 300 │ │ │ │ model = RecognitionModel.load_from_checkpoint(load, **params) │ │ 301 │ │ │ else: │ │ │ │ /home/jesus/venv/party/party/model.py:204 in load_from_safetensors │ │ │ │ 201 │ │ from safetensors.torch import load_file │ │ 202 │ │ │ │ 203 │ │ module = cls(*args, **kwargs, pretrained=False) │ │ ❱ 204 │ │ module.model.load_state_dict(load_file(file)) │ │ 205 │ │ module.model.train() │ │ 206 │ │ return module │ │ 207 │ │ │ │ /home/jesus/venv/lib/python3.12/site-packages/torch/nn/modules/module.py:2629 in load_state_dict │ │ │ │ 2626 │ │ │ │ ) │ │ 2627 │ │ │ │ 2628 │ │ if len(error_msgs) > 0: │ │ ❱ 2629 │ │ │ raise RuntimeError( │ │ 2630 │ │ │ │ "Error(s) in loading state_dict for {}:\n\t{}".format( │ │ 2631 │ │ │ │ │ self.__class__.__name__, "\n\t".join(error_msgs) │ │ 2632 │ │ │ │ ) │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ RuntimeError: Error(s) in loading state_dict for PartyModel: Missing key(s) in state_dict: "encoder.layers_0.blocks.0.norm1.weight", "encoder.layers_0.blocks.0.norm1.bias", "encoder.layers_0.blocks.0.attn.relative_position_bias_table", "encoder.layers_0.blocks.0.attn.qkv.weight", "encoder.layers_0.blocks.0.attn.qkv.bias", "encoder.layers_0.blocks.0.attn.proj.weight", "encoder.layers_0.blocks.0.attn.proj.bias", "encoder.layers_0.blocks.0.norm2.weight", "encoder.layers_0.blocks.0.norm2.bias", "encoder.layers_0.blocks.0.mlp.fc1.weight", "encoder.layers_0.blocks.0.mlp.fc1.bias", "encoder.layers_0.blocks.0.mlp.fc2.weight", "encoder.layers_0.blocks.0.mlp.fc2.bias", "encoder.layers_0.blocks.1.norm1.weight", "encoder.layers_0.blocks.1.norm1.bias", "encoder.layers_0.blocks.1.attn.relative_position_bias_table", "encoder.layers_0.blocks.1.attn.qkv.weight", "encoder.layers_0.blocks.1.attn.qkv.bias", "encoder.layers_0.blocks.1.attn.proj.weight", "encoder.layers_0.blocks.1.attn.proj.bias", "encoder.layers_0.blocks.1.norm2.weight", "encoder.layers_0.blocks.1.norm2.bias", "encoder.layers_0.blocks.1.mlp.fc1.weight", "encoder.layers_0.blocks.1.mlp.fc1.bias", "encoder.layers_0.blocks.1.mlp.fc2.weight", "encoder.layers_0.blocks.1.mlp.fc2.bias", "encoder.layers_1.downsample.norm.weight", "encoder.layers_1.downsample.norm.bias", "encoder.layers_1.downsample.reduction.weight", "encoder.layers_1.blocks.0.norm1.weight", "encoder.layers_1.blocks.0.norm1.bias", "encoder.layers_1.blocks.0.attn.relative_position_bias_table", "encoder.layers_1.blocks.0.attn.qkv.weight", "encoder.layers_1.blocks.0.attn.qkv.bias", "encoder.layers_1.blocks.0.attn.proj.weight", "encoder.layers_1.blocks.0.attn.proj.bias", "encoder.layers_1.blocks.0.norm2.weight", "encoder.layers_1.blocks.0.norm2.bias", "encoder.layers_1.blocks.0.mlp.fc1.weight", "encoder.layers_1.blocks.0.mlp.fc1.bias", "encoder.layers_1.blocks.0.mlp.fc2.weight", "encoder.layers_1.blocks.0.mlp.fc2.bias", "encoder.layers_1.blocks.1.norm1.weight", "encoder.layers_1.blocks.1.norm1.bias", "encoder.layers_1.blocks.1.attn.relative_position_bias_table", "encoder.layers_1.blocks.1.attn.qkv.weight", "encoder.layers_1.blocks.1.attn.qkv.bias", "encoder.layers_1.blocks.1.attn.proj.weight", "encoder.layers_1.blocks.1.attn.proj.bias", "encoder.layers_1.blocks.1.norm2.weight", "encoder.layers_1.blocks.1.norm2.bias", "encoder.layers_1.blocks.1.mlp.fc1.weight", "encoder.layers_1.blocks.1.mlp.fc1.bias", "encoder.layers_1.blocks.1.mlp.fc2.weight", "encoder.layers_1.blocks.1.mlp.fc2.bias", "encoder.layers_2.downsample.norm.weight", "encoder.layers_2.downsample.norm.bias", "encoder.layers_2.downsample.reduction.weight", "encoder.layers_2.blocks.0.norm1.weight", "encoder.layers_2.blocks.0.norm1.bias", "encoder.layers_2.blocks.0.attn.relative_position_bias_table", "encoder.layers_2.blocks.0.attn.qkv.weight", "encoder.layers_2.blocks.0.attn.qkv.bias", "encoder.layers_2.blocks.0.attn.proj.weight", "encoder.layers_2.blocks.0.attn.proj.bias", "encoder.layers_2.blocks.0.norm2.weight", "encoder.layers_2.blocks.0.norm2.bias", "encoder.layers_2.blocks.0.mlp.fc1.weight", "encoder.layers_2.blocks.0.mlp.fc1.bias", "encoder.layers_2.blocks.0.mlp.fc2.weight", "encoder.layers_2.blocks.0.mlp.fc2.bias", "encoder.layers_2.blocks.1.norm1.weight", "encoder.layers_2.blocks.1.norm1.bias", "encoder.layers_2.blocks.1.attn.relative_position_bias_table", "encoder.layers_2.blocks.1.attn.qkv.weight", "encoder.layers_2.blocks.1.attn.qkv.bias", "encoder.layers_2.blocks.1.attn.proj.weight", "encoder.layers_2.blocks.1.attn.proj.bias", "encoder.layers_2.blocks.1.norm2.weight", "encoder.layers_2.blocks.1.norm2.bias", "encoder.layers_2.blocks.1.mlp.fc1.weight", "encoder.layers_2.blocks.1.mlp.fc1.bias", "encoder.layers_2.blocks.1.mlp.fc2.weight", "encoder.layers_2.blocks.1.mlp.fc2.bias", "encoder.layers_2.blocks.2.norm1.weight", "encoder.layers_2.blocks.2.norm1.bias", "encoder.layers_2.blocks.2.attn.relative_position_bias_table", "encoder.layers_2.blocks.2.attn.qkv.weight", "encoder.layers_2.blocks.2.attn.qkv.bias", "encoder.layers_2.blocks.2.attn.proj.weight", "encoder.layers_2.blocks.2.attn.proj.bias", "encoder.layers_2.blocks.2.norm2.weight", "encoder.layers_2.blocks.2.norm2.bias", "encoder.layers_2.blocks.2.mlp.fc1.weight", "encoder.layers_2.blocks.2.mlp.fc1.bias", "encoder.layers_2.blocks.2.mlp.fc2.weight", "encoder.layers_2.blocks.2.mlp.fc2.bias", "encoder.layers_2.blocks.3.norm1.weight", "encoder.layers_2.blocks.3.norm1.bias", "encoder.layers_2.blocks.3.attn.relative_position_bias_table", "encoder.layers_2.blocks.3.attn.qkv.weight", "encoder.layers_2.blocks.3.attn.qkv.bias", "encoder.layers_2.blocks.3.attn.proj.weight", "encoder.layers_2.blocks.3.attn.proj.bias", "encoder.layers_2.blocks.3.norm2.weight", "encoder.layers_2.blocks.3.norm2.bias", "encoder.layers_2.blocks.3.mlp.fc1.weight", "encoder.layers_2.blocks.3.mlp.fc1.bias", "encoder.layers_2.blocks.3.mlp.fc2.weight", "encoder.layers_2.blocks.3.mlp.fc2.bias", "encoder.layers_2.blocks.4.norm1.weight", "encoder.layers_2.blocks.4.norm1.bias", "encoder.layers_2.blocks.4.attn.relative_position_bias_table", "encoder.layers_2.blocks.4.attn.qkv.weight", "encoder.layers_2.blocks.4.attn.qkv.bias", "encoder.layers_2.blocks.4.attn.proj.weight", "encoder.layers_2.blocks.4.attn.proj.bias", "encoder.layers_2.blocks.4.norm2.weight", "encoder.layers_2.blocks.4.norm2.bias", "encoder.layers_2.blocks.4.mlp.fc1.weight", "encoder.layers_2.blocks.4.mlp.fc1.bias", "encoder.layers_2.blocks.4.mlp.fc2.weight", "encoder.layers_2.blocks.4.mlp.fc2.bias", "encoder.layers_2.blocks.5.norm1.weight", "encoder.layers_2.blocks.5.norm1.bias", "encoder.layers_2.blocks.5.attn.relative_position_bias_table", "encoder.layers_2.blocks.5.attn.qkv.weight", "encoder.layers_2.blocks.5.attn.qkv.bias", "encoder.layers_2.blocks.5.attn.proj.weight", "encoder.layers_2.blocks.5.attn.proj.bias", "encoder.layers_2.blocks.5.norm2.weight", "encoder.layers_2.blocks.5.norm2.bias", "encoder.layers_2.blocks.5.mlp.fc1.weight", "encoder.layers_2.blocks.5.mlp.fc1.bias", "encoder.layers_2.blocks.5.mlp.fc2.weight", "encoder.layers_2.blocks.5.mlp.fc2.bias", "encoder.layers_2.blocks.6.norm1.weight", "encoder.layers_2.blocks.6.norm1.bias", "encoder.layers_2.blocks.6.attn.relative_position_bias_table", "encoder.layers_2.blocks.6.attn.qkv.weight", "encoder.layers_2.blocks.6.attn.qkv.bias", "encoder.layers_2.blocks.6.attn.proj.weight", "encoder.layers_2.blocks.6.attn.proj.bias", "encoder.layers_2.blocks.6.norm2.weight", "encoder.layers_2.blocks.6.norm2.bias", "encoder.layers_2.blocks.6.mlp.fc1.weight", "encoder.layers_2.blocks.6.mlp.fc1.bias", "encoder.layers_2.blocks.6.mlp.fc2.weight", "encoder.layers_2.blocks.6.mlp.fc2.bias", "encoder.layers_2.blocks.7.norm1.weight", "encoder.layers_2.blocks.7.norm1.bias", "encoder.layers_2.blocks.7.attn.relative_position_bias_table", "encoder.layers_2.blocks.7.attn.qkv.weight", "encoder.layers_2.blocks.7.attn.qkv.bias", "encoder.layers_2.blocks.7.attn.proj.weight", "encoder.layers_2.blocks.7.attn.proj.bias", "encoder.layers_2.blocks.7.norm2.weight", "encoder.layers_2.blocks.7.norm2.bias", "encoder.layers_2.blocks.7.mlp.fc1.weight", "encoder.layers_2.blocks.7.mlp.fc1.bias", "encoder.layers_2.blocks.7.mlp.fc2.weight", "encoder.layers_2.blocks.7.mlp.fc2.bias", "encoder.layers_2.blocks.8.norm1.weight", "encoder.layers_2.blocks.8.norm1.bias", "encoder.layers_2.blocks.8.attn.relative_position_bias_table", "encoder.layers_2.blocks.8.attn.qkv.weight", "encoder.layers_2.blocks.8.attn.qkv.bias", "encoder.layers_2.blocks.8.attn.proj.weight", "encoder.layers_2.blocks.8.attn.proj.bias", "encoder.layers_2.blocks.8.norm2.weight", "encoder.layers_2.blocks.8.norm2.bias", "encoder.layers_2.blocks.8.mlp.fc1.weight", "encoder.layers_2.blocks.8.mlp.fc1.bias", "encoder.layers_2.blocks.8.mlp.fc2.weight", "encoder.layers_2.blocks.8.mlp.fc2.bias", "encoder.layers_2.blocks.9.norm1.weight", "encoder.layers_2.blocks.9.norm1.bias", "encoder.layers_2.blocks.9.attn.relative_position_bias_table", "encoder.layers_2.blocks.9.attn.qkv.weight", "encoder.layers_2.blocks.9.attn.qkv.bias", "encoder.layers_2.blocks.9.attn.proj.weight", "encoder.layers_2.blocks.9.attn.proj.bias", "encoder.layers_2.blocks.9.norm2.weight", "encoder.layers_2.blocks.9.norm2.bias", "encoder.layers_2.blocks.9.mlp.fc1.weight", "encoder.layers_2.blocks.9.mlp.fc1.bias", "encoder.layers_2.blocks.9.mlp.fc2.weight", "encoder.layers_2.blocks.9.mlp.fc2.bias", "encoder.layers_2.blocks.10.norm1.weight", "encoder.layers_2.blocks.10.norm1.bias", "encoder.layers_2.blocks.10.attn.relative_position_bias_table", "encoder.layers_2.blocks.10.attn.qkv.weight", "encoder.layers_2.blocks.10.attn.qkv.bias", "encoder.layers_2.blocks.10.attn.proj.weight", "encoder.layers_2.blocks.10.attn.proj.bias", "encoder.layers_2.blocks.10.norm2.weight", "encoder.layers_2.blocks.10.norm2.bias", "encoder.layers_2.blocks.10.mlp.fc1.weight", "encoder.layers_2.blocks.10.mlp.fc1.bias", "encoder.layers_2.blocks.10.mlp.fc2.weight", "encoder.layers_2.blocks.10.mlp.fc2.bias", "encoder.layers_2.blocks.11.norm1.weight", "encoder.layers_2.blocks.11.norm1.bias", "encoder.layers_2.blocks.11.attn.relative_position_bias_table", "encoder.layers_2.blocks.11.attn.qkv.weight", "encoder.layers_2.blocks.11.attn.qkv.bias", "encoder.layers_2.blocks.11.attn.proj.weight", "encoder.layers_2.blocks.11.attn.proj.bias", "encoder.layers_2.blocks.11.norm2.weight", "encoder.layers_2.blocks.11.norm2.bias", "encoder.layers_2.blocks.11.mlp.fc1.weight", "encoder.layers_2.blocks.11.mlp.fc1.bias", "encoder.layers_2.blocks.11.mlp.fc2.weight", "encoder.layers_2.blocks.11.mlp.fc2.bias", "encoder.layers_2.blocks.12.norm1.weight", "encoder.layers_2.blocks.12.norm1.bias", "encoder.layers_2.blocks.12.attn.relative_position_bias_table", "encoder.layers_2.blocks.12.attn.qkv.weight", "encoder.layers_2.blocks.12.attn.qkv.bias", "encoder.layers_2.blocks.12.attn.proj.weight", "encoder.layers_2.blocks.12.attn.proj.bias", "encoder.layers_2.blocks.12.norm2.weight", "encoder.layers_2.blocks.12.norm2.bias", "encoder.layers_2.blocks.12.mlp.fc1.weight", "encoder.layers_2.blocks.12.mlp.fc1.bias", "encoder.layers_2.blocks.12.mlp.fc2.weight", "encoder.layers_2.blocks.12.mlp.fc2.bias", "encoder.layers_2.blocks.13.norm1.weight", "encoder.layers_2.blocks.13.norm1.bias", "encoder.layers_2.blocks.13.attn.relative_position_bias_table", "encoder.layers_2.blocks.13.attn.qkv.weight", "encoder.layers_2.blocks.13.attn.qkv.bias", "encoder.layers_2.blocks.13.attn.proj.weight", "encoder.layers_2.blocks.13.attn.proj.bias", "encoder.layers_2.blocks.13.norm2.weight", "encoder.layers_2.blocks.13.norm2.bias", "encoder.layers_2.blocks.13.mlp.fc1.weight", "encoder.layers_2.blocks.13.mlp.fc1.bias", "encoder.layers_2.blocks.13.mlp.fc2.weight", "encoder.layers_2.blocks.13.mlp.fc2.bias", "encoder.layers_2.blocks.14.norm1.weight", "encoder.layers_2.blocks.14.norm1.bias", "encoder.layers_2.blocks.14.attn.relative_position_bias_table", "encoder.layers_2.blocks.14.attn.qkv.weight", "encoder.layers_2.blocks.14.attn.qkv.bias", "encoder.layers_2.blocks.14.attn.proj.weight", "encoder.layers_2.blocks.14.attn.proj.bias", "encoder.layers_2.blocks.14.norm2.weight", "encoder.layers_2.blocks.14.norm2.bias", "encoder.layers_2.blocks.14.mlp.fc1.weight", "encoder.layers_2.blocks.14.mlp.fc1.bias", "encoder.layers_2.blocks.14.mlp.fc2.weight", "encoder.layers_2.blocks.14.mlp.fc2.bias", "encoder.layers_2.blocks.15.norm1.weight", "encoder.layers_2.blocks.15.norm1.bias", "encoder.layers_2.blocks.15.attn.relative_position_bias_table", "encoder.layers_2.blocks.15.attn.qkv.weight", "encoder.layers_2.blocks.15.attn.qkv.bias", "encoder.layers_2.blocks.15.attn.proj.weight", "encoder.layers_2.blocks.15.attn.proj.bias", "encoder.layers_2.blocks.15.norm2.weight", "encoder.layers_2.blocks.15.norm2.bias", "encoder.layers_2.blocks.15.mlp.fc1.weight", "encoder.layers_2.blocks.15.mlp.fc1.bias", "encoder.layers_2.blocks.15.mlp.fc2.weight", "encoder.layers_2.blocks.15.mlp.fc2.bias", "encoder.layers_2.blocks.16.norm1.weight", "encoder.layers_2.blocks.16.norm1.bias", "encoder.layers_2.blocks.16.attn.relative_position_bias_table", "encoder.layers_2.blocks.16.attn.qkv.weight", "encoder.layers_2.blocks.16.attn.qkv.bias", "encoder.layers_2.blocks.16.attn.proj.weight", "encoder.layers_2.blocks.16.attn.proj.bias", "encoder.layers_2.blocks.16.norm2.weight", "encoder.layers_2.blocks.16.norm2.bias", "encoder.layers_2.blocks.16.mlp.fc1.weight", "encoder.layers_2.blocks.16.mlp.fc1.bias", "encoder.layers_2.blocks.16.mlp.fc2.weight", "encoder.layers_2.blocks.16.mlp.fc2.bias", "encoder.layers_2.blocks.17.norm1.weight", "encoder.layers_2.blocks.17.norm1.bias", "encoder.layers_2.blocks.17.attn.relative_position_bias_table", "encoder.layers_2.blocks.17.attn.qkv.weight", "encoder.layers_2.blocks.17.attn.qkv.bias", "encoder.layers_2.blocks.17.attn.proj.weight", "encoder.layers_2.blocks.17.attn.proj.bias", "encoder.layers_2.blocks.17.norm2.weight", "encoder.layers_2.blocks.17.norm2.bias", "encoder.layers_2.blocks.17.mlp.fc1.weight", "encoder.layers_2.blocks.17.mlp.fc1.bias", "encoder.layers_2.blocks.17.mlp.fc2.weight", "encoder.layers_2.blocks.17.mlp.fc2.bias", "adapter.adapter.0.attn.q_proj.weight", "adapter.adapter.0.attn.k_proj.weight", "adapter.adapter.0.attn.v_proj.weight", "adapter.adapter.0.attn.output_proj.weight", "adapter.adapter.0.mlp.w1.weight", "adapter.adapter.0.mlp.w1.bias", "adapter.adapter.0.mlp.w2.weight", "adapter.adapter.0.mlp.w2.bias", "adapter.adapter.0.sa_norm.scale", "adapter.adapter.0.mlp_norm.scale", "adapter.adapter.0.sa_scale.scale", "adapter.adapter.0.mlp_scale.scale", "adapter.adapter.1.attn.q_proj.weight", "adapter.adapter.1.attn.k_proj.weight", "adapter.adapter.1.attn.v_proj.weight", "adapter.adapter.1.attn.output_proj.weight", "adapter.adapter.1.mlp.w1.weight", "adapter.adapter.1.mlp.w1.bias", "adapter.adapter.1.mlp.w2.weight", "adapter.adapter.1.mlp.w2.bias", "adapter.adapter.1.sa_norm.scale", "adapter.adapter.1.mlp_norm.scale", "adapter.adapter.1.sa_scale.scale", "adapter.adapter.1.mlp_scale.scale", "adapter.adapter.2.attn.q_proj.weight", "adapter.adapter.2.attn.k_proj.weight", "adapter.adapter.2.attn.v_proj.weight", "adapter.adapter.2.attn.output_proj.weight", "adapter.adapter.2.mlp.w1.weight", "adapter.adapter.2.mlp.w1.bias", "adapter.adapter.2.mlp.w2.weight", "adapter.adapter.2.mlp.w2.bias", "adapter.adapter.2.sa_norm.scale", "adapter.adapter.2.mlp_norm.scale", "adapter.adapter.2.sa_scale.scale", "adapter.adapter.2.mlp_scale.scale", "adapter.adapter.3.attn.q_proj.weight", "adapter.adapter.3.attn.k_proj.weight", "adapter.adapter.3.attn.v_proj.weight", "adapter.adapter.3.attn.output_proj.weight", "adapter.adapter.3.mlp.w1.weight", "adapter.adapter.3.mlp.w1.bias", "adapter.adapter.3.mlp.w2.weight", "adapter.adapter.3.mlp.w2.bias", "adapter.adapter.3.sa_norm.scale", "adapter.adapter.3.mlp_norm.scale", "adapter.adapter.3.sa_scale.scale", "adapter.adapter.3.mlp_scale.scale", "adapter.adapter.4.weight", "adapter.adapter.4.bias". Unexpected key(s) in state_dict: "encoder.layers.0.blocks.0.attn.proj.bias", "encoder.layers.0.blocks.0.attn.proj.weight", "encoder.layers.0.blocks.0.attn.qkv.bias", "encoder.layers.0.blocks.0.attn.qkv.weight", "encoder.layers.0.blocks.0.attn.relative_position_bias_table", "encoder.layers.0.blocks.0.mlp.fc1.bias", "encoder.layers.0.blocks.0.mlp.fc1.weight", "encoder.layers.0.blocks.0.mlp.fc2.bias", "encoder.layers.0.blocks.0.mlp.fc2.weight", "encoder.layers.0.blocks.0.norm1.bias", "encoder.layers.0.blocks.0.norm1.weight", "encoder.layers.0.blocks.0.norm2.bias", "encoder.layers.0.blocks.0.norm2.weight", "encoder.layers.0.blocks.1.attn.proj.bias", "encoder.layers.0.blocks.1.attn.proj.weight", "encoder.layers.0.blocks.1.attn.qkv.bias", "encoder.layers.0.blocks.1.attn.qkv.weight", "encoder.layers.0.blocks.1.attn.relative_position_bias_table", "encoder.layers.0.blocks.1.mlp.fc1.bias", "encoder.layers.0.blocks.1.mlp.fc1.weight", "encoder.layers.0.blocks.1.mlp.fc2.bias", "encoder.layers.0.blocks.1.mlp.fc2.weight", "encoder.layers.0.blocks.1.norm1.bias", "encoder.layers.0.blocks.1.norm1.weight", "encoder.layers.0.blocks.1.norm2.bias", "encoder.layers.0.blocks.1.norm2.weight", "encoder.layers.1.blocks.0.attn.proj.bias", "encoder.layers.1.blocks.0.attn.proj.weight", "encoder.layers.1.blocks.0.attn.qkv.bias", "encoder.layers.1.blocks.0.attn.qkv.weight", "encoder.layers.1.blocks.0.attn.relative_position_bias_table", "encoder.layers.1.blocks.0.mlp.fc1.bias", "encoder.layers.1.blocks.0.mlp.fc1.weight", "encoder.layers.1.blocks.0.mlp.fc2.bias", "encoder.layers.1.blocks.0.mlp.fc2.weight", "encoder.layers.1.blocks.0.norm1.bias", "encoder.layers.1.blocks.0.norm1.weight", "encoder.layers.1.blocks.0.norm2.bias", "encoder.layers.1.blocks.0.norm2.weight", "encoder.layers.1.blocks.1.attn.proj.bias", "encoder.layers.1.blocks.1.attn.proj.weight", "encoder.layers.1.blocks.1.attn.qkv.bias", "encoder.layers.1.blocks.1.attn.qkv.weight", "encoder.layers.1.blocks.1.attn.relative_position_bias_table", "encoder.layers.1.blocks.1.mlp.fc1.bias", "encoder.layers.1.blocks.1.mlp.fc1.weight", "encoder.layers.1.blocks.1.mlp.fc2.bias", "encoder.layers.1.blocks.1.mlp.fc2.weight", "encoder.layers.1.blocks.1.norm1.bias", "encoder.layers.1.blocks.1.norm1.weight", "encoder.layers.1.blocks.1.norm2.bias", "encoder.layers.1.blocks.1.norm2.weight", "encoder.layers.1.downsample.norm.bias", "encoder.layers.1.downsample.norm.weight", "encoder.layers.1.downsample.reduction.weight", "encoder.layers.2.blocks.0.attn.proj.bias", "encoder.layers.2.blocks.0.attn.proj.weight", "encoder.layers.2.blocks.0.attn.qkv.bias", "encoder.layers.2.blocks.0.attn.qkv.weight", "encoder.layers.2.blocks.0.attn.relative_position_bias_table", "encoder.layers.2.blocks.0.mlp.fc1.bias", "encoder.layers.2.blocks.0.mlp.fc1.weight", "encoder.layers.2.blocks.0.mlp.fc2.bias", "encoder.layers.2.blocks.0.mlp.fc2.weight", "encoder.layers.2.blocks.0.norm1.bias", "encoder.layers.2.blocks.0.norm1.weight", "encoder.layers.2.blocks.0.norm2.bias", "encoder.layers.2.blocks.0.norm2.weight", "encoder.layers.2.blocks.1.attn.proj.bias", "encoder.layers.2.blocks.1.attn.proj.weight", "encoder.layers.2.blocks.1.attn.qkv.bias", "encoder.layers.2.blocks.1.attn.qkv.weight", "encoder.layers.2.blocks.1.attn.relative_position_bias_table", "encoder.layers.2.blocks.1.mlp.fc1.bias", "encoder.layers.2.blocks.1.mlp.fc1.weight", "encoder.layers.2.blocks.1.mlp.fc2.bias", "encoder.layers.2.blocks.1.mlp.fc2.weight", "encoder.layers.2.blocks.1.norm1.bias", "encoder.layers.2.blocks.1.norm1.weight", "encoder.layers.2.blocks.1.norm2.bias", "encoder.layers.2.blocks.1.norm2.weight", "encoder.layers.2.blocks.10.attn.proj.bias", "encoder.layers.2.blocks.10.attn.proj.weight", "encoder.layers.2.blocks.10.attn.qkv.bias", "encoder.layers.2.blocks.10.attn.qkv.weight", "encoder.layers.2.blocks.10.attn.relative_position_bias_table", "encoder.layers.2.blocks.10.mlp.fc1.bias", "encoder.layers.2.blocks.10.mlp.fc1.weight", "encoder.layers.2.blocks.10.mlp.fc2.bias", "encoder.layers.2.blocks.10.mlp.fc2.weight", "encoder.layers.2.blocks.10.norm1.bias", "encoder.layers.2.blocks.10.norm1.weight", "encoder.layers.2.blocks.10.norm2.bias", "encoder.layers.2.blocks.10.norm2.weight", "encoder.layers.2.blocks.11.attn.proj.bias", "encoder.layers.2.blocks.11.attn.proj.weight", "encoder.layers.2.blocks.11.attn.qkv.bias", "encoder.layers.2.blocks.11.attn.qkv.weight", "encoder.layers.2.blocks.11.attn.relative_position_bias_table", "encoder.layers.2.blocks.11.mlp.fc1.bias", "encoder.layers.2.blocks.11.mlp.fc1.weight", "encoder.layers.2.blocks.11.mlp.fc2.bias", "encoder.layers.2.blocks.11.mlp.fc2.weight", "encoder.layers.2.blocks.11.norm1.bias", "encoder.layers.2.blocks.11.norm1.weight", "encoder.layers.2.blocks.11.norm2.bias", "encoder.layers.2.blocks.11.norm2.weight", "encoder.layers.2.blocks.12.attn.proj.bias", "encoder.layers.2.blocks.12.attn.proj.weight", "encoder.layers.2.blocks.12.attn.qkv.bias", "encoder.layers.2.blocks.12.attn.qkv.weight", "encoder.layers.2.blocks.12.attn.relative_position_bias_table", "encoder.layers.2.blocks.12.mlp.fc1.bias", "encoder.layers.2.blocks.12.mlp.fc1.weight", "encoder.layers.2.blocks.12.mlp.fc2.bias", "encoder.layers.2.blocks.12.mlp.fc2.weight", "encoder.layers.2.blocks.12.norm1.bias", "encoder.layers.2.blocks.12.norm1.weight", "encoder.layers.2.blocks.12.norm2.bias", "encoder.layers.2.blocks.12.norm2.weight", "encoder.layers.2.blocks.13.attn.proj.bias", "encoder.layers.2.blocks.13.attn.proj.weight", "encoder.layers.2.blocks.13.attn.qkv.bias", "encoder.layers.2.blocks.13.attn.qkv.weight", "encoder.layers.2.blocks.13.attn.relative_position_bias_table", "encoder.layers.2.blocks.13.mlp.fc1.bias", "encoder.layers.2.blocks.13.mlp.fc1.weight", "encoder.layers.2.blocks.13.mlp.fc2.bias", "encoder.layers.2.blocks.13.mlp.fc2.weight", "encoder.layers.2.blocks.13.norm1.bias", "encoder.layers.2.blocks.13.norm1.weight", "encoder.layers.2.blocks.13.norm2.bias", "encoder.layers.2.blocks.13.norm2.weight", "encoder.layers.2.blocks.14.attn.proj.bias", "encoder.layers.2.blocks.14.attn.proj.weight", "encoder.layers.2.blocks.14.attn.qkv.bias", "encoder.layers.2.blocks.14.attn.qkv.weight", "encoder.layers.2.blocks.14.attn.relative_position_bias_table", "encoder.layers.2.blocks.14.mlp.fc1.bias", "encoder.layers.2.blocks.14.mlp.fc1.weight", "encoder.layers.2.blocks.14.mlp.fc2.bias", "encoder.layers.2.blocks.14.mlp.fc2.weight", "encoder.layers.2.blocks.14.norm1.bias", "encoder.layers.2.blocks.14.norm1.weight", "encoder.layers.2.blocks.14.norm2.bias", "encoder.layers.2.blocks.14.norm2.weight", "encoder.layers.2.blocks.15.attn.proj.bias", "encoder.layers.2.blocks.15.attn.proj.weight", "encoder.layers.2.blocks.15.attn.qkv.bias", "encoder.layers.2.blocks.15.attn.qkv.weight", "encoder.layers.2.blocks.15.attn.relative_position_bias_table", "encoder.layers.2.blocks.15.mlp.fc1.bias", "encoder.layers.2.blocks.15.mlp.fc1.weight", "encoder.layers.2.blocks.15.mlp.fc2.bias", "encoder.layers.2.blocks.15.mlp.fc2.weight", "encoder.layers.2.blocks.15.norm1.bias", "encoder.layers.2.blocks.15.norm1.weight", "encoder.layers.2.blocks.15.norm2.bias", "encoder.layers.2.blocks.15.norm2.weight", "encoder.layers.2.blocks.16.attn.proj.bias", "encoder.layers.2.blocks.16.attn.proj.weight", "encoder.layers.2.blocks.16.attn.qkv.bias", "encoder.layers.2.blocks.16.attn.qkv.weight", "encoder.layers.2.blocks.16.attn.relative_position_bias_table", "encoder.layers.2.blocks.16.mlp.fc1.bias", "encoder.layers.2.blocks.16.mlp.fc1.weight", "encoder.layers.2.blocks.16.mlp.fc2.bias", "encoder.layers.2.blocks.16.mlp.fc2.weight", "encoder.layers.2.blocks.16.norm1.bias", "encoder.layers.2.blocks.16.norm1.weight", "encoder.layers.2.blocks.16.norm2.bias", "encoder.layers.2.blocks.16.norm2.weight", "encoder.layers.2.blocks.17.attn.proj.bias", "encoder.layers.2.blocks.17.attn.proj.weight", "encoder.layers.2.blocks.17.attn.qkv.bias", "encoder.layers.2.blocks.17.attn.qkv.weight", "encoder.layers.2.blocks.17.attn.relative_position_bias_table", "encoder.layers.2.blocks.17.mlp.fc1.bias", "encoder.layers.2.blocks.17.mlp.fc1.weight", "encoder.layers.2.blocks.17.mlp.fc2.bias", "encoder.layers.2.blocks.17.mlp.fc2.weight", "encoder.layers.2.blocks.17.norm1.bias", "encoder.layers.2.blocks.17.norm1.weight", "encoder.layers.2.blocks.17.norm2.bias", "encoder.layers.2.blocks.17.norm2.weight", "encoder.layers.2.blocks.2.attn.proj.bias", "encoder.layers.2.blocks.2.attn.proj.weight", "encoder.layers.2.blocks.2.attn.qkv.bias", "encoder.layers.2.blocks.2.attn.qkv.weight", "encoder.layers.2.blocks.2.attn.relative_position_bias_table", "encoder.layers.2.blocks.2.mlp.fc1.bias", "encoder.layers.2.blocks.2.mlp.fc1.weight", "encoder.layers.2.blocks.2.mlp.fc2.bias", "encoder.layers.2.blocks.2.mlp.fc2.weight", "encoder.layers.2.blocks.2.norm1.bias", "encoder.layers.2.blocks.2.norm1.weight", "encoder.layers.2.blocks.2.norm2.bias", "encoder.layers.2.blocks.2.norm2.weight", "encoder.layers.2.blocks.3.attn.proj.bias", "encoder.layers.2.blocks.3.attn.proj.weight", "encoder.layers.2.blocks.3.attn.qkv.bias", "encoder.layers.2.blocks.3.attn.qkv.weight", "encoder.layers.2.blocks.3.attn.relative_position_bias_table", "encoder.layers.2.blocks.3.mlp.fc1.bias", "encoder.layers.2.blocks.3.mlp.fc1.weight", "encoder.layers.2.blocks.3.mlp.fc2.bias", "encoder.layers.2.blocks.3.mlp.fc2.weight", "encoder.layers.2.blocks.3.norm1.bias", "encoder.layers.2.blocks.3.norm1.weight", "encoder.layers.2.blocks.3.norm2.bias", "encoder.layers.2.blocks.3.norm2.weight", "encoder.layers.2.blocks.4.attn.proj.bias", "encoder.layers.2.blocks.4.attn.proj.weight", "encoder.layers.2.blocks.4.attn.qkv.bias", "encoder.layers.2.blocks.4.attn.qkv.weight", "encoder.layers.2.blocks.4.attn.relative_position_bias_table", "encoder.layers.2.blocks.4.mlp.fc1.bias", "encoder.layers.2.blocks.4.mlp.fc1.weight", "encoder.layers.2.blocks.4.mlp.fc2.bias", "encoder.layers.2.blocks.4.mlp.fc2.weight", "encoder.layers.2.blocks.4.norm1.bias", "encoder.layers.2.blocks.4.norm1.weight", "encoder.layers.2.blocks.4.norm2.bias", "encoder.layers.2.blocks.4.norm2.weight", "encoder.layers.2.blocks.5.attn.proj.bias", "encoder.layers.2.blocks.5.attn.proj.weight", "encoder.layers.2.blocks.5.attn.qkv.bias", "encoder.layers.2.blocks.5.attn.qkv.weight", "encoder.layers.2.blocks.5.attn.relative_position_bias_table", "encoder.layers.2.blocks.5.mlp.fc1.bias", "encoder.layers.2.blocks.5.mlp.fc1.weight", "encoder.layers.2.blocks.5.mlp.fc2.bias", "encoder.layers.2.blocks.5.mlp.fc2.weight", "encoder.layers.2.blocks.5.norm1.bias", "encoder.layers.2.blocks.5.norm1.weight", "encoder.layers.2.blocks.5.norm2.bias", "encoder.layers.2.blocks.5.norm2.weight", "encoder.layers.2.blocks.6.attn.proj.bias", "encoder.layers.2.blocks.6.attn.proj.weight", "encoder.layers.2.blocks.6.attn.qkv.bias", "encoder.layers.2.blocks.6.attn.qkv.weight", "encoder.layers.2.blocks.6.attn.relative_position_bias_table", "encoder.layers.2.blocks.6.mlp.fc1.bias", "encoder.layers.2.blocks.6.mlp.fc1.weight", "encoder.layers.2.blocks.6.mlp.fc2.bias", "encoder.layers.2.blocks.6.mlp.fc2.weight", "encoder.layers.2.blocks.6.norm1.bias", "encoder.layers.2.blocks.6.norm1.weight", "encoder.layers.2.blocks.6.norm2.bias", "encoder.layers.2.blocks.6.norm2.weight", "encoder.layers.2.blocks.7.attn.proj.bias", "encoder.layers.2.blocks.7.attn.proj.weight", "encoder.layers.2.blocks.7.attn.qkv.bias", "encoder.layers.2.blocks.7.attn.qkv.weight", "encoder.layers.2.blocks.7.attn.relative_position_bias_table", "encoder.layers.2.blocks.7.mlp.fc1.bias", "encoder.layers.2.blocks.7.mlp.fc1.weight", "encoder.layers.2.blocks.7.mlp.fc2.bias", "encoder.layers.2.blocks.7.mlp.fc2.weight", "encoder.layers.2.blocks.7.norm1.bias", "encoder.layers.2.blocks.7.norm1.weight", "encoder.layers.2.blocks.7.norm2.bias", "encoder.layers.2.blocks.7.norm2.weight", "encoder.layers.2.blocks.8.attn.proj.bias", "encoder.layers.2.blocks.8.attn.proj.weight", "encoder.layers.2.blocks.8.attn.qkv.bias", "encoder.layers.2.blocks.8.attn.qkv.weight", "encoder.layers.2.blocks.8.attn.relative_position_bias_table", "encoder.layers.2.blocks.8.mlp.fc1.bias", "encoder.layers.2.blocks.8.mlp.fc1.weight", "encoder.layers.2.blocks.8.mlp.fc2.bias", "encoder.layers.2.blocks.8.mlp.fc2.weight", "encoder.layers.2.blocks.8.norm1.bias", "encoder.layers.2.blocks.8.norm1.weight", "encoder.layers.2.blocks.8.norm2.bias", "encoder.layers.2.blocks.8.norm2.weight", "encoder.layers.2.blocks.9.attn.proj.bias", "encoder.layers.2.blocks.9.attn.proj.weight", "encoder.layers.2.blocks.9.attn.qkv.bias", "encoder.layers.2.blocks.9.attn.qkv.weight", "encoder.layers.2.blocks.9.attn.relative_position_bias_table", "encoder.layers.2.blocks.9.mlp.fc1.bias", "encoder.layers.2.blocks.9.mlp.fc1.weight", "encoder.layers.2.blocks.9.mlp.fc2.bias", "encoder.layers.2.blocks.9.mlp.fc2.weight", "encoder.layers.2.blocks.9.norm1.bias", "encoder.layers.2.blocks.9.norm1.weight", "encoder.layers.2.blocks.9.norm2.bias", "encoder.layers.2.blocks.9.norm2.weight", "encoder.layers.2.downsample.norm.bias", "encoder.layers.2.downsample.norm.weight", "encoder.layers.2.downsample.reduction.weight", "adapter.0.attn.k_proj.weight", "adapter.0.attn.output_proj.weight", "adapter.0.attn.q_proj.weight", "adapter.0.attn.v_proj.weight", "adapter.0.mlp.w1.bias", "adapter.0.mlp.w1.weight", "adapter.0.mlp.w2.bias", "adapter.0.mlp.w2.weight", "adapter.0.mlp_norm.scale", "adapter.0.mlp_scale.scale", "adapter.0.sa_norm.scale", "adapter.0.sa_scale.scale", "adapter.1.attn.k_proj.weight", "adapter.1.attn.output_proj.weight", "adapter.1.attn.q_proj.weight", "adapter.1.attn.v_proj.weight", "adapter.1.mlp.w1.bias", "adapter.1.mlp.w1.weight", "adapter.1.mlp.w2.bias", "adapter.1.mlp.w2.weight", "adapter.1.mlp_norm.scale", "adapter.1.mlp_scale.scale", "adapter.1.sa_norm.scale", "adapter.1.sa_scale.scale", "adapter.2.attn.k_proj.weight", "adapter.2.attn.output_proj.weight", "adapter.2.attn.q_proj.weight", "adapter.2.attn.v_proj.weight", "adapter.2.mlp.w1.bias", "adapter.2.mlp.w1.weight", "adapter.2.mlp.w2.bias", "adapter.2.mlp.w2.weight", "adapter.2.mlp_norm.scale", "adapter.2.mlp_scale.scale", "adapter.2.sa_norm.scale", "adapter.2.sa_scale.scale", "adapter.3.attn.k_proj.weight", "adapter.3.attn.output_proj.weight", "adapter.3.attn.q_proj.weight", "adapter.3.attn.v_proj.weight", "adapter.3.mlp.w1.bias", "adapter.3.mlp.w1.weight", "adapter.3.mlp.w2.bias", "adapter.3.mlp.w2.weight", "adapter.3.mlp_norm.scale", "adapter.3.mlp_scale.scale", "adapter.3.sa_norm.scale", "adapter.3.sa_scale.scale", "adapter.4.bias", "adapter.4.weight". ``` -- Reply to this email directly or view it on GitHub: #21 You are receiving this because you are subscribed to this thread. Message ID: ***@***.***>

[jesusbft]

Thank you for letting me know, I'll test it