Skip to content

Error while loading checkpoint when training #41

Description

@ManoniLo

I have to train ICNet on Camvid dataset.
I initialize the network with classification weights of the ResNet with a single stage training.

I set .config file and train_mem_saving.py input arguments as specified in https://github.com/oandrienko/fast-semantic-segmentation/blob/master/docs/icnet.md.

When initializing the model from the checkpoint "tmp/resnet_v1_50.ckpt", I get this error:

"Restoring from checkpoint failed. This is most likely due to a Variable name or other graph key that is missing from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:
Key AuxOutput/biases not found in checkpoint
[[node save_1/RestoreV2 (defined at /content/libs/trainer.py:286) ]]".

It is strange since it sholdn't try to load variable 'AuxOutput/biases', since it does not belong to ResNet checkpoint.
I correctly set 'fine_tune_checkpoint_type' to 'classification' in .config file.

I am running the project on Google Colab which supports which supports tensorflow versions only greater than 1.13.1.
Has anyone successfully run the project with these versions of tf?

Thank you in advance

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions