Feature request: aim-mlflow option to skip artifact downloads

[josiahbjorgaard]

Could an option be added to the aim-mlflow sync operation to skip downloading artifacts?

[arkkln]

@josiahbjorgaard thanks for the suggestion! Could you please describe the motivation?

[josiahbjorgaard]

For example, we have multiple GBs of artifacts attached to our MLFlow logs (model weights, png plots, data), mostly stored on S3 (not colocated with the MLFlow GUI). Automatically syncing it all causes me to run out of disk space. Many of the artifacts are intended for later usage, like for restarting training or pushing model weights to production, rather than for analysis.

[arkkln]

@josiahbjorgaard this makes sense. I think it would be even better to provide a more flexible interface to enable filtering artifacts rather than completely disabling syncing. Something like:

• aimlflow sync --excluded-artifacts="*.png" (exclude artifacts based on name)
• aimlflow sync --included-artifacts="*.png" (sync only matching artifacts)
  Thoughts?

[pezosanta]

+1 for this feature request