Skip to content

Bundler::Source::Git: avoid re-fetching the repo - #9806

Merged
hsbt merged 1 commit into
ruby:masterfrom
byroot:git-gem-avoid-fetching
Aug 25, 2026
Merged

Bundler::Source::Git: avoid re-fetching the repo#9806
hsbt merged 1 commit into
ruby:masterfrom
byroot:git-gem-avoid-fetching

Conversation

@byroot

@byroot byroot commented Aug 24, 2026

Copy link
Copy Markdown
Member

In install, self.to_s is called by print_using_message "Using ... from #{self}. This in turn calls current_branch which if the local git cache is missing will re-clone it from scratch.

This is an issue, because a common optimisation for build system that are bundler aware is to prune bundler's cache to remove large git repositories.

See my feature request from 2020 for context: #7018

In our case, these git repositories account for over 50% of the bundler cache, causing the cache restoration and persist steps to be very significantly slowed down.

Overall I don't think triggering a full on git clone operation from a call to to_s make sense.

NB: This can be worked around by always specifying a ref: argument on the gem.

@byroot
byroot force-pushed the git-gem-avoid-fetching branch 2 times, most recently from 9c0411f to e548525 Compare August 24, 2026 15:13
Comment thread lib/bundler/source/git.rb Outdated
at = humanized_ref || current_branch

rev = "at #{at}@#{shortref_for_display(revision)}"
at = humanized_ref || @branch

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

|| @branch looks unreachable: @ref includes options["branch"], so whenever @branch is set humanized_ref already returns it. at = humanized_ref alone should be equivalent.

@hsbt

hsbt commented Aug 25, 2026

Copy link
Copy Markdown
Member

👍 Can you look my comment? I will merge this after that.

In `install`, `self.to_s` is called by `print_using_message "Using ... from #{self}`.
This in turn calls `current_branch` which if the local git cache
is missing will re-clone it from scratch.

This is an issue, because a common optimization for build system
that are bundler aware is to prune bundler's cache to remove
large git repositories.

See my feature request from 2020 for context:
ruby#7018

In our case, these git repositories account for over 50%
of the bundler cache, causing the cache restoration and
persist steps to be very significantly slowed down.

Overall I don't think triggering a full on git clone operation
from a call to `to_s` make sense.
@byroot
byroot force-pushed the git-gem-avoid-fetching branch from e548525 to 0733910 Compare August 25, 2026 06:06
@byroot

byroot commented Aug 25, 2026

Copy link
Copy Markdown
Member Author

@hsbt done!

Also while I have you, what do you think of that old feature request? #7018

I think it could turn into either a bundle prune command, or bundle install --prune or bundle clean --prune.

It's a thing lots of people are doing including the Ruby heroku buildpack:

@hsbt

hsbt commented Aug 25, 2026

Copy link
Copy Markdown
Member

@byroot I'm positive about providing an official prune feature.

I'm not sure what the right interface would be. An environment variable, a flag on an existing command, and a separate bundle prune subcommand all seem plausible to me. Do you have a preference method?

@byroot

byroot commented Aug 25, 2026

Copy link
Copy Markdown
Member Author

Do you have a preference method?

I suppose we could have them all, but if we go to the absolutely essential one, then I think it would be an environment variable, e.g. BUNDLE_PRUNE and BUNDLE_DEPLOYMENT would implies BUNDLE_PRUNE by default.

@byroot

byroot commented Aug 25, 2026

Copy link
Copy Markdown
Member Author

It's a thing lots of people are doing including the Ruby heroku buildpack:

And also the Rails default Dockerfile: https://github.com/rails/rails/blob/8d07fafa447e4979da96389a3f58cace92964ef2/railties/lib/rails/generators/rails/app/templates/Dockerfile.tt#L68

@byroot

byroot commented Aug 25, 2026

Copy link
Copy Markdown
Member Author

NB: the lint step failed because of some GItHub issue, but I don't have the permissions to re-run it. I think other than that the PR is green.

@hsbt

hsbt commented Aug 25, 2026

Copy link
Copy Markdown
Member

oh, thanks. I re-run that job.

@hsbt
hsbt merged commit a3c30db into ruby:master Aug 25, 2026
116 of 118 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants