Skip to content

add: tutorial about multilora using inference components - #2675

Merged
dwarez merged 2 commits into
mainfrom
multilora-tutorial
Aug 5, 2026
Merged

add: tutorial about multilora using inference components#2675
dwarez merged 2 commits into
mainfrom
multilora-tutorial

Conversation

@dwarez

@dwarez dwarez commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Note

Low Risk
Documentation-only addition (a Jupyter notebook); no application or library code changes.

Overview
Adds a new SageMaker SDK notebook at docs/sagemaker/notebooks/sagemaker-sdk/multi-lora-adapters-vllm/sagemaker-notebook.ipynb that walks through serving multiple LoRA adapters from one GPU endpoint using the Hugging Face vLLM DLC and SageMaker inference components.

The tutorial covers packaging Hub adapters into S3 tar.gz archives, enabling LoRA and runtime adapter updates via vLLM env vars (including VLLM_ALLOW_RUNTIME_LORA_UPDATING), deploying Mistral 7B as a base inference component with SageMaker Python SDK v3 (Model, Endpoint, InferenceComponent), attaching adapter components, routing with InferenceComponentName and route=/v1/completions, and adding/removing adapters without redeploying. It also includes production notes and a cleanup section for billable resources.

Reviewed by Cursor Bugbot for commit 9c392c8. Bugbot is set up for automated code reviews on this repo. Configure here.

Signed-off-by: DWarez <dario.salvati@huggingface.co>
@dwarez
dwarez requested review from alvarobartt and ehcalabres July 31, 2026 14:27
@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 819b995. Configure here.

Signed-off-by: DWarez <dario.salvati@huggingface.co>
@dwarez
dwarez merged commit f31a31b into main Aug 5, 2026
3 checks passed
@dwarez
dwarez deleted the multilora-tutorial branch August 5, 2026 16:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants