SIP-024: spin deps cli dx

[fibonacci1729]

This SIP proposes a Dx for guiding a user into adding a dependency to a component in a Spin.toml. It builds on #3445 and the implementation #3450.

[itowlson]

LGTM. I like the capabilities stuff; in fact I almost wonder if I should be able to run the capabilities stuff outside of doing an add.

[itowlson]

This is not surfaced interactively, right? (I would like it not to be - I just want to check we are on the same page.)

oh wait, this is the interface on the left hand side, not the export = option? That coincidence of terminology is going to be a trap

[fibonacci1729]

They're equivalent when selecting interfaces. I couldnt think of a better name here that encapsulated packages and interfaces. How do you feel about --use?

[itowlson]

Wait once again it seems I am utterly confused about a feature we have been shipping for a gazillion years - I thought export was about remapping interfaces... sigh

I was initially going to suggest --interface, but I now realise that if you want all a package's APIs, you have to tell it the package name if you don't want it to interact. --use seems a bit open-ended but could be an option. We could try --import perhaps? It's not great though. Well maybe it's okay, I originally thought "ugh, importing exports" but well, huh, that is what importers do, so maybe it makes sense?

Or send up the BIKESHED BAT-SIGNAL and let Lann come up with something.

[fibonacci1729]

I think import works. We could also keep export for the remapping scenario you mention which is the point of it. All I meant was that if the LHS is an interface, it implies that an export with that name exists with that name so its technically equivalent.

[fibonacci1729]

I.e. "a:b/c" = { export = none, ... } is equivalent to "a:b/c" = { export = "a:b/c", ... }

[itowlson]

Aha, I think I see what you mean now. Thanks!

[fibonacci1729]

We could allow add a mutually exclusive --package/-p, --interface/-i.

[fibonacci1729]

Actually never mind, I forgot about plain names.

[itowlson]

Presumably these will be check boxes rather than a single-select?

[fibonacci1729]

Indeed

[itowlson]

Or --inherit allowed_outbound_hosts --inherit ai_models, right? (that is, as well as not instead of)

[itowlson]

I worry with this that we may be asking a question the user doesn't know how to answer. I want the authentication interface, is that in client or util? If I guess wrong, is there a way back?

Possible alternative approaches:

1. Show them a list of everything they could import - all the interfaces plus an "everything" for each package.
2. When they come to interface selection, offer a "The interface I'm looking for isn't here. Take me back to package selection" option

Or maybe this isn't a concern and we have high confidence that users will know what they are looking for and just want to save on ye olde typing, I am not sure.

[fibonacci1729]

Good call out. Ill have a tinker and see if i could smooth this out. I think listing everything with all options for each package is the way to go.

[rmarx]

I was thinking something similar, but you might run into an issue where there's just a TON of exports in a given dependency, and navigating them manually would get annoying.

Counterpoint is of course that that won't happen super-often and you only need to do this once/infrequently, so I'd also be in favor of having just 1 flat list of everything with the "All from DEP" in the list as just another option.

[itowlson]

Nit: I would avoid saying 'export' here (because I want to import something, where is the choice for that). Consider e.g. "Which interface do you want to import?" or some better wording that avoids the whole import/export debacle altogether (a la "which interface do you want to use" although that's hardly going to set the poetry world alight either)

[rmarx]

I had a similar feeling when seeing the CLI argument is called --export instead of --import... it depends on how you want to look at it of course.

However, since I'm adding a dependency to a specific component, I probably want that component to import/use stuff from the dependency (I don't necessarily want to think about the stuff the dependency exports).

Kind of like a github PR should really be a Merge Request instead? :D (imo)

Either way, if we keep --export (instead of --import or --interfaces) I agree it would be more sensible to have some other language here like suggested by @itowlson

EDIT: just now reading the whole discussion on this below :)

[itowlson]

wasi:http/client, spin:postgres, fermyon:spin/mysql, spin:mqtt, Redis itfs

[itowlson]

All of these have spin:* forms as of Spin 4 (actually SQLite as of Spin 3.2 I think)

[karthik2804]

Should we also support adding by component ID? I am imagining a case where we have a component that we already build as part of the manifest that we would like to add as dependency to another.

[rmarx]

Strong agree on this one. I don't want to have to know/copy-paste the build output path of my Rust component to add it as a dependency of my TS component in the same spin project. I want that to be auto-figured out from the component ID

[rmarx]

Nit: This feels a bit weird... I get that it's consistent with e.g., spin build, but there it makes more semantic sense (you build "from" a .toml).

In this setup, it's a bit weird given that you're adding dependencies TO something.

In spin add the -f is --file not --from, which would make more sense here as well imo.

[rmarx]

Suggested change

	| `--export <name>` | Export to use from the dependency. Prompted if omitted and the component has multiple exports. |
	| `--export <name>` | Export to use from the dependency. Prompted if omitted and the dependency has multiple exports. |

[rmarx]

I'm wondering if this document should also detail the failure path? Or if there's somewhere else that describes exactly how you "resolve" a Wasm component from an HTTP URL/registry and what happens if it can't be found / if it doesn't actually export anything that can be used?

What would be the error message shown if the dependency can't be found? If it's somehow incompatible (e.g., wasm-bindgen .wasm from HTTP URL)?

When does spin try the resolution? Before or after or parallel with showing the interactive prompt for --to? Is there a progress-bar or something for large/slow downloads?

[rmarx]

EDIT: just read the bottom of the doc saying that multiple exports are not yet in-scope. Decided to leave the comment though, as I do need we can/should make it a bit clearer in this doc/eventual documentation that it's intended you can only add 1 interface at a time.

Just want to confirm if/how multiple exports work. I assume that

--export aws:client/s3@1.0.0,aws:client/sqs@1.0.0
and
--export aws:client/s3@1.0.0 --export aws:client/sqs@1.0.0

would both work? Or is it always just 1 export at a time?

If there are multiple possible at once, then some text above needs to be adjusted (e.g., "Which exportS should be used").

If however you can only do 1 interface at a time, that can probably also be made a bit clearer in the text/description that people would run multiple spin deps add commands for multiple exports, even if from the same package.

[rmarx]

Suggested change

	| `--to <component-id>` | Target component to add the dependency to. Prompted if omitted and the app has multiple components. |
	| `--to <component-id>` | Parent component to add the dependency to. Prompted if omitted and the app has multiple components. |

To be more consistent with later prose (i.e., inheriting "parent" component capabilities)

[rmarx]

So is that inherit_configuration = [] or simply no inherit_configuration?

[rmarx]

Is this printed only for capabilities that were not inherited?

Would be confusing if you explicitly chose --inherit all or --inherit ai_models and then still saw this kind of message at the end?

The example below does indicate this warning would be present if inheriting, so I think that's not ideal.

[rmarx]

Reading it a bit more, it seems it's saying you might need to actually fill in ai_models on the PARENT component, even if you added it as --inherit here.

Conceptually that makes sense, but it's confusing imo.

Is there a way to check if the parent component has the capabilities already filled out and only print this type of warning if it doesn't?

[rmarx]

Suggested change

	"my-export" = { path = "my-component.wasm" }
	"my-export" = { path = "./my-component.wasm" }

Nit for consistency

[rmarx]

Reading this made me wonder what happens if a user wants to REMOVE a dependency instead of adding it (obviously not covered by spin deps ADD but still).

Is it just removing the line in the .toml and spin build picks that up and re-generates the spin-dependencies.wit to remove the dependency? Or do you need a separate command to regen spin-dependencies.wit?

In the first case: why would regeneration be part of spin deps add instead of just spin build?
In the second case: do we have a separate spin deps remove or something to help re-trigger?

[rmarx]

Hey all,

Thanks for working on this important part of the spin ecosystem.

I added a few comments, but most of them are nits and only few have to do with the underlying proposal/design, which in general I think is solid.

I'm quite new to spin (Akamai employee working with the Fermyon team) so I might have made some wrong assumptions/basic errors (e.g., not knowing how external dependency resolution error handling is done, not knowing if there's a way to remove added dependencies, ...). Feel free to correct me where wrong :)

One of the bigger missing pieces in this proposal as a higher-level user is the tie-in with WIT and auto-generation of programming language bindings when a dependency is added. I reckon that's documented elsewhere / seen as a separate project (which is fine), but I might have expected just a bit more tie-in to how that's supposed to interact here. Currently the text just talks about re-generating the spin-dependencies.wit file, but it's not entirely clear what that does exactly/how that flows into getting the dependency actually ready for use from within other components. If another command is needed (say... spin deps bind who knows) it would be good if that next step would e.g., be mentioned in the output text after a spin deps add potentially).