Skip to content
Draft
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
104 changes: 104 additions & 0 deletions spec/wrap.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
# WRAP Specification

WRAP defines the rules for bi-directional communication between a Host and a Wasm module by defining the processes of invocation and subinvocation. It makes use of Host allocated Wasm shared memory to pass encoded buffers between the Host and the Wasm module.

The Wasm module must define the following imports and exports:

#### Exports
- `_invoke: (bufferLen: u32) => u32`
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What were your reasons for removing the "wrap" prefix? I do like the look of this, just curious. Maybe have a section in the doc that covers the considerations & design choices made from 0.1 -> 0.2, like we did in the ABI PR.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm torn, we can talk about it on the ACDC

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we rename "bufferLen" to "argsLen"? I think this buffer-use-case-specific naming is helpful for readers.

Copy link
Copy Markdown

@nerfZael nerfZael Feb 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed to optBufferLen to match the convention we have in the rest of the document.
It's the invoke options buffer length not the args buffer. Args buffer is inside of the invoke options as we defined below.

- sends the length of input buffer to Wasm module and it returns the length and the ptr to the new 8 byte long response buffer which contains length of the result buffer + pointer to the result buffer
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this sentence can be rewritten for better understanding. Maybe "The invoke export function receives the length of the invocation arguments buffer as its first and only argument. The invocation arguments buffer resides in an external host-controlled memory space, and must be copied into the receiving wasm module. To do this, the invoke export function's job is to allocate a new buffer of this size, and return a pointer to it, such that the host can copy the invocations arguments into it.

^^^ this might not be accurate, but you get my point, I think the wording of this paragraph can be massaged.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: it doesn't return the buffer ptr, but instead it uses the __fill_buffer import fn.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reworded this. The full explanation is in the invocation process below.


#### Imports
- `__subinvoke: (bufferPtr: u32, bufferLen: u32) => u32`
- Allows sending any buffer from the Wasm module to the host. Host returns the length of the result buffer to the Wasm module.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be great to specify here that subinvokes are "calls to other wrap module functions, where the invocation method and arguments are packed into a buffer". Being specific about how the buffer is packed here is necessary IMO. Sure this could change, but that'd require a new minor version rev of WRAP (0.3 for ex).

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reworded this. It's still not calls to other wrap module functions, it's calls to the host functions.
We have the buffer structure defined below, these are just the export/import signatures and their descriptions.

- `__fill_buffer: (bufferPtr: u32) => void`
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function feels too generic, and lacks context as to "what buffer is being filled". I think we could add a "bufferId" alongside the "bufferPtr" here. Also would need to add "bufferId" alongside the "bufferLen" in the _invoke export.

Copy link
Copy Markdown

@nerfZael nerfZael Feb 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reworded this a bit. But it is still a generic function, it can be either the invOptBuffer or subInvResBuffer that the host is sending to the wasm module. It doesn't need an ID to identify it because there can be only one type depending on context

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function can also be named load_buffer
What do you guys prefer: fill_buffer 🍏 vs load_buffer 🍎

- Fills the buffer at given buffer pointer with the data host wants to send to the Wasm module

### Invocation

Invocation is a process of calling any functions defined in the Wasm module directly from the Host.

To perform an invocation, one must provide Invocation options which consists of the function name and arguments. Before passing these options to the Wasm module, host must encode it into an invocation options buffer. The function arguments are serialized into a buffer using MsgPack. Then the invoking function name is UTF-8 encoded and concatenated with the encoded function name length and the arguments buffer into an invocation options buffer.

The host can call the `_invoke` export with the length of the invocation options buffer.

**Invocation options buffer:**

| funcNameLen | funcName | argsBuffer |
| -------- | -------- | -------- |

The Wasm module can use the`__fill_buffer` import to fill an allocated invocation options buffer.

Once the invocation completes, the Wasm module returns the response buffer pointer.

The response buffer is a helper buffer used to encode the pointer and length of the invocation result buffer.

**Response buffer:**

| resultBufferLength | resultBufferPointer |
| -------- | -------- |

The result buffer contains the Msgpack encoded return value of the invoked Wasm function.

### Subinvocation
Subinvocation is a process of calling any functions defined in the Host from the Wasm module.

Similar to an invocation, to perform a subinvocation, one must provide Invocation options which consists of function name and arguments. The Wasm module must encode it into an invocation options buffer. The function arguments are serialized into a buffer using MsgPack. Then the invoking function name is UTF-8 encoded and concatenated with the encoded function name length and the arguments buffer into an invocation options buffer.

The Wasm module can then call the `__subinvoke` import with the pointer and length of that buffer.

Once the subinvocation completes, the host returns the length of the result buffer.

The result buffer contains the Msgpack encoded return value of the subinvoked host function.


### Invocation sequence diagram

[![](https://mermaid.ink/img/pako:eNqFlVFvmzAQx7_KyU-JSqxBkpXykIdpmlZp2qb2YS9IkQOmtQY2MyZSVfW776gJ2EC2PKBg_j_f3f8O80oylXOSkIb_abnM-GfBnjSrUgn4Y5lRGu7lWf3m2i7VTBuRiZpJA19VY-arv1hTgV3uyc3hcNNpEwgpCHn-URuhZGM13QMU2OcRBUwCM1pZ2Ro2h5741BYecNPFSWBL4Sjew6wG3Tcu11bbaVBrpTsKrCxVxgxHUcG1TyyE6vE--T2GOhaiLI-nCf7T6PVCNR8pZKp-GXUBXEOQ2dgsFxK_pZBzxxXEnWwtoHlmQD-dVuFdGEAUbfGy3_chvivDQZ25BrthTOHRYMtsr4pWZl1DIEN7LKBqA6KApj1Zb-3qLLGYhkPDUHs_9my4G7yc-xnTqHN0CLJyIfQn8HYZmzrzOabbwSEX8TMZ4dGOC797_7fkxEKw_aTmB944kfDOq3nS3pjiVLhKrOuqu7eziZ2Q_4g79TqeTq-LjdO4UO9dP8YuEMB_8OlAL83Oh0nX7GZ-TZbmMl-eZMzti5Cieb46ygPqRe9ijyfNpYPCt9EnwgnhvIUPTe1DXfX9iRfRUYE-LRwT4dZ9v63S2XmJ2HnEpQfBWIBzoA0mDn3pj2XcZ097BQlIxXXFRI7fgtdOnxLzzCueEuwgyXnB2tKkJJVvKGWtUY8vMiOJ0S0PSFvnOKL9p4MkBSsb_vYXcHIEnw?type=png)](https://mermaid.live/edit#pako:eNqFlVFvmzAQx7_KyU-JSqxBkpXykIdpmlZp2qb2YS9IkQOmtQY2MyZSVfW776gJ2EC2PKBg_j_f3f8O80oylXOSkIb_abnM-GfBnjSrUgn4Y5lRGu7lWf3m2i7VTBuRiZpJA19VY-arv1hTgV3uyc3hcNNpEwgpCHn-URuhZGM13QMU2OcRBUwCM1pZ2Ro2h5741BYecNPFSWBL4Sjew6wG3Tcu11bbaVBrpTsKrCxVxgxHUcG1TyyE6vE--T2GOhaiLI-nCf7T6PVCNR8pZKp-GXUBXEOQ2dgsFxK_pZBzxxXEnWwtoHlmQD-dVuFdGEAUbfGy3_chvivDQZ25BrthTOHRYMtsr4pWZl1DIEN7LKBqA6KApj1Zb-3qLLGYhkPDUHs_9my4G7yc-xnTqHN0CLJyIfQn8HYZmzrzOabbwSEX8TMZ4dGOC797_7fkxEKw_aTmB944kfDOq3nS3pjiVLhKrOuqu7eziZ2Q_4g79TqeTq-LjdO4UO9dP8YuEMB_8OlAL83Oh0nX7GZ-TZbmMl-eZMzti5Cieb46ygPqRe9ijyfNpYPCt9EnwgnhvIUPTe1DXfX9iRfRUYE-LRwT4dZ9v63S2XmJ2HnEpQfBWIBzoA0mDn3pj2XcZ097BQlIxXXFRI7fgtdOnxLzzCueEuwgyXnB2tKkJJVvKGWtUY8vMiOJ0S0PSFvnOKL9p4MkBSsb_vYXcHIEnw)

#### Glossary
- `invOpt`: Invocation options passed by Invoker which are the function name of the invoked Wasm function and its arguments.
- `invOptBuf`: Encoded invocation options buffer
- `invOptBufLen`: Length of the invocation options buffer
- `invOptBufPtr`: Reference pointer to the invocation options buffer allocated in the Wasm shared memory
- `subInvOpt`: Subinvocation options needed to subinvoke a Host function. They are the function name of the invoked Host function and its arguments.
- `subInvOptBuf`: Encoded subinvocation options buffer
- `subInvOptBufPtr`: Reference pointer to the Subinvocation options buffer allocated in the Wasm shared memory
- `subInvOptBufLen`: Length of the subinvocation options buffer
- `subInvResBufLen`: Length of the subinvocation result buffer
- `invRes`: Result of the invoked Wasm function.
- `invResBuf`: Encoded invocation result
- `invResBufPtr`: Reference pointer to the invocation result buffer allocated in the Wasm shared memory
- `invResBufLen`: Length of the invocation result buffer
- `invRspBuf`: Invocation response buffer which contains the pointer and length of the result buffer.
- `invRspBufPtr`: Reference pointer to the invocation response buffer
- `invRspBufLen`: Length of the invocation response buffer

#### Summary

1. Invoker calls the host with invocation options(`invOpt`) which consists of the name of a Wasm module function and its arguments.
2. The host encodes invocation options (`invOpt`) into an invocation options buffer(`invOptBuf`).
3. The host calls the `_invoke` Wasm export with the length of the invocation options buffer(`invOptBufLen`)
4. The Wasm module allocates a new invocation options buffer(`invOptBuf`) in the shared memory. This will be used to store the original invocation options buffer(`invOptbuf`), that exists inside the host, for use in the Wasm module.
5. The Wasm module calls `__fill_buffer` with the pointer to the newly allocated invocation options buffer(`invOptBufPtr`).
6. The host copies the invocation options buffer(`invOptBuf`) to the given pointer in the shared Wasm memory.
7. After the call to `__fill_buffer`, the Wasm module reads and decodes the invocation options buffer (`invOptBuf`).
8. The Wasm module calls the requested Wasm module function with the given arguments.
- If the invoked Wasm module function needs to subinvoke a function defined in the host, It performs the subinvocation with following steps
1. The Wasm module encodes the subinvocation options (`subInvOpt`) into a subinvocation options buffer (`subInvOptBuf`).
2. The Wasm module calls the imported `__subinvoke` function with the pointer and length to this buffer.
3. The host reads and decodes the subinvocation options buffer(`subInvOptBuf`) from shared memory.
4. The host calls the requested host function with the given arguments from the decoded subinvocation options(`subInvOpt`)
5. Host will get the result of the subinvoked function(`subInvRes`) and encode it into a buffer(`subInvResBuf`)
6. Host returns the length of the subinvocation result buffer(`subInvResBufLen`) as a result of the `__subinvoke` function.
7. The Wasm module allocates a new subinvocation result buffer(`subInvResBuf`) in the shared memory with the given length(`subInvResBuflen`). This will be used to store the original subinvocation result buffer(`subInvResBuf`), that exists inside the host, for use in the Wasm module.
8. The Wasm module calls the `__fill_buffer` with the pointer to the newly allocated subinvocation result buffer(`subInvResBuf`)
9. The host copies the subinvocation result buffer(`subInvResBuf`) to the given pointer in shared memory.
10. After the call to `__fill_buffer`, the Wasm module reads and decodes the subinvocation result buffer(`subInvResBuf`) from shared memory and can use it wherever needed.
9. After the Wasm module finish execution of the invoked Wasm function, it receives the invocation result(`invRes`) as return value of the invoked function.
10. The Wasm module encodes the invocation result(`invRes`) into an invocation result buffer(`invResBuf`)
11. The Wasm module then allocates a new invocation response buffer(`invRspBuf`) and stores the pointer(`invResPtr`) and length(`invResLen`) of the invocation result buffer inside.
12. The Wasm module returns the pointer of the invocation response buffer(`invRspPtr`) to the Host.
13. The host reads(from shared memory) and decodes the invocation response buffer.
14. The host uses the pointer and length of the invocation result buffer from the response buffer to read and decode the invocation result buffer into the invocation result.
15. The host returns the invocation result to the invoker.