-
Notifications
You must be signed in to change notification settings - Fork 68
Adds Batching Proposal #307
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
ac935e4
ba03118
e7b37d9
f159e73
6c995bd
c8755c8
654165b
c9c4253
d1212a7
16be80a
57cb6e7
a4ee63e
cfad269
fb404ac
313eb3d
1510e9c
d1e31b2
0780fe8
dd2515e
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,183 @@ | ||||||||||||
| ## B. Appendix: Batching | ||||||||||||
|
|
||||||||||||
| This appendix defines optional batching extensions for GraphQL over HTTP. It | ||||||||||||
| covers both: | ||||||||||||
|
|
||||||||||||
| - **Variable batching**: one request object executes one operation multiple | ||||||||||||
| times by using a list of variable maps. | ||||||||||||
| - **Request batching**: one HTTP request contains a list of request objects. | ||||||||||||
|
|
||||||||||||
| Variable batching and request batching MAY be used independently or together. | ||||||||||||
|
|
||||||||||||
| Note: GraphQL subscriptions are beyond the scope of this specification, so this | ||||||||||||
| appendix defines batching semantics for query and mutation operations only. | ||||||||||||
|
|
||||||||||||
| ### Motivation | ||||||||||||
|
|
||||||||||||
| Field-level batching can solve some cases, but it does not cover all shapes of | ||||||||||||
| variable-dependent queries. For example: | ||||||||||||
|
|
||||||||||||
| ```graphql example | ||||||||||||
| query ($vendorId: ID!, $productId: ID!) { | ||||||||||||
| vendor(id: $vendorId) { | ||||||||||||
| id | ||||||||||||
| name | ||||||||||||
| stock(productId: $productId) { | ||||||||||||
| count | ||||||||||||
| nextDeliveryDue | ||||||||||||
| product { | ||||||||||||
| id | ||||||||||||
| name | ||||||||||||
| } | ||||||||||||
| } | ||||||||||||
| } | ||||||||||||
| } | ||||||||||||
| ``` | ||||||||||||
|
|
||||||||||||
| Supplying the document to execute alongside a _list_ of {variableValues} maps | ||||||||||||
| allows this operation to execute multiple times in a single HTTP request, each | ||||||||||||
| time with a different {variableValues} map. | ||||||||||||
|
|
||||||||||||
| ### Request Shapes | ||||||||||||
|
|
||||||||||||
| A batching request is sent as a POST request with a JSON-encoded body. | ||||||||||||
|
|
||||||||||||
| To add support for variable batching, the {variables} _request parameter_ is extended such that it may now also accept a list of maps. | ||||||||||||
|
|
||||||||||||
| To add support for request batching, the JSON encoding of a _GraphQL-over-HTTP | ||||||||||||
| request_ is extended such that the body of the request may also be a list of _GraphQL-over-HTTP request_ maps. | ||||||||||||
|
|
||||||||||||
| If a request object's `{variables}` value is a list, the server MUST execute | ||||||||||||
| that request once per variables entry. | ||||||||||||
|
|
||||||||||||
| If the request body is a list, each entry in that list MUST be processed as an | ||||||||||||
| independent _GraphQL-over-HTTP request_. | ||||||||||||
|
Comment on lines
+50
to
+54
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We need to encourage them to discard requests where the total number of requests to execute is over some threshold. But also, I think this is more part of execution behavior than request shape.
Suggested change
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If the server is allowed to discard requests then this must be specified in some way. Like it cannot simply discard requests in the middle without any notice. Otherwise we need another marker to pin which requests have to be reissued. I a server like executes 20 requests and then says limit reached a stop now that could be handled... although to be honest I would find it better if the server would simply reject it all and say i can only process 20 requests at a time so i reject your batch. This ties btw back in with capabilities. In the gateway case the gateway could inspect the capabilities and send exactly the correct amount of requests in a batch.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, I think all or none - it should decide up front if it's attempting them or not. That said, mutations would be serialized and may hit timeouts, and the side effects might not be able to be undone, so... 😬 I guess just execute them all and the later ones produce errors... ... We really need to add some standard error codes to GraphQL errors. |
||||||||||||
|
|
||||||||||||
| ### Execution Semantics | ||||||||||||
|
|
||||||||||||
| When a request object's `{variables}` value is a list, each variables map | ||||||||||||
| represents a distinct execution of that request object. | ||||||||||||
| The server MAY execute these executions in any order, including concurrently, | ||||||||||||
| and the client MUST NOT assume serial execution or any ordering based on | ||||||||||||
| variables list position. | ||||||||||||
|
michaelstaib marked this conversation as resolved.
|
||||||||||||
|
|
||||||||||||
| If the request body is a list of request objects, the server MAY process those | ||||||||||||
| request objects in any order, including concurrently, and the client MUST NOT | ||||||||||||
| assume ordering based on request list position. | ||||||||||||
|
|
||||||||||||
| Within each individual execution, GraphQL execution semantics continue to apply | ||||||||||||
| unchanged. | ||||||||||||
|
|
||||||||||||
| Note: Implementations often use batching mechanisms such as DataLoaders or batch | ||||||||||||
| resolvers. A server may improve efficiency by sharing such batching mechanisms | ||||||||||||
| across all executions produced from one batching request, rather than isolating | ||||||||||||
| each execution. | ||||||||||||
|
|
||||||||||||
| ### Accept | ||||||||||||
|
|
||||||||||||
| For batching responses, a client SHOULD include `application/jsonl` in the | ||||||||||||
| `Accept` header. | ||||||||||||
|
|
||||||||||||
| ### Response | ||||||||||||
|
|
||||||||||||
| A server implementing this appendix MUST support `application/jsonl` responses | ||||||||||||
| for batching requests. | ||||||||||||
|
|
||||||||||||
| If `application/jsonl` is acceptable to the client, the server SHOULD respond | ||||||||||||
| using `application/jsonl`. | ||||||||||||
|
|
||||||||||||
| A batching response is a list of _GraphQL responses_. When encoded as | ||||||||||||
| `application/jsonl`, each list entry MUST be encoded as one JSON object per | ||||||||||||
| line. | ||||||||||||
|
|
||||||||||||
| In addition to the standard fields of a _GraphQL response_, each batching | ||||||||||||
| response entry MUST include index fields as follows: | ||||||||||||
|
|
||||||||||||
| - `requestIndex` (integer) is REQUIRED when the request body was a list of | ||||||||||||
| request objects. The value MUST be the 0-based index of the request object in | ||||||||||||
| that list. | ||||||||||||
| - `variableIndex` (integer) is REQUIRED when the corresponding request object's | ||||||||||||
| `{variables}` value was a list. The value MUST be the 0-based index of that | ||||||||||||
| variables map. | ||||||||||||
| - If both batching modes apply, both `requestIndex` and `variableIndex` are | ||||||||||||
| REQUIRED. | ||||||||||||
|
|
||||||||||||
| The server MAY return response entries in any order. These index fields allow | ||||||||||||
| clients to correlate each entry with the corresponding request object and | ||||||||||||
| variables entry. | ||||||||||||
|
|
||||||||||||
| If a client needs to process results in request-list order or variables-list | ||||||||||||
| order, it MUST reorder entries using `requestIndex` and `variableIndex`. | ||||||||||||
|
|
||||||||||||
| When using `application/jsonl`, the server MAY deliver each response entry as | ||||||||||||
| soon as it becomes available. | ||||||||||||
|
|
||||||||||||
| #### Variable Batching Example | ||||||||||||
|
|
||||||||||||
| ```headers example | ||||||||||||
| Content-Type: application/json | ||||||||||||
| Accept: application/jsonl | ||||||||||||
| ``` | ||||||||||||
|
|
||||||||||||
| ```json example | ||||||||||||
| { | ||||||||||||
| "query": "query getFoo($a: Int!) { foo(a: $a) }", | ||||||||||||
| "variables": [{ "a": 1 }, { "a": 2 }] | ||||||||||||
| } | ||||||||||||
| ``` | ||||||||||||
|
|
||||||||||||
| ```jsonl example | ||||||||||||
| {"variableIndex":1,"data":{"foo":2}} | ||||||||||||
| {"variableIndex":0,"data":{"foo":1}} | ||||||||||||
| ``` | ||||||||||||
|
|
||||||||||||
| #### Request Batching Example | ||||||||||||
|
|
||||||||||||
| ```headers example | ||||||||||||
| Content-Type: application/json | ||||||||||||
| Accept: application/jsonl | ||||||||||||
| ``` | ||||||||||||
|
|
||||||||||||
| ```json example | ||||||||||||
| [ | ||||||||||||
| { | ||||||||||||
| "query": "query getFoo($a: Int!) { foo(a: $a) }", | ||||||||||||
| "variables": { "a": 1 } | ||||||||||||
| }, | ||||||||||||
| { | ||||||||||||
| "query": "query getBar($b: Int!) { bar(b: $b) }", | ||||||||||||
| "variables": { "b": 1 } | ||||||||||||
| } | ||||||||||||
| ] | ||||||||||||
| ``` | ||||||||||||
|
|
||||||||||||
| ```jsonl example | ||||||||||||
| {"requestIndex":1,"data":{"bar":1}} | ||||||||||||
| {"requestIndex":0,"data":{"foo":1}} | ||||||||||||
| ``` | ||||||||||||
|
|
||||||||||||
| #### Combined Request + Variable Batching Example | ||||||||||||
|
|
||||||||||||
| ```headers example | ||||||||||||
| Content-Type: application/json | ||||||||||||
| Accept: application/jsonl | ||||||||||||
| ``` | ||||||||||||
|
|
||||||||||||
| ```json example | ||||||||||||
| [ | ||||||||||||
| { | ||||||||||||
| "query": "query getFoo($a: Int!) { foo(a: $a) }", | ||||||||||||
| "variables": [{ "a": 1 }, { "a": 2 }] | ||||||||||||
| }, | ||||||||||||
| { | ||||||||||||
| "query": "query getBar($b: Int!) { bar(b: $b) }", | ||||||||||||
| "variables": { "b": 1 } | ||||||||||||
| } | ||||||||||||
| ] | ||||||||||||
| ``` | ||||||||||||
|
|
||||||||||||
| ```jsonl example | ||||||||||||
| {"requestIndex":0,"variableIndex":1,"data":{"foo":2}} | ||||||||||||
| {"requestIndex":1,"data":{"bar":1}} | ||||||||||||
| {"requestIndex":0,"variableIndex":0,"data":{"foo":1}} | ||||||||||||
| ``` | ||||||||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -432,6 +432,15 @@ A server must comply with | |
| The body of the server's response MUST follow the requirements for a | ||
| [GraphQL response](#sec-Response), encoded directly in the chosen media type. | ||
|
|
||
| A server MUST NOT include additional top-level entries in a _GraphQL response_ | ||
| unless those entries are explicitly defined by one of: | ||
|
|
||
| 1. The GraphQL specification. | ||
| 2. This specification, including its appendices. | ||
| 3. Another specification or appendix published under the `graphql.org` domain. | ||
|
Comment on lines
+438
to
+440
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Item 3 implies GAPs can extend it. I don't think that's wise - GAPs should use
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. My thinking was exactly that there might be transport specific spec bound to something like the composite schema spec. But I do not have anything in mind yet, just thought this gives flexibility. But i maybe not needed. I was not thinking about GAPs at the time.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Let's remove it and add it back when needed. |
||
|
|
||
| Clients MUST ignore any top-level response entries they do not understand. | ||
|
|
||
| A server MUST indicate the media type of the response with a `Content-Type` | ||
| header, and SHOULD indicate the encoding (e.g. | ||
| `application/graphql-response+json; charset=utf-8`). | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"field-level batching" is not well defined.