Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
183 changes: 183 additions & 0 deletions spec/Appendix B -- Batching.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,183 @@
## B. Appendix: Batching

This appendix defines optional batching extensions for GraphQL over HTTP. It
covers both:

- **Variable batching**: one request object executes one operation multiple
times by using a list of variable maps.
- **Request batching**: one HTTP request contains a list of request objects.

Variable batching and request batching MAY be used independently or together.

Note: GraphQL subscriptions are beyond the scope of this specification, so this
appendix defines batching semantics for query and mutation operations only.

### Motivation

Field-level batching can solve some cases, but it does not cover all shapes of
variable-dependent queries. For example:
Comment on lines +17 to +18

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"field-level batching" is not well defined.


```graphql example
query ($vendorId: ID!, $productId: ID!) {
vendor(id: $vendorId) {
id
name
stock(productId: $productId) {
count
nextDeliveryDue
product {
id
name
}
}
}
}
```

Supplying the document to execute alongside a _list_ of {variableValues} maps
allows this operation to execute multiple times in a single HTTP request, each
time with a different {variableValues} map.

### Request Shapes

A batching request is sent as a POST request with a JSON-encoded body.

To add support for variable batching, the {variables} _request parameter_ is extended such that it may now also accept a list of maps.

To add support for request batching, the JSON encoding of a _GraphQL-over-HTTP
request_ is extended such that the body of the request may also be a list of _GraphQL-over-HTTP request_ maps.

If a request object's `{variables}` value is a list, the server MUST execute
that request once per variables entry.

If the request body is a list, each entry in that list MUST be processed as an
independent _GraphQL-over-HTTP request_.
Comment on lines +50 to +54

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to encourage them to discard requests where the total number of requests to execute is over some threshold. But also, I think this is more part of execution behavior than request shape.

Suggested change
If a request object's `{variables}` value is a list, the server MUST execute
that request once per variables entry.
If the request body is a list, each entry in that list MUST be processed as an
independent _GraphQL-over-HTTP request_.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the server is allowed to discard requests then this must be specified in some way. Like it cannot simply discard requests in the middle without any notice. Otherwise we need another marker to pin which requests have to be reissued. I a server like executes 20 requests and then says limit reached a stop now that could be handled... although to be honest I would find it better if the server would simply reject it all and say i can only process 20 requests at a time so i reject your batch.

This ties btw back in with capabilities. In the gateway case the gateway could inspect the capabilities and send exactly the correct amount of requests in a batch.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think all or none - it should decide up front if it's attempting them or not.

That said, mutations would be serialized and may hit timeouts, and the side effects might not be able to be undone, so... 😬 I guess just execute them all and the later ones produce errors...

... We really need to add some standard error codes to GraphQL errors.


### Execution Semantics

When a request object's `{variables}` value is a list, each variables map
represents a distinct execution of that request object.
The server MAY execute these executions in any order, including concurrently,
and the client MUST NOT assume serial execution or any ordering based on
variables list position.
Comment thread
michaelstaib marked this conversation as resolved.

If the request body is a list of request objects, the server MAY process those
request objects in any order, including concurrently, and the client MUST NOT
assume ordering based on request list position.

Within each individual execution, GraphQL execution semantics continue to apply
unchanged.

Note: Implementations often use batching mechanisms such as DataLoaders or batch
resolvers. A server may improve efficiency by sharing such batching mechanisms
across all executions produced from one batching request, rather than isolating
each execution.

### Accept

For batching responses, a client SHOULD include `application/jsonl` in the
`Accept` header.

### Response

A server implementing this appendix MUST support `application/jsonl` responses
for batching requests.

If `application/jsonl` is acceptable to the client, the server SHOULD respond
using `application/jsonl`.

A batching response is a list of _GraphQL responses_. When encoded as
`application/jsonl`, each list entry MUST be encoded as one JSON object per
line.

In addition to the standard fields of a _GraphQL response_, each batching
response entry MUST include index fields as follows:

- `requestIndex` (integer) is REQUIRED when the request body was a list of
request objects. The value MUST be the 0-based index of the request object in
that list.
- `variableIndex` (integer) is REQUIRED when the corresponding request object's
`{variables}` value was a list. The value MUST be the 0-based index of that
variables map.
- If both batching modes apply, both `requestIndex` and `variableIndex` are
REQUIRED.

The server MAY return response entries in any order. These index fields allow
clients to correlate each entry with the corresponding request object and
variables entry.

If a client needs to process results in request-list order or variables-list
order, it MUST reorder entries using `requestIndex` and `variableIndex`.

When using `application/jsonl`, the server MAY deliver each response entry as
soon as it becomes available.

#### Variable Batching Example

```headers example
Content-Type: application/json
Accept: application/jsonl
```

```json example
{
"query": "query getFoo($a: Int!) { foo(a: $a) }",
"variables": [{ "a": 1 }, { "a": 2 }]
}
```

```jsonl example
{"variableIndex":1,"data":{"foo":2}}
{"variableIndex":0,"data":{"foo":1}}
```

#### Request Batching Example

```headers example
Content-Type: application/json
Accept: application/jsonl
```

```json example
[
{
"query": "query getFoo($a: Int!) { foo(a: $a) }",
"variables": { "a": 1 }
},
{
"query": "query getBar($b: Int!) { bar(b: $b) }",
"variables": { "b": 1 }
}
]
```

```jsonl example
{"requestIndex":1,"data":{"bar":1}}
{"requestIndex":0,"data":{"foo":1}}
```

#### Combined Request + Variable Batching Example

```headers example
Content-Type: application/json
Accept: application/jsonl
```

```json example
[
{
"query": "query getFoo($a: Int!) { foo(a: $a) }",
"variables": [{ "a": 1 }, { "a": 2 }]
},
{
"query": "query getBar($b: Int!) { bar(b: $b) }",
"variables": { "b": 1 }
}
]
```

```jsonl example
{"requestIndex":0,"variableIndex":1,"data":{"foo":2}}
{"requestIndex":1,"data":{"bar":1}}
{"requestIndex":0,"variableIndex":0,"data":{"foo":1}}
```
9 changes: 9 additions & 0 deletions spec/GraphQLOverHTTP.md
Original file line number Diff line number Diff line change
Expand Up @@ -432,6 +432,15 @@ A server must comply with
The body of the server's response MUST follow the requirements for a
[GraphQL response](#sec-Response), encoded directly in the chosen media type.

A server MUST NOT include additional top-level entries in a _GraphQL response_
unless those entries are explicitly defined by one of:

1. The GraphQL specification.
2. This specification, including its appendices.
3. Another specification or appendix published under the `graphql.org` domain.
Comment on lines +438 to +440

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Item 3 implies GAPs can extend it. I don't think that's wise - GAPs should use extensions. Is this specifically so the composite schemas spec can extend it? What are you thinking about here?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My thinking was exactly that there might be transport specific spec bound to something like the composite schema spec. But I do not have anything in mind yet, just thought this gives flexibility. But i maybe not needed.

I was not thinking about GAPs at the time.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's remove it and add it back when needed.


Clients MUST ignore any top-level response entries they do not understand.

A server MUST indicate the media type of the response with a `Content-Type`
header, and SHOULD indicate the encoding (e.g.
`application/graphql-response+json; charset=utf-8`).
Expand Down
Loading