-
Notifications
You must be signed in to change notification settings - Fork 11
BP-2407: Pathfinding for OpenGraph #200
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 4 commits
781ee33
5e1e9cd
e6d0c59
fb361cf
9d0413d
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -6,25 +6,30 @@ description: "Description of the OpenGraph JSON Schema" | |
|
|
||
| <img noZoom src="/assets/enterprise-AND-community-edition-pill-tag.svg" alt="Applies to BloodHound Enterprise and CE"/> | ||
|
|
||
| # Schema | ||
| This page explains the JSON payload structure and minimum JSON schema requirements that BloodHound uses to ingest OpenGraph nodes and edges. | ||
|
|
||
| In this section, you will find all the information to create a JSON file that BloodHound can ingest and use to display your Nodes and Edges. | ||
| Terms used on this page: | ||
|
|
||
| The most up-to-date JSON Schema can always be found in our CE repository. Currently, the location of the node and edge schema files in our source code can be found [here](https://github.com/SpecterOps/BloodHound/tree/main/cmd/api/src/services/upload/jsonschema). | ||
| - **Schema-less (generic) data** refers to OpenGraph payloads that follow the minimum node and edge JSON schemas on this page. | ||
| - **Schema-based data** refers to OpenGraph payloads that use an extension-defined schema outside the payload itself. | ||
|
|
||
| # Ingesting Generic Formatted Data | ||
| This page focuses on the JSON requirements for generic data. | ||
|
|
||
| ## File Requirements | ||
| <Tip>You can find the latest node and edge schemas in the BloodHound [source code](https://github.com/SpecterOps/BloodHound/tree/main/cmd/api/src/services/upload/jsonschema) on GitHub.</Tip> | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. RE: tighter definitions/descriptions This is specifically pointing to the JSON schema for data payload files. |
||
|
|
||
| ## Ingesting Schema-less (Generic) Data | ||
|
|
||
| ### File Requirements | ||
|
|
||
| Acceptable formats: `.json`, `.zip` | ||
|
|
||
| You can mix file types in a single upload (e.g. Sharphound + Generic). | ||
|
|
||
| Compressed ZIPs containing multiple file types are supported. | ||
|
|
||
| ## JSON Format | ||
| ### JSON Payload Structure | ||
|
jeff-matthews marked this conversation as resolved.
Outdated
|
||
|
|
||
| The standard BloodHound UI upload screen now accepts files in a generic format. You can continue using it as before. | ||
| The standard BloodHound UI upload screen accepts schema-less (generic) OpenGraph payloads. You can continue using it as before. | ||
|
jeff-matthews marked this conversation as resolved.
Outdated
|
||
|
|
||
| At minimum, your JSON file should have these elements: | ||
|
|
||
|
|
@@ -37,16 +42,22 @@ At minimum, your JSON file should have these elements: | |
| } | ||
| ``` | ||
|
|
||
| The `nodes` and `edges` must conform to our JSON Schema, see details below. The validation of the data occurs at upload time. | ||
| The `nodes` and `edges` must conform to the minimum JSON schemas (see details below). BloodHound validates that the JSON is well-formed and that nodes and edges meet these schema requirements, but it does not enforce additional structure or constraints beyond them. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think this is an area where we can tighten up "schema" referencing. |
||
|
|
||
| When ingest completes, generic data is available through all [Explore](/analyze-data/explore/search) options: | ||
|
jeff-matthews marked this conversation as resolved.
Outdated
|
||
|
|
||
| - **Search** | ||
| - **Pathfinding** | ||
| - **Cypher** | ||
|
|
||
| When ingest completes, the generic data will be available via **Cypher search ONLY**. Generic data is not searchable via the pathfinding feature (yet). | ||
| <Note>In **Pathfinding**, edge filtering options are available for schema-based data only. For generic data, edge filtering options are not available.</Note> | ||
|
jeff-matthews marked this conversation as resolved.
Outdated
|
||
|
|
||
| **Entity Panels**: clicking on a generic node or edge will only render the entity’s property bag. At this time there is no support for defining entity panels for generic entities. | ||
| **Entity Panels**: clicking on a generic node or edge will only render the entity's property bag. At this time there is no support for defining entity panels for generic entities. | ||
|
jeff-matthews marked this conversation as resolved.
Outdated
|
||
|
|
||
|
|
||
| ## Nodes | ||
| ### Nodes | ||
|
|
||
| ### Property Rules | ||
| #### Property Rules | ||
|
|
||
| Properties must be primitive types or arrays of primitive types | ||
|
|
||
|
|
@@ -56,7 +67,7 @@ Arrays must be homogeneous (e.g. all strings or all numbers) | |
|
|
||
| An array of kind labels for the node. The first element is treated as the node's primary kind and is used to determine which icon to display in the graph UI. This primary kind is only used for visual representation and has no semantic significance for data processing. | ||
|
|
||
| ### Node JSON | ||
| #### Node JSON | ||
|
|
||
| The following is the <a href="/assets/opengraph/opengraph-node.json" download>JSON schema</a> that all nodes must conform to. | ||
|
|
||
|
|
@@ -112,15 +123,15 @@ The following is the <a href="/assets/opengraph/opengraph-node.json" download>JS | |
| } | ||
| ``` | ||
|
|
||
| ## Edges | ||
| ### Edges | ||
|
|
||
| Edges names cannot contain dash `-`. It is highly recommended to use Pascal Case and no special characters. | ||
|
|
||
| From [tuple.nl](https://www.tuple.nl/en/knowledge-base/pascal-case): Pascal Case is a naming convention used in programming where compound words are written without spaces, and each word starts with an uppercase letter. It is commonly used for naming variables, functions, classes, and other identifiers in code. Pascal Case helps create descriptive and easily distinguishable names, contributing to the clarity of your code. | ||
|
|
||
| See Neo4j [Naming and Conventions](https://neo4j.com/docs/cypher-manual/current/syntax/naming/) for more details. | ||
|
|
||
| ### Edge JSON | ||
| #### Edge JSON | ||
|
|
||
| The following is the <a href="/assets/opengraph/opengraph-edge.json" download>JSON schema</a> that all edges must conform to. | ||
|
|
||
|
|
@@ -247,7 +258,7 @@ The following is the <a href="/assets/opengraph/opengraph-edge.json" download>JS | |
| } | ||
| ``` | ||
|
|
||
| ### Post-processing | ||
| #### Post-processing | ||
|
|
||
| Post-processing in BloodHound refers to the analysis phase where the system creates certain edges after ingesting data to identify attack paths. | ||
|
jeff-matthews marked this conversation as resolved.
Outdated
|
||
|
|
||
|
|
@@ -338,7 +349,7 @@ See the following example OpenGraph payload that produces the effect: | |
| } | ||
| ``` | ||
|
Comment on lines
322
to
346
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is this actually sufficient to generate the pattern? We seem to be implying that the user would need to create both edges to have the post-processed edge created. But in this example we only create one of two edges.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not sure. That's good feedback. I'll make a note for the rewrite since it's out of scope for this PR. |
||
|
|
||
| ## Optional Metadata Field | ||
| ### Optional Metadata Field | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We should add to this section to specify that adding a source_kind via metadata is what registers that kind as a source_kind in the first place. So if they want to track data via sources, they'll need to upload a data payload with this source_kind explicitly called out at least once. It can even be a blank payload aside from the source_kind, I think (we should double check that). Once they've done that, anytime that kind is used, it's pre-defined as a source_kind and will be treated as such.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That's good feedback. I'll make a note for the rewrite since it's out of scope for this PR. FWIW, the unmerged quickstart #155 attempts to provide more details about this. |
||
|
|
||
| You can optionally include a metadata object at the top level of your JSON payload. This metadata currently supports a single field: | ||
|
jeff-matthews marked this conversation as resolved.
Outdated
|
||
|
|
||
|
|
@@ -360,7 +371,7 @@ Example: | |
|
|
||
| If present, the `source_kind` will be added to the `kinds` list of all nodes in the file during ingest. This feature is optional. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I specified in my earlier comment how source_kind is properly handled: https://github.com/SpecterOps/bloodhound-docs/pull/200/changes#r2855480768 The main aspect that I think we should highlight here is that – once the source_kind has been registered – the main benefit of using the metadata field is that it automatically applies the source_kind to all of the nodes in the payload. This is accurate, but actually undesirable behavior. It should only apply the source_kind to the nodes defined within the payload, but not the ones just referenced within it. This is the source of bug behavior, and the reason we have an FAQ and workaround in place.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That's good feedback. I'll make a note for the rewrite since it's out of scope for this PR. |
||
|
|
||
| ## Minimal Working JSON | ||
| ### Minimal Working JSON | ||
|
jeff-matthews marked this conversation as resolved.
Outdated
|
||
|
|
||
| The following is a minimal example payload that conforms to the node and edge schemas above. You can use this as a starting point to build your own OpenGraph. Copy and paste the following example into a new `.json` file or <a href="/assets/opengraph/opengraph-minimal.json" download>download this example file</a>. | ||
|
|
||
|
|
||
Uh oh!
There was an error while loading. Please reload this page.