Memory leak in neo4j-upload

[Schmoho]

Describe the bug
Running the neo4j-upload with kg-microbe-biomedical-function (~30GB edges + 15GB nodes) uses up all memory in the python process and eventually is killed by my OS OOM killer on a 64GB RAM machine.

To Reproduce
kgx neo4j-upload -i tsv --uri bolt://localhost:7687 -u neo4j -p demo kg-microbe-biomedical-function/merged-kg_nodes.tsv kg-microbe-biomedical-function/merged-kg_edges.tsv

on a machine with 64GB memory, running neo4j:2025.10.1 in a container.

Expected behavior
Don't use that much memory.

[Schmoho]

I increased the memory to 128GB and it still manages to consume it all. I have observed the whole process now and I want to stress two points:

1. The memory is consumed by the python process running kgx. Not by the database, or anything else.
2. The data does reach Neo4J, i.e. nodes and edges are actually being created.

If someone can give me a hint where the problem might lie it would be very much appreciated.

[caufieldjh]

Thanks for your issue, @Schmoho - this is good to know.
What version of kgx are you using (newest is v2.6.0)?

You may want to try the same command with the --stream flag to reduce memory peaks.

[Schmoho]

I am on version 2.6.0

I tried --stream but the behaviour was exactly the same.

It turns out that I also cannot actually reproduce that the data really reaches Neo4J. That is, it does not seem that anything actually is loaded by the script.

The main problem I see here is that the script does not produce any log output whatsoever, regardless of whether it can successfully authenticate against Neo4J or anything of the sort.