Skip to content

Patch @nicnocquee/dataqueue to fix processor stalling under groupConcurrency #786

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kevinhermawan merged 4 commits into from
Jun 12, 2026
Merged

Patch @nicnocquee/dataqueue to fix processor stalling under groupConcurrency #786

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
294610a
Patch @nicnocquee/dataqueue to refill concurrency slots continuously
kevin-hyperjump Jun 12, 2026
ec56b94
Copy patches dir into Docker build context for pnpm patch support
kevin-hyperjump Jun 12, 2026
49cfb34
Stream each collected URL to the Agent Data API as its fetch resolves
kevin-hyperjump Jun 12, 2026
ac7fe26
Skip patches/ dir in cursor-pr-review kebab-case check
kevin-hyperjump Jun 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions apps/hermes/dashboard/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,16 +17,17 @@

COPY packages ./packages
COPY apps ./apps
COPY patches ./patches

RUN pnpm install --frozen-lockfile

ENV ORCHESTRATION_DATABASE_URL="postgresql://localhost:5432/dummy"
ENV TEMP_ADMIN_USERNAME="dummy"
ENV TEMP_ADMIN_PASSWORD="dummy"

Check warning on line 26 in apps/hermes/dashboard/Dockerfile

View workflow job for this annotation

GitHub Actions / Build hermes

Sensitive data should not be used in the ARG or ENV commands 

SecretsUsedInArgOrEnv: Do not use ARG or ENV instructions for sensitive data (ENV "TEMP_ADMIN_PASSWORD")
More info: https://docs.docker.com/go/dockerfile/rule/secrets-used-in-arg-or-env/
ENV HERMES_INTERNAL_API_KEY="dummy-internal-key"

Check warning on line 27 in apps/hermes/dashboard/Dockerfile

View workflow job for this annotation

GitHub Actions / Build hermes

Sensitive data should not be used in the ARG or ENV commands 

SecretsUsedInArgOrEnv: Do not use ARG or ENV instructions for sensitive data (ENV "HERMES_INTERNAL_API_KEY")
More info: https://docs.docker.com/go/dockerfile/rule/secrets-used-in-arg-or-env/
ENV HERMES_MCP_API_KEY_PEPPER="dummy-mcp-api-key-pepper"

Check warning on line 28 in apps/hermes/dashboard/Dockerfile

View workflow job for this annotation

GitHub Actions / Build hermes

Sensitive data should not be used in the ARG or ENV commands 

SecretsUsedInArgOrEnv: Do not use ARG or ENV instructions for sensitive data (ENV "HERMES_MCP_API_KEY_PEPPER")
More info: https://docs.docker.com/go/dockerfile/rule/secrets-used-in-arg-or-env/
ENV AGENT_AUTH_API_URL="http://localhost:8080"

Check warning on line 29 in apps/hermes/dashboard/Dockerfile

View workflow job for this annotation

GitHub Actions / Build hermes

Sensitive data should not be used in the ARG or ENV commands 

SecretsUsedInArgOrEnv: Do not use ARG or ENV instructions for sensitive data (ENV "AGENT_AUTH_API_URL")
More info: https://docs.docker.com/go/dockerfile/rule/secrets-used-in-arg-or-env/
ENV AGENT_AUTH_JWT_SECRET="dummy-jwt-secret-for-build"

Check warning on line 30 in apps/hermes/dashboard/Dockerfile

View workflow job for this annotation

GitHub Actions / Build hermes

Sensitive data should not be used in the ARG or ENV commands 

SecretsUsedInArgOrEnv: Do not use ARG or ENV instructions for sensitive data (ENV "AGENT_AUTH_JWT_SECRET")
More info: https://docs.docker.com/go/dockerfile/rule/secrets-used-in-arg-or-env/
RUN rm -f apps/hermes/dashboard/.env apps/hermes/dashboard/.env.local && \
touch apps/hermes/dashboard/.env apps/hermes/dashboard/.env.local

Expand Down
15 changes: 14 additions & 1 deletion apps/mediapulse/agents/data-collection/src/index.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,20 @@ describe("data-collection agent (HTTP)", () => {
beforeEach(() => {
vi.clearAllMocks();
performWebSearchMock.mockResolvedValue(defaultSearchSuccess);
performWebFetchMock.mockResolvedValue(defaultFetchSuccess);
// performWebFetch now streams each outcome through the onOutcome hook (so
// run.ts persists per URL); mirror that contract instead of just resolving.
performWebFetchMock.mockImplementation(
async (
_searchResults: unknown,
deps: { onOutcome?: (outcome: unknown) => unknown },
) => {
for (const outcome of defaultFetchSuccess) {
await deps?.onOutcome?.(outcome);
}

return defaultFetchSuccess;
},
);
existingUrlsCreateMock.mockResolvedValue({
existingUrls: [],
hostCounts: {},
Expand Down
Loading
Loading