Set default batch size to 16

[anasdorbani]

No description provided.

[Copilot]

Pull request overview

This PR standardizes the model default batch_size to 16 across the extension by introducing a shared constant and using it when model configuration doesn’t specify a batch size. It also simplifies llm_rerank’s batching logic to always clamp the batch size to the number of tuples, and updates unit tests accordingly.

Changes:

• Introduce DEFAULT_BATCH_SIZE = 16 as a shared constant in the model manager public header.
• Use DEFAULT_BATCH_SIZE when a model’s batch_size is not provided via config/DB args.
• Simplify llm_rerank sliding-window batching by clamping batch_size to num_tuples, and update unit tests to match the new default.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File	Description
test/unit/model_manager/model_manager_test.cpp	Updates the minimal initialization test to assert the new default batch size constant.
src/model_manager/model.cpp	Replaces the hardcoded default batch size with DEFAULT_BATCH_SIZE when missing from config/DB args.
src/include/flock/model_manager/repository.hpp	Adds DEFAULT_BATCH_SIZE constant (16) for shared use across the codebase.
src/functions/aggregate/llm_rerank/implementation.cpp	Removes special-casing tied to the old default and always clamps batch size to tuple count.

[queryproc]

We should make sure that we take the min of 2: (user_batch, default_batch = 16). Using this batch size, we then expand the data section of the prompt while not going beyond the max input context window size.

[queryproc]

Alternatively, if the user provided a batch_size, it overrides the default size and then we expand till we reach the max input context window size.