Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion .env.example
Original file line number Diff line number Diff line change
@@ -1,4 +1,12 @@
POSTGRES_PRISMA_URL=postgresql://postgres:1234@localhost
POSTGRES_URL_NON_POOLING=postgresql://postgres:1234@localhost

NEXT_PUBLIC_DEFAULT_CURRENCY_CODE=""
NEXT_PUBLIC_DEFAULT_CURRENCY_CODE=""

# OpenAI (category extraction / receipt scanning). All optional.
# OPENAI_API_KEY=
# Point at a self-hosted / OpenAI-compatible endpoint (defaults to OpenAI):
# OPENAI_BASE_URL=
# Override the models (both default to gpt-5.4-nano):
# OPENAI_MODEL_CATEGORY_EXTRACT=gpt-5.4-nano
# OPENAI_MODEL_RECEIPT_EXTRACT=gpt-5.4-nano
10 changes: 8 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,19 +100,21 @@ S3_UPLOAD_ENDPOINT=http://localhost:9000

### Create expense from receipt

You can offer users to create expense by uploading a receipt. This feature relies on [OpenAI GPT-4 with Vision](https://platform.openai.com/docs/guides/vision) and a public S3 storage endpoint.
You can offer users to create expense by uploading a receipt. This feature relies on a vision-capable [OpenAI](https://platform.openai.com/docs/guides/vision) model and a public S3 storage endpoint.

To enable the feature:

- You must enable expense documents feature as well (see section above). That might change in the future, but for now we need to store images to make receipt scanning work.
- Subscribe to OpenAI API and get access to GPT 4 with Vision (you might need to buy credits in advance).
- Subscribe to OpenAI API and get access to a vision-capable model (you might need to buy credits in advance).
- Update your environment variables with appropriate values:

```.env
NEXT_PUBLIC_ENABLE_RECEIPT_EXTRACT=true
OPENAI_API_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXX
```

The model defaults to `gpt-5.4-nano`. You can override it with the optional `OPENAI_MODEL_RECEIPT_EXTRACT` variable (e.g. `gpt-5.4-mini` for higher OCR accuracy on poor-quality photos).

### Deduce category from title

You can offer users to automatically deduce the expense category from the title. Since this feature relies on a OpenAI subscription, follow the signup instructions above and configure the following environment variables:
Expand All @@ -122,6 +124,10 @@ NEXT_PUBLIC_ENABLE_CATEGORY_EXTRACT=true
OPENAI_API_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXX
```

The model defaults to `gpt-5.4-nano`. You can override it with the optional `OPENAI_MODEL_CATEGORY_EXTRACT` variable.

To use a self-hosted or OpenAI-compatible provider for either feature, set the optional `OPENAI_BASE_URL` variable (when unset, the official OpenAI API is used).

## License

MIT, see [LICENSE](./LICENSE).
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,17 @@ import { formatCategoryForAIPrompt } from '@/lib/utils'
import OpenAI from 'openai'
import { ChatCompletionCreateParamsNonStreaming } from 'openai/resources/index.mjs'

const openai = new OpenAI({ apiKey: env.OPENAI_API_KEY })
const openai = new OpenAI({
apiKey: env.OPENAI_API_KEY,
baseURL: env.OPENAI_BASE_URL,
})

export async function extractExpenseInformationFromImage(imageUrl: string) {
'use server'
const categories = await getCategories()

const body: ChatCompletionCreateParamsNonStreaming = {
model: 'gpt-5-nano',
model: env.OPENAI_MODEL_RECEIPT_EXTRACT,
messages: [
{
role: 'user',
Expand Down
7 changes: 5 additions & 2 deletions src/components/expense-form-actions.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,10 @@ import { formatCategoryForAIPrompt } from '@/lib/utils'
import OpenAI from 'openai'
import { ChatCompletionCreateParamsNonStreaming } from 'openai/resources/index.mjs'

const openai = new OpenAI({ apiKey: env.OPENAI_API_KEY })
const openai = new OpenAI({
apiKey: env.OPENAI_API_KEY,
baseURL: env.OPENAI_BASE_URL,
})

/** Limit of characters to be evaluated. May help avoiding abuse when using AI. */
const limit = 40 // ~10 tokens
Expand All @@ -19,7 +22,7 @@ export async function extractCategoryFromTitle(description: string) {
const categories = await getCategories()

const body: ChatCompletionCreateParamsNonStreaming = {
model: 'gpt-3.5-turbo',
model: env.OPENAI_MODEL_CATEGORY_EXTRACT,
temperature: 0.1, // try to be highly deterministic so that each distinct title may lead to the same category every time
max_tokens: 1, // category ids are unlikely to go beyond ~4 digits so limit possible abuse
messages: [
Expand Down
6 changes: 6 additions & 0 deletions src/lib/env.ts
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,12 @@ const envSchema = z
z.boolean().default(false),
),
OPENAI_API_KEY: z.string().optional(),
OPENAI_BASE_URL: z.string().url().optional(),
OPENAI_MODEL_CATEGORY_EXTRACT: z
.string()
.optional()
.default('gpt-5.4-nano'),
OPENAI_MODEL_RECEIPT_EXTRACT: z.string().optional().default('gpt-5.4-nano'),
})
.superRefine((env, ctx) => {
if (
Expand Down