Open Source Software Complexity Report

Measures how the cyclomatic complexity of PHP open source software evolved over time.

christopher-hertel.de/oss-complexity-report

Every repository on github.com that is mostly written in PHP can be submitted - a composer.json is not needed, so wordpress/wordpress works just as well as symfony/console. Submitted repositories are grouped by the GitHub account that owns them, and the start page and the overview chart focus on the most starred ones.

Requirements

• PHP 8.4 or newer
• Node 26 (see .nvmrc) & Yarn
• PostgreSQL - the migrations are written for it, docker-compose.yml starts a 15 on port 8432, which is what DATABASE_URL in .env points at
• git, since analysing a repository shells out to it
• The Symfony CLI - every command below is written as symfony console, which runs bin/console with the environment of the local web server

Setup

git clone git@github.com:chr-hertel/oss-complexity-report
cd oss-complexity-report
composer install
yarn install
yarn build
docker-compose up -d
symfony console doctrine:migrations:migrate
symfony serve -d

That is an empty report - it fills up by submitting repositories, either with the form on the start page or on the command line (see below). The fixtures submit a handful to start with.

Assets are bundled by Vite and wired into Twig by symfony/reprise. Run yarn build for a one-off build, or yarn dev to start the Vite dev server with hot module replacement - reprise picks it up automatically.

Checks

bin/check runs everything a pull request runs, plus prettier and yarn audit:

bin/check

Individually, if only one of them is interesting:

symfony php vendor/bin/phpunit                    # add --filter testName tests/Some/FileTest.php for one
symfony php vendor/bin/php-cs-fixer fix           # --dry-run to only report
symfony php vendor/bin/phpstan analyse            # level 8
symfony console lint:yaml config --parse-tags
symfony console lint:twig templates
symfony console lint:container

The tests cover the domain logic that is pure - parsing what people paste, mapping the GitHub API, deciding which releases count, rolling the report up into the trend. Everything that needs a booted kernel is not covered yet.

Background Processing

Analysing a repository clones it and runs phploc over every release, which is far too slow to happen while someone waits for a HTTP response. It is therefore handled by symfony/messenger with the Doctrine transport, and kept up to date by symfony/scheduler:

Message	Does
ScanForNewReleases	fans out into one ScanRepository per submitted repository
ScanRepository	asks github.com for tags and queues an analysis if a release is missing
AnalyseRepository	clones, checks out every new release and measures it
RefreshRepositories	re-reads stars and metadata, which decide the order of the whole report

Only AnalyseRepository is expensive. ScanRepository reads refs with git ls-remote, so the nightly check neither clones anything nor touches a working copy that is being analysed, and the queue only ever fills up with repositories that really did release something.

Two workers are needed - one for the queue, one for the schedule:

symfony console messenger:consume async -vv
symfony console messenger:consume scheduler_default -vv

The async transport may be consumed by several workers, scheduler_default must stay at exactly one - otherwise the nightly run happens more than once. Checking out a tag rewrites a working copy, so an analysis holds a lock per repository and a second worker on the same one waits instead of measuring whatever the first one just checked out. The lock is a flock by default (see LOCK_DSN), which only works if all workers run on the same machine - switch it to postgresql+advisory:// if they ever do not.

Failed messages are kept: messenger:failed:show lists them, messenger:failed:retry puts them back.

Disk usage

Clones in repositories/ are scratch space, not a cache. A working copy is only needed while releases are measured - looking for new ones reads refs from github.com - so an analysis removes it when it is done, and a repository that never releases again never occupies disk. That bounds the disk by what is being analysed instead of by everything ever submitted, at the price of cloning again when a repository does release.

app:repositories:clean removes what predates that: working copies from before this behaviour, from repositories that were renamed, and from workers that were killed mid-analysis. It skips whatever is being analysed right now, so it is safe to run while workers are busy.

Recreate Dataset

# resetting database and caches
symfony console doctrine:database:drop --force
symfony console doctrine:database:create
symfony console doctrine:migrations:migrate
symfony console cache:pool:clear cache.app

# submits a couple of well known repositories to start with - submitting queues them right away
symfony console doctrine:fixtures:load -n

# this clones and analyses them - the long one, run it until the queue is empty
symfony console messenger:consume async -vv

# fix the datasets where git history lies
symfony console app:data:fix -vv

messenger:stats shows what is left to do. Nothing queues the analysis by hand: a submission dispatches it, and app:releases:scan picks up whatever a run left unfinished, since it asks github.com for the releases a repository is still missing.

One repository that will not get through the queue is app:repository:analyse, which measures it right here instead of dispatching it:

symfony console app:repository:analyse moodle/moodle -vv

That is what a worker would have done, only where it can be watched, given a memory limit and stopped - an analysis a worker loses to a memory limit is never acked, so it comes back on the next delivery and fails the same way, while the report keeps showing the repository as queued. --queue dispatches it for a worker instead.

Schema changes

The schema is managed by doctrine/migrations, never by doctrine:schema:update - run doctrine:migrations:diff after changing an entity and doctrine:migrations:migrate to apply what came out of it. A deploy runs the pending migrations before it switches to the new release.

Error Reporting

Errors are reported to Sentry - uncaught exceptions of the web app, of the console commands and of everything the workers run. It is configured by SENTRY_DSN, which is empty everywhere but production: without a DSN the SDK collects nothing and sends nothing, so nothing has to be switched off for local development. Set it in .env.local to try it out, and in the environment of the deployment to turn it on in production.

Two things are deliberately not reported: 404 and 405, which on a public site are what bots produce rather than what is broken, and messages that are going to be retried. A repository another worker is holding throws by design, and github.com failing once is what the retry strategy is for - only what runs out of retries and lands in the failure transport is an incident.

Every deploy writes the revision it puts live into SENTRY_RELEASE, so an error says which release it happened on. Tracing is off: this is error reporting, not performance monitoring.

Submitting Repositories

Repositories are submitted with the form on the start page, or on the command line:

symfony console app:repository:submit wordpress/wordpress https://github.com/symfony/console

Submitting queues the repository for analysis right away, so it shows up as soon as a worker gets to it. Every night the schedule looks for releases that are missing and refreshes the stars, which means there is nothing left to run by hand - app:releases:scan and app:repositories:refresh only trigger the same work earlier.

The form is protected by a stateless CSRF token - a double submit cookie written by the csrf-protection Stimulus controller, so readers of the report never get a session - and submissions are limited to five per quarter of an hour and IP, since each one spends github.com API quota and ends in a clone.

Set GITHUB_TOKEN in .env.local to raise the github.com API rate limit from 60 to 5.000 requests per hour. The token only reads public data, so it does not need any scope.

License

MIT, see LICENSE.