🚗 Cars_DQLN — Deep Q-Learning on a custom 2D racing environment

This repository is a hands-on Deep Reinforcement Learning project combining a playable 2D racing simulator with a DQN agent trained on a custom Gym-like environment.

About this README
The original documentation was written as a very detailed, tutorial-style French README.
This English README is a condensed, “project-first” version.
If you want the full deep dive (RL foundations + why each design choice matters), see THEORY.md.
The original French long-form version remains available as README_FR_LEGACY.md.

---

What it is

Cars_DQLN is a 2D racing simulator where an agent learns to drive to a finish line using Deep Q-Networks (DQN).
The goal is to bridge theory and practice: you can play manually, test a trained agent, or train a new agent on different tracks through an interactive menu.

---

How it works (high-level)

• The car observes the world through 7 ray-based distance sensors (a simple Lidar-like perception).
• A custom environment RacingEnv exposes the classic RL loop: step(action) → observation, reward, done.
• Observations are the sensor distances; actions control acceleration + steering.
• Rewards encourage progress / staying on track, and penalize crashes / leaving the track.
• Training uses Stable-Baselines3 (DQN):
  • train_agent.py trains and saves a model
  • test_agent.py reloads a model and runs inference

For the detailed RL foundations (Q-learning → MDPs → Bellman → DQN + replay buffer + target network) and the exact reasoning behind design choices, see THEORY.md.

---

Run it

1) Install dependencies

pip install -r requirements.txt

2) Launch the main menu

python main.py

From the menu you can typically:

• Play manually
• TEST (run a trained agent)
• TRAIN (start a new training session)

Notes: Use arrow keys to navigate the menu. You can toggle sensor visualization (e.g., key v, depending on the menu options / version).

Docs & references

Project structure / code design

• Class & dependency diagrams (auto-generated):

  • Class diagram:
  • Dependency diagram:

• Full Generated documentation (html):

Deep dive / theory

THEORY.md — RL foundations + Q-learning → DQN, with explanations and visuals & explained programm. README_FR_LEGACY.md — original French long-form documentation (historical / complete)

External references

• Deep Reinforcement Learning Hands-On, Maxim Lapan, Third Edition, Packt — A practical and easy-to-follow guide to RL, from Q-Learning and DQNs to PPO and RLHF.
• Pygame Documentation 1 : https://he-arc.github.io/livre-python/pygame/index.html
• Pygame Documentation 2 : https://www.pygame.org/news
• OpenAI Spinning Up (Reinforcement Learning guide) : https://spinningup.openai.com/en/latest/
• Gym Documentation : https://www.gymlibrary.dev/
• Stable-Baselines3 (Reinforcement Learning Library) : https://github.com/DLR-RM/stable-baselines3
• YouTube Videos :
  • Google's DeepMind AI Just Taught Itself To Walk
  • AI Learns to Speedrun Mario
  • MarI/O - Machine Learning for Video Games
  • DeepMind Atari DQN era (context)

Credits

Project built with contributions from Kérian Dorey (game setup + oral presentation) and Julien Escot (agent, documentation, additional features).