PyMolAI on Open-Source PyMOL

PyMolAI extends open-source PyMOL with an integrated AI assistant panel for molecular workflows. It adds provider-backed reasoning, tool execution, and optional OpenBio API gateway access while keeping normal PyMOL usage unchanged.

The upstream open-source foundation remains this repository: pymol-open-source.

Feature Matrix

Capability	Status
Chat panel in Qt desktop UI	Included
Direct Anthropic API access	Included
OpenRouter-backed model access	Included
Internal PyMOL tool: run_pymol_command	Included
Internal validation tool: capture_viewer_snapshot	Included
OpenBio API gateway tools	Optional (only when OPENBIO_API_KEY is set)
CLI mode fallback (/cli)	Included

Requirements

Python Compatibility

• Base package supports Python >= 3.9.
• Full Claude SDK agent path requires Python >= 3.10 because claude-agent-sdk is only installed on 3.10+.

Runtime Dependencies (AI path)

From pyproject.toml:

• openai>=1.0.0
• keyring>=24
• anthropic>=0.40.0 (Python 3.10+)
• claude-agent-sdk>=0.1.38 (Python 3.10+)

GUI Requirement

PyMolAI chat UI is designed for Qt desktop usage (pmg_qt).

Network/API Requirements

• Anthropic API key for direct access, or OpenRouter API key for multi-provider access. At least one is required.
• Optional OpenBio access for OpenBio tools (default base URL: https://api.openbio.tech).

Installation Quickstarts

macOS (source install with uv)

Step 0 — Clone the repository

git clone https://github.com/ravishar313/PyMolAI
cd PyMolAI

Step 1 — Install system dependencies (Homebrew)

The C++ build requires several native libraries. Install them before building:

brew install netcdf glew glm

libxml2 and freetype are typically already present via Homebrew. If the build still fails, install them too:

brew install libxml2 freetype libpng

Step 2 — Create virtualenv and install

Use uv (recommended):

uv venv .venv
source .venv/bin/activate
make install

Or without Make:

PREFIX_PATH=/opt/homebrew:/opt/homebrew/opt/libxml2:/opt/homebrew/opt/netcdf uv pip install --python .venv/bin/python --reinstall --config-settings="use-msgpackc=no" .

Step 3 — Install PyQt5

PyMOL's Qt GUI layer must use PyQt5. PyQt6 is detected later in the fallback chain but has enum API incompatibilities with this codebase. Install PyQt5 explicitly:

uv pip install --python .venv/bin/python PyQt5

Step 4 — Verify imports

.venv/bin/python -c "import keyring, openai; print('ok: keyring/openai')"
.venv/bin/python -c "import anthropic; print('ok: anthropic')"
.venv/bin/python -c "import claude_agent_sdk; print('ok: claude-agent-sdk')"
.venv/bin/python -c "from PyQt5 import QtWidgets; print('ok: PyQt5')"

If claude_agent_sdk import fails, verify the interpreter is Python 3.10+.

Makefile

A Makefile is provided for common workflows (macOS):

Command	Description
make install	Full rebuild (C++ and Python)
make editable	Editable install (Python changes take effect on next launch without rebuilding)
make run	Launch PyMolAI
make clean	Remove build artifacts

For iterative Python development, run make editable once, then make run after each change. Only run make install when C++ code (layer*, contrib/) changes.

Windows (PowerShell, source install with uv)

git clone https://github.com/ravishar313/PyMolAI
cd PyMolAI
uv venv .venv --python 3.10
.\.venv\Scripts\Activate.ps1
$env:PREFIX_PATH = "C:\path\to\deps"; uv pip install --python .venv\Scripts\python.exe --reinstall .
uv pip install --python .venv\Scripts\python.exe PyQt5

Optional import verification:

python -c "import keyring, openai; print('ok: keyring/openai')"
python -c "import claude_agent_sdk; print('ok: claude-agent-sdk')"
python -c "from PyQt5 import QtWidgets; print('ok: PyQt5')"

First-Run Setup

1. Launch PyMOL (make run or .venv/bin/pymol).
2. Select your provider:
   • Display -> PyMolAI Settings -> Provider -> Anthropic (Direct) for direct Anthropic API access
   • Display -> PyMolAI Settings -> Provider -> OpenRouter for multi-provider access via OpenRouter
3. Save your API key:
   • Display -> PyMolAI Settings -> API Key... — enter, save, and test your key (stored in system keychain)
4. Optionally configure OpenBio tools:
   • Display -> PyMolAI Settings -> OpenBio API Key... — sign up at openbio.tech first

Important behavior:

• Without an API key for the selected provider (ANTHROPIC_API_KEY or OPENROUTER_API_KEY), AI mode is disabled.
• Without OpenBio key (OPENBIO_API_KEY), OpenBio tools are not registered, but the rest of app behavior remains unchanged.

Model Selector

Model selection is available at:

• Display -> PyMolAI Settings -> Model

Supported models:

• google/gemini-3.1-pro-preview
• anthropic/claude-sonnet-4.6 (default)
• z-ai/glm-5
• minimax/minimax-m2.5
• moonshotai/kimi-k2.5
• google/gemini-3-flash-preview
• anthropic/claude-haiku-4.5

Behavior:

• The UI selector only lists and selects the supported models above.
• /ai model <id> remains flexible and accepts custom model IDs.
• If you switch model while a run is active, the current run continues and the new model applies on the next turn.

Example scene output:

Configuration Reference

Core Provider Variables

• ANTHROPIC_API_KEY — direct Anthropic provider key
• OPENROUTER_API_KEY — OpenRouter provider key
• ANTHROPIC_AUTH_TOKEN — legacy fallback (used by OpenRouter proxy path)
• OPENBIO_API_KEY
• OPENBIO_BASE_URL (optional override, default is https://api.openbio.tech)

Key Source Tracking Variables

• PYMOL_AI_OPENROUTER_KEY_SOURCE
• PYMOL_AI_OPENBIO_KEY_SOURCE

Common Runtime Toggles

• PYMOL_AI_DISABLE
• PYMOL_AI_DEFAULT_PROVIDER — set default provider at startup (openrouter or anthropic)
• PYMOL_AI_DEFAULT_MODEL
• PYMOL_AI_REASONING_DEFAULT
• PYMOL_AI_AGENT_MODE
• PYMOL_AI_CONVERSATION_MODE
• PYMOL_AI_TRACE_STREAM

Troubleshooting

Keychain Unavailable

Symptom:

• Save/Test dialogs report no keychain backend.

Cause:

• keyring is installed but no OS keychain backend is available.

Action:

• Configure OS credential storage (macOS Keychain / Windows Credential Manager), then retry.

API Key Validation Fails

Symptom:

• Dialog returns invalid/revoked or auth errors.

Action:

• Confirm the key is from the correct provider.
• Re-test with current network/proxy.
• For OpenBio, verify endpoint/network path to api.openbio.tech.

Python 3.9 Environment

Symptom:

• AI runtime cannot use Claude SDK features.

Cause:

• claude-agent-sdk is not installed for Python 3.9.

Action:

• Use Python 3.10+ for full PyMolAI agent behavior.

PyMOL Crashes on Launch with AttributeError in Qt Enums

Symptom:

• AttributeError: type object 'Qt' has no attribute 'ScrollBarAlwaysOff' (or similar enum errors).

Cause:

• PyQt6 is installed but not PyQt5. PyMOL detects PyQt6 as a fallback, but the GUI code uses PyQt5-style flat enum access (Qt.ScrollBarAlwaysOff) which was namespaced in PyQt6 (Qt.ScrollBarPolicy.ScrollBarAlwaysOff).

Action:

• Install PyQt5 explicitly. PyMOL's Qt detection tries PyQt5 first, so it will be used instead of PyQt6:

  uv pip install --python .venv/bin/python PyQt5

macOS Build Fails: mmtf.hpp not found

Symptom:

• Build error: fatal error: 'mmtf.hpp' file not found

Cause:

• The build system detects msgpack-cxx headers (e.g. from Homebrew) and enables MMTF support, but the mmtf-cpp header-only library is not installed.

Action:

• Disable MMTF support by passing use-msgpackc=no. The Makefile already handles this:

  make install

  Or manually:

  PREFIX_PATH=... uv pip install --python .venv/bin/python --reinstall --config-settings="use-msgpackc=no" .

macOS Build Fails: netcdf.h not found

Symptom:

• Build error: fatal error: 'netcdf.h' file not found

Action:

• Install netcdf via Homebrew and add it to PREFIX_PATH:

  brew install netcdf
  PREFIX_PATH=/opt/homebrew:/opt/homebrew/opt/libxml2:/opt/homebrew/opt/netcdf uv pip install ...

macOS Build Fails: GL/glew.h not found

Symptom:

• Build error: fatal error: 'GL/glew.h' file not found

Action:

• Install GLEW and GLM via Homebrew:

  brew install glew glm

OpenBio Unavailable but OpenRouter Works

Symptom:

• Regular AI works, OpenBio tools do not appear or fail.

Action:

• Ensure OPENBIO_API_KEY exists (or save in OpenBio dialog).
• Confirm OPENBIO_BASE_URL if overridden.
• Check network restrictions/firewall policy.

Security Notes

• API keys are masked in UI status (for example ****abcd).
• Key save/clear flows use system keychain via keyring.
• Saved keys can be loaded into environment at runtime startup.
• Avoid putting real keys in scripts or checked-in files.

License and Attribution

• The legal project license remains unchanged in LICENSE.
• PyMolAI additions in this fork are documented under the same repository licensing context.
• AI service/library usage notes are listed in docs/THIRD_PARTY_AI.md.
• Fork maintainer attribution for PyMolAI additions is included in LICENSE, AUTHORS, and DEVELOPERS, with upstream Schrodinger notices preserved.

Maintainer Contact

• Website: proteinlanguagemodel.com
• X/Twitter: @ravishar313

Legacy Upstream Notes

For historical upstream notes, see README and INSTALL.