-
Notifications
You must be signed in to change notification settings - Fork 85
Add conda development install guidance #660
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
ericdill
wants to merge
4
commits into
pyOpenSci:main
Choose a base branch
from
ericdill:doc/why-pip-install-editable
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,54 @@ | ||
| ## Installing your own code | ||
|
|
||
| You have a conda environment. It works. Maybe it has packages that were hard to install, like GDAL, HDF5, or other compiled scientific dependencies. | ||
|
|
||
| You also have code that you are writing locally. Maybe it started as a script, or maybe it is already organized as a Python package. You want to use that code inside the same environment with GDAL, HDF5, and the other tools you already installed. | ||
|
|
||
| The instructions to install your code into a conda environment is to first activate your conda environment `conda activate your_env_name` and then run this: `python -m pip install -e . --no-deps`. You may also see this written as `pip install -e .`. See [The Full Command](the-full-command) section below for more info as to the details of this command. | ||
|
|
||
| If this is the first time you're seeing pip install commands, you may not be totally sure what is going on here. Conda created the environment, why am I using `pip` to install things now? You may have heard guidance to generally try and avoid mixing conda and pip? You may already be mixing conda and pip and things are totally fine. You may also not care at all because `pip install -e .` seems to work fine and you can get back to what you're actually trying to do. (If that last one is you, you're also probably not reading this page). In any event, all of these situations are perfectly understandable and totally okay for you to be going through. | ||
|
|
||
| So... why pip? The short answer is that conda and pip are doing different jobs here. | ||
|
|
||
| The slightly longer, mostly apologetic, answer is this is just sort of the current ergonomics of how python packaging works and, honestly? Most of us have turned this confusing pain point into muscle memory. But not you. You're new here. And you're like... wat? And you're totally justified to feel this way. | ||
|
|
||
| So, what is happening here? `conda` manages the environment: the Python runtime, compiled libraries, command line tools, and the packages your project depends on. This is stuff that you've already been doing and you're comfortable with (or at least familiar with). `pip` is doing one Python-packaging-specific job: installing your local package into the active environment. | ||
|
|
||
| In editable mode, the `-e` flag, `pip` connects the active environment to the source files you are editing. And... why exactly is that useful? | ||
|
|
||
| It's useful because it gives you pretty quick and simple development loop: | ||
|
|
||
| 1. Edit your code in your editor. | ||
| 2. Run it from a terminal, test suite, or Jupyter notebook. | ||
| 3. Edit the code again. | ||
| 4. Run it again without reinstalling your package. | ||
|
|
||
| So the goal is not to switch from conda to pip. The goal is to keep using your conda environment while making your local package importable inside that environment. | ||
|
|
||
| ## Should I use pip for everything now? | ||
|
|
||
| Probably not? | ||
|
|
||
| If conda is already working well for your project, keep using conda to manage the environment. Use pip only for this one task: installing your local package in editable mode. | ||
|
|
||
| If you are curious about other tools like uv, pixi, Hatch, or pip-only workflows, see [Environment Managers](environment-managers.md). Those tools can be great choices. But you do not need to switch tools just to develop your local package inside a conda environment. | ||
|
|
||
| As a final note, people in the conda ecosystem are actively working on better conda/pip interop. In the future, this workflow may become less awkward. For now, `python -m pip install -e . --no-deps` is the standard bridge. | ||
|
ericdill marked this conversation as resolved.
Outdated
|
||
|
|
||
| (the-full-command)= | ||
| ## The full command | ||
|
|
||
| `python -m pip install -e . --no-deps` is a mouthful. I know it. You know it. Why do we do these things? | ||
|
|
||
| The simplest version of this is: | ||
| ``` | ||
| pip install -e . | ||
| ``` | ||
|
|
||
| But we recommend the longer version in conda environments for two reasons. | ||
|
|
||
| The `python -m pip` part ensures that you're using pip from the active conda environment. Sometimes this results in `pip not found`, which is actually a good error to get because it means you prevented an annoying-to-debug failure mode. If this happens just `conda install pip` and try again. So, why? Sometimes `pip` from a different python environment can be on your PATH which means that you'll accidentally install your code into an unrelated python environment. This can be confusing to debug. This has happened to most (all?) of us. It usually hits when you're least prepared to debug and fix it. So we recommend the `python -m` in front to prevent this from happening. But it does add to the length of the command. | ||
|
|
||
| The `--no-deps` flag tells pip not to install your package's dependencies, if you have any listed in your project. If you do have them listed, probably in your `pyproject.toml` file, then `pip install -e .` will try to install the dependencies that are listed in that file. In a conda environment, that can range from "mostly fine" to "now my environment is broken and I am not sure how to recover." | ||
|
|
||
| With `--no-deps`, pip installs only your local package. You remain responsible for managing the environment dependencies with conda. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.