python development environment on a mac

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

python development environment on a mac

macports-2
my son needs for his music studies a python development environment 2.7.13 plus the actual 3.x.
his imac is on osx 10.11 el capitan.

so i installed from macports python 27 and 33, ipython, selects:

 python27 @2.7.13_0+readline (active)
 python36 @3.6.0_0 (active)
 python2_select @0.0_2 (active)
 python3_select @0.0_1 (active)
 python_select @0.3_7 (active)

 py27-ipython @5.3.0_0 (active)
 py36-ipython @5.3.0_0 (active)
 ipython2_select @0.1_0 (active)
 ipython3_select @0.1_0 (active)
 ipython_select @0.4_0 (active)

 py27-future @0.15.2_0 (active)
 py36-future @0.15.2_0 (active)

 py27-six @1.10.0_0 (active)
 py36-six @1.10.0_0 (active)

 py27-pip @9.0.1_0 (active)
 py36-pip @9.0.1_0 (active)
 pip_select @0.1_0 (active)

 py27-virtualenv @15.1.0_0 (active)
 py27-virtualenv-clone @0.2.6_0 (active)
 py27-virtualenvwrapper @4.7.1_0 (active)
 py36-virtualenv @15.1.0_0 (active)
 py36-virtualenv-clone @0.2.6_0 (active)
 py36-virtualenvwrapper @4.7.1_0 (active)
 virtualenv_select @0.1_0 (active)

port select --summary
Name        Selected      Options
====        ========      =======
ipython     py27-ipython  py27-ipython py36-ipython none
ipython2    py27-ipython  py27-ipython none
ipython3    py36-ipython  py36-ipython none
pip         pip27         pip27 pip36 none
python      python27      python26-apple python27 python27-apple python36 none
python2     python27      python26-apple python27 python27-apple none
python3     python36      python36 none
virtualenv  virtualenv27  virtualenv27 virtualenv36 none

$ which virtualenv
/opt/local/bin/virtualenv
$ virtualenv $HOME/PyEnv/python/27
New python executable in /Users/paul/PyEnv/python/27/bin/python
Installing setuptools, pip, wheel...done.
$ virtualenv $HOME/PyEnv/python/36
New python executable in /Users/paul/PyEnv/python/36/bin/python
Installing setuptools, pip, wheel...done.

$ . $HOME/PyEnv/python/27/bin/activate
$ python --version
Python 2.7.13
$ ipython --version
5.3.0
$ pip --version
pip 9.0.1 from /Users/me/PyEnv/python/27/lib/python2.7/site-packages (python 2.7)

$ ipython
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/IPython/core/interactiveshell.py:724: UserWarning: Attempting to work in a virtualenv. If you encounter problems, please install IPython inside the virtualenv.
 warn("Attempting to work in a virtualenv. If you encounter problems, please "

questions:
- should i install ipython in this virtualenv
- if so, how (macports or not)?

$ ls /$HOME/me/PyEnv/python/*/bin
/$HOME/me/PyEnv/python/27/bin:
activate activate_this.py pip python python2.7
activate.csh easy_install pip2 python-config wheel
activate.fish easy_install-2.7 pip2.7 python2
/$HOME/me/PyEnv/python/36/bin:
activate activate_this.py pip python python2.7
activate.csh easy_install pip2 python-config wheel
activate.fish easy_install-2.7 pip2.7 python2

questions:
- does virtuelenv36 uses 2.7 dus to port --select?
- if so, how should i have done it?

the big question: please explain to me the relation between macports python/ipython/pip/virtualenv and how i should use it? when & how macports, when & how not, why???






Reply | Threaded
Open this post in threaded view
|

Re: python development environment on a mac

Mojca Miklavec-2
On 13 March 2017 at 14:28,  <[hidden email]> wrote:
> my son needs for his music studies a python development environment 2.7.13 plus the actual 3.x.
> his imac is on osx 10.11 el capitan.
>
> so i installed from macports python 27 and 33, ipython, selects:

You mean Python 3.6?

> questions:
> - should i install ipython in this virtualenv
> - if so, how (macports or not)?
>
> questions:
> - does virtuelenv36 uses 2.7 dus to port --select?

I don't understand this question.

> - if so, how should i have done it?
>
> the big question: please explain to me the relation between macports python/ipython/pip/virtualenv and how i should use it? when & how macports, when & how not, why???

I'm not able to answer all of your questions, but you probably want to run
    sudo port select --set python2 python27
    sudo port select --set python3 python36
    sudo port select --set python python27

so that python, python2 and python3 will launch the desired version of
Python (you can change the versions according to your needs),
otherwise "python" will launch the system python.

Generally I would say that if python packages from MacPorts satisfy
your needs, it's probably better to stick with Python shipped by
MacPorts and not use virtualenv at all (and use "sudo port install
py27-whatever" rather than "pip-2.7 install whatever" or "pip install
whatever").

Virtualenv is particularly important when using an "ancient
distribution". (A lot of things become "ancient" pretty soon. Even the
latest stable linux distribution might provide packages that are a few
years old due to their strict release policies, while you might want
to use either the bleeding edge version of a package (maybe because
you filed a bug report that's blocking your work and they fixed the
error, but didn't even provide a new release yet) or because the
software you want to use no longer works with the latest package and
you need an older one.

Your son might get instructions telling him to run "virtualenv ... &&
pip install foo bar whatever". When using MacPorts, those commands
should be slightly altered (you would want to find the suitable
package), but in most cases the result should be perfectly fine. Only
if some problems arise that cannot be addressed by MacPorts (or if the
person doesn't know how to adapt the installation commands and doesn't
want to bother), then it makes sense to use virtualenv.

If some packages are missing, you could also request them.

Mojca
Reply | Threaded
Open this post in threaded view
|

Re: python development environment on a mac

Joshua Root-8
In reply to this post by macports-2
> $ which virtualenv
> /opt/local/bin/virtualenv
> $ virtualenv $HOME/PyEnv/python/27
> New python executable in /Users/paul/PyEnv/python/27/bin/python
> Installing setuptools, pip, wheel...done.
> $ virtualenv $HOME/PyEnv/python/36
> New python executable in /Users/paul/PyEnv/python/36/bin/python
> Installing setuptools, pip, wheel...done.

I notice you ran 'virtualenv' here both times, which is linked to
virtualenv27.

> $ ls /$HOME/me/PyEnv/python/*/bin
> /$HOME/me/PyEnv/python/27/bin:
> activate activate_this.py pip python python2.7
> activate.csh easy_install pip2 python-config wheel
> activate.fish easy_install-2.7 pip2.7 python2
> /$HOME/me/PyEnv/python/36/bin:
> activate activate_this.py pip python python2.7
> activate.csh easy_install pip2 python-config wheel
> activate.fish easy_install-2.7 pip2.7 python2

So this is not surprising given the above. If you want a python 3.6
virtualenv, use virtualenv36 (or whatever the 3.6 version is called).

> the big question: please explain to me the relation between macports python/ipython/pip/virtualenv and how i should use it? when & how macports, when & how not, why???

Well, IPython is a python module that provides an enhanced interactive
shell for python. Pip is the official python module installer.
Virtualenv is a system for making isolated python environments.

MacPorts manages most of the things that pip does itself, so you
normally shouldn't use pip to install modules into the macports prefix.
It's fine to use it to install into a separate location (e.g. in your
home directory) or into a virtualenv.

Use a virtualenv when you want a python environment containing a
specific set of modules for a certain task. The documentation has more
info: <https://virtualenv.pypa.io/en/stable/>

- Josh
Reply | Threaded
Open this post in threaded view
|

Re: python development environment on a mac

Eric A. Borisch-2
You might also look at pyNN-spyder-devel (NN=27, 33, etc) -- it can be very nice for someone getting started.

(I intend to update the -spyder port to match what is currently -spyder-devel, but haven't for lack of time...)

 - Eric

On Tue, Mar 14, 2017 at 11:54 AM, Joshua Root <[hidden email]> wrote:
$ which virtualenv
/opt/local/bin/virtualenv
$ virtualenv $HOME/PyEnv/python/27
New python executable in /Users/paul/PyEnv/python/27/bin/python
Installing setuptools, pip, wheel...done.
$ virtualenv $HOME/PyEnv/python/36
New python executable in /Users/paul/PyEnv/python/36/bin/python
Installing setuptools, pip, wheel...done.

I notice you ran 'virtualenv' here both times, which is linked to virtualenv27.

$ ls /$HOME/me/PyEnv/python/*/bin
/$HOME/me/PyEnv/python/27/bin:
activate                activate_this.py        pip                     python                  python2.7
activate.csh            easy_install            pip2                    python-config           wheel
activate.fish           easy_install-2.7        pip2.7                  python2
/$HOME/me/PyEnv/python/36/bin:
activate                activate_this.py        pip                     python                  python2.7
activate.csh            easy_install            pip2                    python-config           wheel
activate.fish           easy_install-2.7        pip2.7                  python2

So this is not surprising given the above. If you want a python 3.6 virtualenv, use virtualenv36 (or whatever the 3.6 version is called).

the big question: please explain to me the relation between macports python/ipython/pip/virtualenv and how i should use it? when & how macports, when & how not, why???

Well, IPython is a python module that provides an enhanced interactive shell for python. Pip is the official python module installer. Virtualenv is a system for making isolated python environments.

MacPorts manages most of the things that pip does itself, so you normally shouldn't use pip to install modules into the macports prefix. It's fine to use it to install into a separate location (e.g. in your home directory) or into a virtualenv.

Use a virtualenv when you want a python environment containing a specific set of modules for a certain task. The documentation has more info: <https://virtualenv.pypa.io/en/stable/>

- Josh

Reply | Threaded
Open this post in threaded view
|

Re: python development environment on a mac

Kendall Shaw-3
In reply to this post by macports-2
On 03/13/2017 06:28 AM, [hidden email] wrote:

> my son needs for his music studies a python development environment 2.7.13 plus the actual 3.x.
> his imac is on osx 10.11 el capitan.
>
...
> the big question: please explain to me the relation between macports python/ipython/pip/virtualenv and how i should use it? when & how macports, when & how not, why???

I can offer some information that isn't macports specific.

In some unspecified OS packaging system that includes python and python
packages within it's own package management system, without using
packages installed using the python package management system, you would
be installing packages  system wide which would have updates available
when updates were available in the OS package management system, with
the cascading dependencies associated with that.

In that scenario, your python project would work for you because of the
state of your OS. If you then shared the source with someone else they
would have to try to make their system match your OS environment
sufficiently to have the same python modules available.

pip is  a tool for interacting with a python package management system:

https://packaging.python.org/

Python packages, generally, will make available python modules which a
python project will import.

When it is used as part of a software project it implies that you would
organize your python software package in a way that it is compatible
with a standard which makes it easy for people to know how to configure
and build your project and to incorporate it into their projects.

virtualenv allows you to isolate your projects from each other and from
a surrounding python system. So, for example, 1 project that has a
dependency  that is incompatible with a dependency in another project
can still work since they are isolated from each other.

Kendall

Reply | Threaded
Open this post in threaded view
|

Re: python development environment on a mac

Jeremy Lavergne-2
To that end, also check out using pip-tools along with virtualenv. This
will allow you to determine the whole set of dependencies and their
versions.

Example shell script using pip-tools inside a virtualenv for producing a
requirements.txt for `pip install --upgrade -r requirements.txt`:

#!/usr/bin/env bash
set -eux
# needs pip-tools: sudo pip install -U pip-tools

HERE="$(dirname $0)"
REQUIREMENTS_FILE="${HERE}/requirements.txt"

REQUIREMENTS_VE="/tmp/requirements_update_ve"

rm -rf "${REQUIREMENTS_VE}"
virtualenv "${REQUIREMENTS_VE}"
set +eux
. "${REQUIREMENTS_VE}/bin/activate"
set -eux
pip install -U pip-tools

pip-compile --upgrade --rebuild --verbose --annotate
--output-file=${REQUIREMENTS_FILE} - <<REQUIREMENTS
awscli
boto
boto3
lxml
netaddr
requests
beautifulsoup4
REQUIREMENTS

rm -rf "${REQUIREMENTS_VE}"


On 03/14/2017 10:42 PM, Kendall Shaw wrote:
> virtualenv allows you to isolate your projects from each other and from
> a surrounding python system. So, for example, 1 project that has a
> dependency  that is incompatible with a dependency in another project
> can still work since they are isolated from each other.

Reply | Threaded
Open this post in threaded view
|

Re: python development environment on a mac

Alexander Garcia Castro
this may be unrelated but do anyone knows how can I change the python version in spyder? I have some scripts that work with python 2 and others with python 3. 

On Wed, Mar 15, 2017 at 7:45 AM, Jeremy Lavergne <[hidden email]> wrote:
To that end, also check out using pip-tools along with virtualenv. This
will allow you to determine the whole set of dependencies and their
versions.

Example shell script using pip-tools inside a virtualenv for producing a
requirements.txt for `pip install --upgrade -r requirements.txt`:

#!/usr/bin/env bash
set -eux
# needs pip-tools: sudo pip install -U pip-tools

HERE="$(dirname $0)"
REQUIREMENTS_FILE="${HERE}/requirements.txt"

REQUIREMENTS_VE="/tmp/requirements_update_ve"

rm -rf "${REQUIREMENTS_VE}"
virtualenv "${REQUIREMENTS_VE}"
set +eux
. "${REQUIREMENTS_VE}/bin/activate"
set -eux
pip install -U pip-tools

pip-compile --upgrade --rebuild --verbose --annotate
--output-file=${REQUIREMENTS_FILE} - <<REQUIREMENTS
awscli
boto
boto3
lxml
netaddr
requests
beautifulsoup4
REQUIREMENTS

rm -rf "${REQUIREMENTS_VE}"


On 03/14/2017 10:42 PM, Kendall Shaw wrote:
> virtualenv allows you to isolate your projects from each other and from
> a surrounding python system. So, for example, 1 project that has a
> dependency  that is incompatible with a dependency in another project
> can still work since they are isolated from each other.




--
Reply | Threaded
Open this post in threaded view
|

Re: python development environment on a mac

Eric A. Borisch-2
On Wed, Mar 15, 2017 at 8:39 AM, Alexander Garcia Castro <[hidden email]> wrote:
this may be unrelated but do anyone knows how can I change the python version in spyder? I have some scripts that work with python 2 and others with python 3. 

You _can_ set it in Preferences: Python interpreter

Not that you _should_ as "mixing python versions is not really supported." You can install pyNN-spyder-devel and pyMM-spyder-devel concurrently; the preference directories are kept separate in the MacPorts versions. (Again, I'd recommend the -devel versions.)

 - Eric
Reply | Threaded
Open this post in threaded view
|

Re: python development environment on a mac

macports-2
In reply to this post by Mojca Miklavec-2

> On 15 Mar 2017, at 08:51, Mojca Miklavec <[hidden email]> wrote:
>
> Any reason why you sent the response off-list?

no excuse but it was 04:15 AM.

> I can answer, but it makes more sense to continue on the list.
>
> But in short: you don't need virtualenv just for the sake of
> supporting both python 2 and 3.

i understand.

> Mojca

Reply | Threaded
Open this post in threaded view
|

Re: python development environment on a mac

macports-2

> On 15 Mar 2017, at 15:49, [hidden email] wrote:
>
>
>> On 15 Mar 2017, at 08:51, Mojca Miklavec <[hidden email]> wrote:
>>
>> Any reason why you sent the response off-list?
>
> no excuse but it was 04:15 AM.
>
>> I can answer, but it makes more sense to continue on the list.
>>
>> But in short: you don't need virtualenv just for the sake of
>> supporting both python 2 and 3.
>
> i understand.
>
>> Mojca



OK. next problem.

he has an older imac and a newer macbook pro (soon), with different versions of osx/macos, so the python system environment will be different. to be able to work on both macs i think virtualenv may be right way to go.

for me, developing on osx and production on linux is about the same thing.

do you agree or not?

paul.
Reply | Threaded
Open this post in threaded view
|

Re: python development environment on a mac

Mojca Miklavec
In reply to this post by macports-2
On 15 March 2017 at 15:49,  <[hidden email]> wrote:
>
>> On 15 Mar 2017, at 08:51, Mojca Miklavec <[hidden email]> wrote:
>>
>> Any reason why you sent the response off-list?
>
> no excuse but it was 04:15 AM.

So just send the same reply to the list then.

Mojca
Reply | Threaded
Open this post in threaded view
|

Re: python development environment on a mac

Mojca Miklavec
In reply to this post by macports-2
On 15 March 2017 at 18:03,  <[hidden email]> wrote:
>
> OK. next problem.
>
> he has an older imac and a newer macbook pro (soon), with different versions of osx/macos, so the python system environment will be different.

No. If you install python27 and python36 with MacPorts, he'll have
exactly the same version(s) of Python available.

The only important thing is to call the right python when executing
the script. If you do "port select" for "python2" and "python3", just
call "python2 your-python-script.py" and/or "python3
your-python-script.py" and the result should be the same on any
machine (unless that version of python would not compile on that
particular OS, but I don't think this is an issue unless you are on
10.4).

> to be able to work on both macs i think virtualenv may be right way to go.
>
> do you agree or not?

No. Virtualenv is when you require very specific versions of
dependencies that MacPorts cannot fulfill.

Mojca
Reply | Threaded
Open this post in threaded view
|

Re: python development environment on a mac

macports-2

> On 15 Mar 2017, at 18:48, Mojca Miklavec <[hidden email]> wrote:
>
> On 15 March 2017 at 18:03,  <[hidden email]> wrote:
>>
>> OK. next problem.
>>
>> he has an older imac and a newer macbook pro (soon), with different versions of osx/macos, so the python system environment will be different.
>
> No. If you install python27 and python36 with MacPorts, he'll have
> exactly the same version(s) of Python available.
>
> The only important thing is to call the right python when executing
> the script. If you do "port select" for "python2" and "python3", just
> call "python2 your-python-script.py" and/or "python3
> your-python-script.py" and the result should be the same on any
> machine (unless that version of python would not compile on that
> particular OS, but I don't think this is an issue unless you are on
> 10.4).
>
>> to be able to work on both macs i think virtualenv may be right way to go.
>>
>> do you agree or not?
>
> No. Virtualenv is when you require very specific versions of
> dependencies that MacPorts cannot fulfill.
>
> Mojca

OK. i'll try to behave ;-)
Reply | Threaded
Open this post in threaded view
|

Re: python development environment on a mac

Kendall Shaw-2
In reply to this post by macports-2
On 3/15/17, 10:03 AM, "macports-users on behalf of [hidden email]" <[hidden email] on behalf of [hidden email]> wrote:

    OK. next problem.
   
    he has an older imac and a newer macbook pro (soon), with different versions of osx/macos, so the python system environment will be different. to be able to work on both macs i think virtualenv may be right way to go.
   
    for me, developing on osx and production on linux is about the same thing.
   
    do you agree or not?
   
    paul.

So, I guess there can be differences of philosophy. But, if python projects will be written so that they include setup.py, your son would list the dependencies in setup.py

https://packaging.python.org/distributing/#setup-args

The way that this gets used is someone runs setup.py which installs the dependencies, from a pypi repository. Unless macports somehow integrates with pip, this means the person’s packages are not being installed by macports. But, that is not really relevant if it is expected that people will be using pip or an equivalent for installing dependencies.

Also, these days other people are likely to be using something other than macports as their OS package management software, e.g. brew.

So, essentially, yes use pip. While you are at it, it’s probably a good idea to use virtualenv so that it’s less likely you will end up introducing a depenendcy that requires someone else to have the same OS environment.

Kendall