Python virtualenv and understanding the AWS Deep Learning AMI.
In a previous blog we saw how to setup the AWS Deep Learning AMI and brief reference to python virtualenv.
ie:
[ec2-user@ip-172–31–43–77 ~]$ source activate python3
(python3) [ec2-user@ip-172–31–43–77 ~]$
This activated a preconfigured python virtual environment using the python Virtual Environment (venv) package. Virtual environments are valuable for fast tracking group classes as package configurations can be pre-prepared and deployed, saving everyone the hassle of tedious pip install operations to install packages required for operation.
That said, it’s worth understanding what your virtualenv setup includes.
When we log into the AWS Deep Learning AMI on EC2, we are greeted with this.
```
Please use one of the following commands to start the required environment with the framework of your choice:
for MXNet(+Keras1) with Python3 (CUDA 9) _____ source activate mxnet_p36
for MXNet(+Keras1) with Python2 (CUDA 9) _____ source activate mxnet_p27
for TensorFlow(+Keras2) with Python3 (CUDA 8) source activate tensorflow_p36
for TensorFlow(+Keras2) with Python2 (CUDA 8) source activate tensorflow_p27
for Theano(+Keras2) with Python3 (CUDA 8) source activate theano_p36
for Theano(+Keras2) with Python2 (CUDA 8) source activate theano_p27
for PyTorch with Python3 (CUDA 9) _source activate pytorch_p36
for PyTorch with Python2 (CUDA 9) _source activate pytorch_p27
for CNTK(+Keras2) with Python3 (CUDA 8) ______ source activate cntk_p36
for CNTK(+Keras2) with Python2 (CUDA 8) ______ source activate cntk_p27
for Caffe2 with Python2 (CUDA 9) _____________ source activate caffe2_p27
for Caffe with Python2 (CUDA 8) ______________ source activate caffe_p27
for Caffe with Python3 (CUDA 8) ______________ source activate caffe_p35
for Chainer with Python2 (CUDA 9) _______source activate chainer_p27
for Chainer with Python3 (CUDA 9) _______source activate chainer_p36
for base Python2 (CUDA 9) ____________________ source activate python2
for base Python3 (CUDA 9) ____________________ source activate python3
Since Fast.ai version 2 is written for python3 we can ignore all the xxxx_p27 virtual environments.
entering the python shell after activating the python virtalenv shown above we can check the version of python we are in and list available libraries.
(python3) [ec2-user@ip-172–31–43–77 ~]$python3
Python 3.6.4 |Anaconda, Inc.| (default, Jan 16 2018, 18:10:19)[GCC 7.2.0] on linuxType "help", "copyright", "credits" or "license" for more information.>>> import sys>>> print(sys.version)3.6.4 |Anaconda, Inc.| (default, Jan 16 2018, 18:10:19)[GCC 7.2.0]
then
import pip
installed_packages = pip.get_installed_distributions()
installed_packages_list = sorted(["%s==%s" % (i.key, i.version)
for i in installed_packages])
for package in installed_packages:
print (package)This will yield
asn1crypto==0.24.0autovizwidget==0.12.5bleach==2.1.2[etc]
We can also use the python package manager pip to list installed packages.
pip freeze<or>
pip freeze | grep <package name to filter for>
Please use one of the following commands to start the required environment with the framework of your choice:
for MXNet(+Keras1) with Python3 (CUDA 9) _____ source activate mxnet_p36
for MXNet(+Keras1) with Python2 (CUDA 9) _____ source activate mxnet_p27
for TensorFlow(+Keras2) with Python3 (CUDA 8) source activate tensorflow_p36
for TensorFlow(+Keras2) with Python2 (CUDA 8) source activate tensorflow_p27
for Theano(+Keras2) with Python3 (CUDA 8) ____ source activate theano_p36
for Theano(+Keras2) with Python2 (CUDA 8) ____ source activate theano_p27
for PyTorch with Python3 (CUDA 9) ____________ source activate pytorch_p36
for PyTorch with Python2 (CUDA 9) ____________ source activate pytorch_p27
for CNTK(+Keras2) with Python3 (CUDA 8) ______ source activate cntk_p36
for CNTK(+Keras2) with Python2 (CUDA 8) ______ source activate cntk_p27
for Caffe2 with Python2 (CUDA 9) _____________ source activate caffe2_p27
for Caffe with Python2 (CUDA 8) ______________ source activate caffe_p27
for Caffe with Python3 (CUDA 8) ______________ source activate caffe_p35
for Chainer with Python2 (CUDA 9) ____________ source activate chainer_p27
for Chainer with Python3 (CUDA 9) ____________ source activate chainer_p36
for base Python2 (CUDA 9) ____________________ source activate python2
for base Python3 (CUDA 9) ____________________ source activate python3
