-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathpython-01-setup.qmd
More file actions
187 lines (120 loc) · 7.23 KB
/
Copy pathpython-01-setup.qmd
File metadata and controls
187 lines (120 loc) · 7.23 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
---
title: "Setup and running Jupyter notebooks"
---
::: {.callout-note}
## Disclaimer
These notes started number of years ago from the [official SWC
lesson](https://software-carpentry.org/lessons){target="_blank"} but then evolved quite a bit to include other
topics.
:::
## Why Python?
Python is a free, open-source programming language first developed in the late 1980s and 90s that became
really popular for scientific computing in the past 15 years. With Python in a few minutes you can:
- analyze thousands of texts,
- process tables with billions of records,
- manipulate thousands of images,
- restructure and process data any way you want.
#### Python vs. Excel
- Unlike Excel, Python can read any type of data, both structured and unstructured.
- Python is free and open-source, so no artificial limitations on where/how you run it.
- Python works on all platforms: Windows, Mac, Linux, Android, etc.
- Data manipulation is much easier in Python. There are hundreds of data processing, machine learning, and
visualization libraries.
- Python can handle much larger amounts of data: limited not by Python, but by your available computing
resources. In addition, Python can run at scale (in parallel) on larger systems.
- Python is more reproducible (rerun / modify the script).
<!-- Python code is easier to reproduce -->
<!-- Python is faster doing difficult calculations. -->
<!-- Python is easier than vba. -->
<!-- Python works better with big data. -->
<!-- Python is open source and has access to an enormous amount of libraries. -->
<!-- On the other hand. -->
<!-- Excel is known by more people. -->
<!-- Excel is faster for simple calculations, graphs etc. -->
#### Python vs. other programming languages
Python pros | Python cons
--------------------------------------------|------------------------
elegant scripting language | slow (interpreted, dynamically typed)
easy to write and read code | uses indentation for code blocks
powerful, compact constructs for many tasks |
very popular across all fields |
huge number of external libraries |
## Installing Python locally
Today we'll be running Python in the cloud, so you can skip this section. I am listing these options in case
you want to install Python on your computer after the workshop.
**Option 1**: Install Python from https://www.python.org/downloads making sure to check the option "Add Python
to PATH" during the installation.
**Option 2**: Install Python and the packages via Anaconda from https://www.anaconda.com/download.
**Option 3**: Install Python via your favourite package manager, e.g. in MacOS -- assuming you have
[Homebrew](https://brew.sh) installed -- run the command `brew install python`.
**Post-installation**: Install 3rd-party Python packages in the Command Prompt / terminal via `pip install
<packageName>`, e.g. to be able to run Python inside a Jupyter Notebook run `pip install jupyter`.
## Starting Python
There are many ways to run Python commands:
* from a Unix shell you can start a Python shell and type commands there,
* you can launch Python scripts saved in plain text *.py files,
* you can execute Python cells inside Jupyter notebooks; the code is stored inside JSON files, displayed as HTML
<!-- Today we will be using a Jupyter notebook. -->
## Today's setup
Today we'll be using JupyterHub on our training cluster. Point your browser to https://hss.c3.ca and log in
with your username and password, then launch a JupyterHub server with time = ***2.5 hours***, **1 CPU core**,
memory = ***3712 MB***, GPU configuration = ***None***, user interface = ***JupyterLab***. Finally, start a
new Python 3 notebook.
After you log in, in the dashboard start a new Python 3 notebook.
<!-- 4. **Community cloud option**: use syzygy.ca with one of the following accounts: -->
<!-- - if you have a university computer ID → go to <a href="https://syzygy.ca" -->
<!-- target="_blank">syzygy.ca</a>, under Launch select your institution, then log in with your university credentials -->
<!-- - if you have a Google account → go to <a href="https://syzygy.ca" target="_blank">syzygy.ca</a>, -->
<!-- under Launch select either Cybera or PIMS, then log in with your Google account -->
<!-- <\!-- - if you have a GitHub account → go to https://westgrid.syzygy.ca, sign in with your GitHub account -\-> -->
<!-- Note that syzygy.ca is a free community service run on the Alliance cloud and used heavily for undergraduate -->
<!-- teaching, with no uptime guarantees. In other words, it usually works, but it could be unstable or down. -->
<!-- ## Virtual Python environments -->
<!-- We talk about creating Virtual Python environments in the HPC course. These environment are very useful, as -->
<!-- not only can you install packages into your directories without being root, but they also let you create -->
<!-- sandbox Python environments with your custom set of packages -- perfect when you work on multiple projects, -->
<!-- each with a different list of dependencies. -->
<!-- To create a Python environment (you do this only once): -->
<!-- ```sh -->
<!-- module avail python # several versions available -->
<!-- module load python/3.10.2 -->
<!-- virtualenv --no-download astro # install Python tools in your $HOME/astro -->
<!-- source astro/bin/activate -->
<!-- pip install --no-index --upgrade pip -->
<!-- pip install --no-index numpy jupyter pandas # all these will go into your $HOME/astro -->
<!-- avail_wheels --name "*tensorflow_gpu*" --all_versions # check out the available packages -->
<!-- pip install --no-index tensorflow_gpu==2.2.0 # if needed, install a specific version -->
<!-- ... -->
<!-- deactivate -->
<!-- ``` -->
<!-- Once created, you would use it with: -->
<!-- ```sh -->
<!-- source ~/astro/bin/activate -->
<!-- python -->
<!-- ... -->
<!-- deactivate -->
<!-- ``` -->
<!-- We'll talk more about virtual Python environments in -->
<!-- [Section 10](../python-10-libraries#virtual-environments-and-packaging). -->
## Navigating Jupyter interface
- File | Save Notebook As - to rename your notebook
- File | Download - download the notebook to your computer
- File | New Launcher - to open a new launcher dashboard, e.g. to start a terminal
- File | Log Out - to terminate your job (everything is running inside a Slurm job!)
::: {.callout-caution}
## Note to self
Explain: tab completion, annotating code, displaying figures inside the notebook.
:::
- <font size="+2">Esc</font> - leave the cell (border changes colour) to the control mode
- <font size="+2">A</font> - insert a cell above the current cell
- <font size="+2">B</font> - insert a cell below the current cell
- <font size="+2">X</font> - delete the current cell
- <font size="+2">M</font> - turn the current cell into the markdown cell
- <font size="+2">H</font> - to display help
- <font size="+2">Enter</font> - re-enter the cell (border becomes green) from the control mode
You can enter Latex expressions in a markdown cell, e.g. try typing `\int_0^\infty f(x)dx` inside two dollar
signs.
```py
print(1/2) # to run all commands in the cell, either use the Run button, or press shift+return
```