Skip to content

Commit

Permalink
slide polish for may 2024
Browse files Browse the repository at this point in the history 
Corrected some old links.
Updated Python version info.
Added some new code examples.
  • Loading branch information
@brianhmiller
brianhmiller committed May 28, 2024
1 parent 28332da commit 1c5071b
Show file tree
Hide file tree
Showing 11 changed files with 445 additions and 854 deletions.
8 changes: 4 additions & 4 deletions content/00_about_the_course.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -64,8 +64,8 @@
"You should already have knowledge of:\n",
"- Python fundamentals (identifier, data types, loops, functions, OOP) is assumed.\n",
"- If you are new to Python, consider completing a beginner Python course first.\n",
"- We assume that we are on Python 3.8+\n",
"- Git (In case you want to clone this repo and run it locally)."
"- Any Python version 3.8+ is compatible with this material.\n",
"- Git (on case you want to clone this repo and run it locally)."
]
},
{
Expand All @@ -76,7 +76,7 @@
"## What can you expect?\n",
"\n",
"- Dive deeper into intermediate and somewhat advanced Python concepts and topics.\n",
"- Gain a comprehensive understanding of intermediate Python features.\n",
"- Understand the mechanics of memory, namespaces, sequences, and other types of Python constructs.\n",
"- Explore real-world applications of intermediate Python concepts."
]
},
Expand Down Expand Up @@ -154,7 +154,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.4"
"version": "3.11.9"
}
},
"nbformat": 4,
Expand Down
140 changes: 111 additions & 29 deletions content/02_memory_management_in_python.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -181,7 +181,7 @@
"\n",
"c_address = id(c)\n",
"print(c_address)\n",
"print(hex(c_address))"
"print(hex(c_address))\n"
]
},
{
Expand All @@ -198,7 +198,12 @@
"print(ctypes.cast(a_address, ctypes.py_object).value)\n",
"\n",
"# *c_address\n",
"print(ctypes.cast(c_address, ctypes.py_object).value)"
"print(ctypes.cast(c_address, ctypes.py_object).value)\n",
"\n",
"a = 10\n",
"a_id = id(a)\n",
"print(a)\n",
"print(ctypes.cast(a_id, ctypes.py_object).value)\n"
]
},
{
Expand All @@ -218,7 +223,9 @@
"metadata": {},
"outputs": [],
"source": [
"my_var = 12324"
"foo = 104\n",
"x = sys.getrefcount(foo)\n",
"print(f'{x = }')\n"
]
},
{
Expand Down Expand Up @@ -303,7 +310,8 @@
"source": [
"print(id(my_var))\n",
"print(id(my_var2))\n",
"id(my_var) == id(my_var2)"
"id(my_var) == id(my_var2)\n",
"my_var is my_var2"
]
},
{
Expand Down Expand Up @@ -396,7 +404,7 @@
"id": "7ae99b2c-84ec-4086-bd9d-bc6afbd7dca2",
"metadata": {},
"source": [
"**Best PyCon talk in Memory management**\n",
"**Excellent PyCon talk in Memory management**\n",
"\n",
"[![](https://img.youtube.com/vi/F6u5rhUQ6dU/0.jpg)](https://youtu.be/F6u5rhUQ6dU)"
]
Expand Down Expand Up @@ -599,7 +607,9 @@
"metadata": {},
"outputs": [],
"source": [
"sys.getsizeof([1, 2, [1, 2, 3, 4]])"
"print(sys.getsizeof([1, 2, [1, 2, 3, 4]]))\n",
"\n",
"print(sys.getsizeof([1,2,[1,2,3,4],(1,2,3,4)]))"
]
},
{
Expand All @@ -618,7 +628,7 @@
"There are snippets and tools which can be used to get the actual size of a object.\n",
"- [COMPUTE MEMORY FOOTPRINT OF AN OBJECT AND ITS CONTENTS (PYTHON RECIPE)](https://code.activestate.com/recipes/577504/)\n",
"- [memray](https://bloomberg.github.io/memray/index.html) - only works on Linux and macOS. \n",
"- [Pympler](https://py`thonhosted.org/Pympler/#)\n",
"- [Pympler](https://pythonhosted.org/Pympler/#)\n",
"- [memory-profiler](https://github.com/pythonprofilers/memory_profiler)\n",
"- [Scalene](https://github.com/plasma-umass/scalene) - only works properly on Linux and macOS."
]
Expand All @@ -642,7 +652,7 @@
"source": [
"from pympler import asizeof\n",
"\n",
"asizeof.asizeof([1, 2, [1, 2, 3, 4]])"
"print(asizeof.asizeof([1, 2, [1, 2, 3, 4]]))"
]
},
{
Expand All @@ -660,11 +670,9 @@
"source": [
"## Garbage collection Gotcha!\n",
"\n",
"**NOTE**: This might seems a bit advance and can be skipped for intermediate level! Maybe revisit this once other topics are covered.\n",
"\n",
"- This section covers some of the gocha of how we can have objects which are still not automatically collected by the garbage collector, sometimes also called as `memory leak`.\n",
"- Garbage collection works 99.9% of the time. Here are some of the 0.1% cases that can cause memory leaks.\n",
"- Some may be collected manually by running `gc.collect()`.\n",
"- Other may not be because of the way they were referencec.\n",
"- Others may not be garbage collected without refactoring.\n",
"We will also see the BKM of maintaining weak references which are removed as soon as the object goes out of scope."
]
},
Expand Down Expand Up @@ -700,7 +708,12 @@
"metadata": {},
"outputs": [],
"source": [
"c = None"
"c = C(11)\n",
"c = None\n",
"print('hi')\n",
"import gc\n",
"gc.collect()\n",
"print('hi2')"
]
},
{
Expand All @@ -719,8 +732,10 @@
"outputs": [],
"source": [
"a = [1, 2]\n",
"a.append(a)\n",
"\n",
"b = [3, 4]\n",
"a.append(b)\n",
"print(a)\n",
"b.append(a)\n",
"print(a)"
]
},
Expand Down Expand Up @@ -765,6 +780,14 @@
"c = None"
]
},
{
"cell_type": "markdown",
"id": "8362a791",
"metadata": {},
"source": [
"`gc.collect()` can be used to explicitly invoke the garbage collector, which will attempt to detect circular references."
]
},
{
"cell_type": "code",
"execution_count": null,
Expand Down Expand Up @@ -982,7 +1005,7 @@
"id": "ca2c0c04-c1be-43f8-9724-746dfb6aeeb9",
"metadata": {},
"source": [
"#### `@lru_cache` in methods can also cause memory leak!\n",
"#### Using `@lru_cache` in methods can be a source of memory leaks.\n",
"\n",
"- ref: [issue19859 in python bugs](https://bugs.python.org/issue19859)\n",
"- ref: [How do I cache method calls](https://docs.python.org/3/faq/programming.html#how-do-i-cache-method-calls)"
Expand Down Expand Up @@ -1322,8 +1345,12 @@
"a_list = [1, 2, 3, 4]\n",
"b_list = [1, 2, 3, 4]\n",
"\n",
"a_list.append(5)\n",
"\n",
"print(a_list == b_list)\n",
"print(a_list is b_list)"
"print(a_list is b_list)\n",
"\n",
"print(a_list,b_list)"
]
},
{
Expand Down Expand Up @@ -1385,6 +1412,22 @@
"- Numbers outside the interned range or those created dynamically are not interned. For example, `x = 1000` and `y = 1000` will have `x is y` evaluate to `False`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3ab0181e",
"metadata": {},
"outputs": [],
"source": [
"smallx = 5\n",
"smally = 5\n",
"print(f'{smallx} is {smally}? {smallx is smally}')\n",
"\n",
"bigx = 5000\n",
"bigy = 5000\n",
"print(f'{bigx} is {bigy}? {bigx is bigy}')\n"
]
},
{
"cell_type": "markdown",
"id": "159bc2f9",
Expand Down Expand Up @@ -1471,48 +1514,85 @@
"\n",
"[From Wikipedia](https://en.wikipedia.org/wiki/Peephole_optimization)\n",
"\n",
"> Peephole optimization is an optimization technique performed on a small set of compiler-generated instructions; the small set is known as the peephole or window."
"> Peephole optimization is an optimization technique performed on a small set of compiler-generated instructions; the small set is known as the peephole or window.\n",
"\n",
"Note: `__code__.co_consts` displays all the literals used in a function's bytecode: [co_consts](https://python-reference.readthedocs.io/en/latest/docs/code/consts.html)"
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 39,
"id": "c03a5b04",
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"(1440,)\n"
]
}
],
"source": [
"c = compile(\"24 * 60\", \"<string>\", \"eval\")\n",
"print(c.co_consts)"
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 40,
"id": "e34bfdb7",
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"((1, 2, 1, 2, 1, 2, 1, 2, 1, 2),)\n"
]
}
],
"source": [
"c = compile(\"(1, 2) * 5\", \"<string>\", \"eval\")\n",
"print(c.co_consts)"
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 41,
"id": "0dfadd57",
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"('xyzxyzxyzxyz',)\n"
]
}
],
"source": [
"c = compile('\"xyz\" * 4', \"<string>\", \"eval\")\n",
"print(c.co_consts)"
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 42,
"id": "5299ffe6",
"metadata": {},
"outputs": [],
"outputs": [
{
"data": {
"text/plain": [
"(None, 1440, (1, 2, 1, 2, 1, 2, 1, 2, 1, 2), 'xyzxyzxyzxyz')"
]
},
"execution_count": 42,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"def a_func():\n",
" a_int = 24 * 60\n",
Expand All @@ -1537,15 +1617,15 @@
"id": "1143c1be-07d9-48c7-9949-478a5af565f9",
"metadata": {},
"source": [
"Use `del` to delete reference when a object is not required anymore"
"`del` can be used to manually delete an object that is no longer required."
]
},
{
"cell_type": "markdown",
"id": "f8001c69-c9e0-4c6a-bf24-da8a8c37327b",
"metadata": {},
"source": [
"Use `__slot__` to save on memory."
"`__slots__` can declare the fields of a class to save memory per instance: [UsingSlots](https://wiki.python.org/moin/UsingSlots)."
]
},
{
Expand Down Expand Up @@ -1604,6 +1684,8 @@
"metadata": {},
"outputs": [],
"source": [
"print(sys.getsizeof(c))\n",
"print(sys.getsizeof(c.__dict__))\n",
"sys.getsizeof(c) + sys.getsizeof(c.__dict__)"
]
},
Expand Down Expand Up @@ -1701,7 +1783,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.6"
"version": "3.11.9"
}
},
"nbformat": 4,
Expand Down
Loading

0 comments on commit 1c5071b

Please sign in to comment.