# 02%20-%20Python%20Advanced%20Cheat%20Sheet%20%281%29
course: Academy — 65-GENAI-for-Engineers
module: Academy/65-GENAI-for-Engineers
type: pdf
source_url: https://personal-learn.armco.dev/files/Academy/65-GENAI-for-Engineers/folders/Python_Cheat_Sheets_Folder/02%20-%20Python%20Advanced%20Cheat%20Sheet%20%281%29.pdf
pages: 10
---
[page 1]
Python Advanced Cheat Sheet
Decorators
Coroutine
Exception Handling
Context Managers
Generators
decorator_function
try
except
finally
with
with open(file_path,mode) as file
yield
contextlib.contextmanager
next()
functools.wraps(function)
def
def
:
:
print(“Before function call”)
print(“After function call”)
return
my_decorator( )
wrapper()
wrapper
func
func()
async
asyncio.sleep(1)
def
await
my_coroutine():
print(“Coroutine started”)
print(“Coroutine ended”)
class
(self):
(self,exc_type,exc_value,
traceback):
print(“Entering context”)
print(“Exiting context”)
MyContext:
def
def
__enter__
__exit__
def
i **2
n :
i n :
my_generator( )
for in range( )
yield
def
while :
yield
count
T rue
(start , step):
star t
start += step
@ my_decorato r
my_function()def :
print(“ F unction is called”)
asyncio.run( ) my_coroutine()
with
print(“inside context”)
MyContext():
my_number =
nu m :
print(nu m )
my_generator( 5 )
for in my_numbers
my_function()
:Apply a decorator to a
:Defines a block of code to try
:Defines a block of code to run if an
:Defines a block of code to run regardless
:Automatically handles exceptions and closes
exception is raised
of whether an exception is raised or not
files when you’re done with them
:Open
:Pause the execution of a generator function
:Define a context
: G et the next v alue from a generator
:Preser v e the
function a file and automatically close it when done
and return a v alue
manager using a generator functionoriginal function’s metadata when creating a decorator
# Defining a decorator
# Defining a coroutine
# Defining a context manager
# Defining a generator
#I mplement iterator protocol
# Applying a decorator
#U sing a coroutine
#U sing a context manager
#U sing a generator
#C alling a decorated function
Example: Example:
Example 1 :
Example 2 :
[page 2]
Concurrency
THREADING
Multiprocessing
Asyncio
Multiprocessing
Import the module:
Import the module:
Create a process:
Import the module:
Import the module:
Create a process:
Start a thread:
Start a process:
Start a process:
Join a thread (wait for it to complete):
Join a process (wait for it to complete):
Join a process (wait for it to complete):
Get the current thread:
Use a for inter-process communication Queue
Use a lock to protect shared resources:
Create a thread: Run multiple async functions concurrently:
Create an async task:
Define an async function:
Run an async task:
Run an async function:
import threading
import multiprocessing
import asyncio
async function_name(): def
asyncio.run(main())
await function_name()
import multiprocessing
t.start()
p.start()
p.start()
q ueue = multiprocessing. Q ueue()
t. j oin()
p. j oin()
p. j oin()
cur_thrd = threading.current_thread()
loc k = threading. L oc k ()
t =
(target=function_name,
args=( ))
threading. T hread
arg 1, arg 2
p =
(target=functio
n_name, args=( ))
multiprocessing. P rocess
arg 1, arg 2
await asyncio.gather (function_name1(),
function_name2(), ...)
tas k =
asyncio.create_tas k(function_name())
p =
multiprocessing. P rocess (target=functio
n_name, args=( )) arg 1, arg 2
Its the a b ility to e x ecute multiple tasks
simultaneously , allowin g for more efficient use of
resources and impro v ed performance in certain
scenarios
It refers to the simultaneous e x ecution of multiple
tasks or su b -tasks to impro v e the o v erall performance
of a pro g ram . P aralleli z ation is particularly b eneficial
for lar g e datasets or comple x computations , as it
ena b les b etter utili z ation of a v aila b le hardware
resources , such as multi-core processor
Pa r all e lism
Use a for parallel e x ecution:P oo l
with
pool: results =
( )
multiprocessing. P ool
pool.map
( p rocesses= 4 )
as
function_name, itera bl e
[page 3]
Concurrent .futures
Joblib
Import the module:
Import the module:
Use Parallel and delayed for parallel execution:
Install the library:
Import Matplotlib
Create a simple line plot
Add labels and title
Change line style, color, and marker
Use a ThreadPoolExecutor for multi-threading:
Use a ProcessPoolExecutor for multi-processing:
Import the module:Use a for parallel execution: Pool
Use a for inter-process communication Queue
Pickle a Python object (serialize to file)
Unpickle a Python object (deserialize from file)
Pickle a Python object to a bytes object
Unpickle a Python object from a bytes object
Change the default pickling protocol
import pickle
pip install joblib
import as matplotlib.pyplot plt
from import joblib Parallel, delayed
queue = multiprocessing.Queue()
with as open ( )
file: pickle.dump(object_to_pickle, file)
" file _ name.pkl " , " wb "
with as open ( )
file: unpickled_object = pickle.load(file)
" file _ name.pkl " , " rb "
pickled_bytes =
(object_to_pickle)pickle.dumps
unpickled_object =
(pickled_bytes)pickle.loads
x = [ ]
y = [ ]
(x, y)
()
1 , 2 , 3 , 4
2 , 4 , 6 , 8
plt.plot
plt.show
plt.plot
plt. x label
plt.ylabel
plt.title
plt.show
(x, y)
( )
( )
( )
()
'X- a x is L abel '
'Y- a x is L abel '
' Plot T itle '
plt.plot(x, y, linestyle= ,
color= , marker= )
()
'--'
' red ' ' o '
plt.show
with as ( )
file:
pickle.dump(object_to_pickle, file,
protocol=pickle.HIGHEST_PROTOCOL)
open " file _ name.pkl " , " wb "
with
pool : results =
( )
multiprocessing.Pool
pool.map
(processes= 4 )
as
function_name, iterable
from import concurrent.futures
T hreadPool E x ecutor,
ProcessPool E x ecutor
It is the process of con v erting a Python object into a
byte stream, w hich can be stored in a file or
transmitted o v er a net w ork . This process is also
kno w n as object serialization .
P ick l ing
Matp lo t l ib
with
as
results =
T hreadPool E x ecutor
e x ecutor :
e x ecutor.map
(max_ w orkers= 4 )
(function_name, iterable)
with
as
results =
ProcessPool E x ecutor
e x ecutor :
e x ecutor.map
(max_ w orkers= 4 )
(function_name, iterable)
results =
for arg in
Parallel (n_jobs= 4 )
(delayed(function_name)(ar g )
iterable)
[page 4]
Create a scatter plot
Create a bar plot
Create a histogram
Create a pie chart
Create subplots
Save a plot to a file
Add a grid
Add a legend
Set axis limits
plt.scatter
plt.show
(x, y)
()
plt.bar
plt.show
(x, y)
()
data = [ ]
(data, bins= )
()
1, 2, 3, 3, 3, 4, 4, 5
plt.hist 5
plt.show
import as
import as
go
px
plotly.graph_objs
plotly.express
fig = (x=x, y=y)
px.scatter
fig.show()
fig = (x=x, y=y)
px.bar
fig.show()
data = [ ]
fig = (data, nbins=5)
1, 2, 3, 3, 3, 4, 4, 5
px.histogram
fig.show()
labels = [ ]
sizes = [ ]
fig = (names=labels, values=sizes)
'A', 'B', 'C', 'D'
25, 30, 20, 25
px.pie
fig.show()
fig. u p d ate_layo u t
fig.show()
(
title= ,
xaxis _ title= ,
yaxis _ title=
)
"P lot T itle "
"X- axis L abel "
"Y- axis L abel "
fig = (g o.Sc atte r (x=x, y=y,
m o de= ' lines + ma rk e r s ' ,
ma rk e r =di c t(size= 10 , co l or = 'r ed ' ),
line=di c t( co l or = ' blue ' , w idt h = 2 )))
go. F ig u re
fig.show()
import as pio
(fig, " pl o t . png " ,
f or mat= " png " )
plotly.io
pio.write_image
labels = [ ]
sizes = [ ]
(sizes, labels=labels,
aut o p c t= '%.1 f %%' )
()
'A', 'B', 'C', 'D'
25, 30, 20, 25
plt.pie
plt.show
x = [ ]
y = [ ]
fig = (x=x, y=y)
1, 2, 3, 4
2, 4, 6 , 8
px.li n e
fig.show()
fig, axs = ( 2 , 2 )
axs[ 0 , 0].
axs[ 0 , 1].
axs[ 1 , 0].
axs[ 1 , 1].
()
plt.s u bplots
plot(x, y )
scatter(x, y )
bar(x, y )
hist( d ata )
plt.show
plt.plot
plt.sa v efig
plt.show
(x, y)
( ' pl o t . png ' , dpi= 300 )
()
plt.plot
plt.gri d
plt.show()
(x, y)
( T r ue)
plt.plot
plt.lege nd
plt.show()
(x, y, label= ' L ine 1' )
(l oc = ' best ' )
plt.plot
plt.xlim
plt.ylim
plt.show()
(x, y)
( 0 , 5)
( 0 , 10 )
I nstall P lotl y
I mport P lotl y
Create a simple line plot
Create a scatter plot
Create a bar plot
Create a histogram
Create a pie chart
Add labels and title
Customi z e mar k er and line st y les
Save a plot to a file
pip i n stall plotly
Plotly
[page 5]
File Handling Regular Expressions
file.write(string) :Writes the string to the file
file.readline() :Reads one line from the file re.findall(pattern, string) :Finds all
occurrences of the given pattern in a string
re.sub(pattern,replacement,string) :
Replaces all occurrences of the given pattern with the
replacement in a string
re.compile(pattern) :Compiles a regular
expression pattern into a regular expression object for
faster matching
open(filename, mode) :Opens a file with the
given filename and mode
import re :Imports the re module for working with
regular expressions
wit h open(filename, mode) as file :Opens
a file and automaticall y closes it when done
re.searc h (pattern, string) : S earches a
string for a match to the given pattern
file.read(si z e) :Reads si z e b y tes from the file
( or the whole file if si z e is not specified )
re.matc h (pattern, string) : S earches the
beginning of a string for a match to the given pattern
file.readlines() :Reads all lines from the file
and returns them as a list of strings
file.writelines(strings) :Writes a list of
strings to the file
file.flus h () :Flushes an y buffered output to the
file
file.close() :Closes the file and frees up an y
s y stem resources used b y it
file.tell() :Returns the current position ( in b y tes
) in the file
file.see k (offset, from) :Changes the current
position in the file
os.remo v e( " filename " ): D eletes the file with the
specified name .
os.pat h .getsi z e( " filename " ):Returns the
si z e of the file in b y tes .
os.pat h .e x ists( " filename " ):Checks if the
file with the specified name exists .
os.pat h .e x ists( " filename " ):Checks if the
file with the specified name exists .
os.rename( " oldfilename " , " newfilename " )
:Renames the file with the old name to the new name .
import r e
if
else :
my_string=”The quick brown fox”
pattern=”fox”
=
:
print (“Match found)
print(“Match not found”)
result
result
re.searc h (pattern,m y_ string )
#U sing regular expression
Example 1:
r= re. ( ) compile r ’\ d \ d \ d -\ d \ d \ d -\ d \ d \ d \ d ’
m=r. ( ‘ My number is 415-555-4242 . ’ )searc h
print(f ’P hone num found : { }’ )mo.group()
# M atching regex objects
Example 2 :
[page 6]
NumPy
import numpy as np
Imports the NumPy library with the alias np
np.array(list)
Creates a NumPy array from a list
np.arange(start,stop,step)
Creates a NumPy array of evenly spaced values from start to stop with a step size of step
np.linspace(start, stop, num)
Creates a NumPy array of num evenly spaced values from start to stop
np.zeros(shape)
Creates a NumPy array of zeros with the given shape
np.ones(shape)
Creates a NumPy array of ones with the given shape
np.random.rand(shape)
Creates a NumPy array of random values with the given shape
np.reshape(array, new_shape)
Reshapes a NumPy arrar into the given new_shape
np.transpose(array)
Transposes a NumPy array
Pandas
PANDAS :A library for data manipulation and analysis
import pandas as pd
Imports Pandas and sets the alias pd
pd. S eries(data,inde x ,dtype)
Creates a Pandas series from data
pd. D ata f rame(data,inde x ,column)
Creates a Pandas dataframe from data
d f .head(n)
Returns the fi rst n rows of a dataframe
d f .tail(n)
Returns the last n rows of a dataframe
d f .shape
Returns the shape of a dataframe ( rows , columns )
d f .columns
Returns the column names of a dataframe
d f .inde x
Returns the inde x of a dataframe
d f .drop(column =[‘ column_name ’] )
D rops a column from a dataframe
d f[ d f[‘ column_name ’]==v alue ]
F ilters a dataframe by a speci fi c value in a column
[page 7]
Seaborn
SEABORN :A library for statistical data visualization
import seaborn as sns
Imports Seaborn and sets the alias sns
sns.set_style(‘whitegrid’)
Sets the Seaborn plot style
sns.countplot(x=’column_name’,data=df)
Creates a count plot of a categorial column in a dataframe
sns.displot(df[‘column_name’],kde=False)
Creates a histogram of a numerical column in a dataframe
sns.boxplot(x='column_name', y='target_column', data=df)
Creates a box plot of a numerical columngrouped by a categorical column in a dataframe
sns.scatterplot(x='column_name', y='target_column', data=df)
Creates a scatter plot of two numerical columns in a dataframe
sns.heatmap(df.corr(), cmap='coolwarm', annot=True)
Creates a heatmap of the correlation between all numerical columns in a dataframe, with annotations
sns.pairplot(df)
Creates a pair plot of all numerical columns in a dataframe
df.loc[row_label, col_label] :Accesses a value in a dataframe by row and column labels
df.iloc[row_index, col_index] :Accesses a value in a dataframe by row and column indices
df[df[‘column_name’]== v alue] : F ilters a dataframe by a speci fi c value in a column
df.groupby(‘column_name’).agg(function) : G roups a dataframe by a column and applies a function
df.rename(column= { ‘old_name’ : ’new name’ } ): R enames a column in a dataframe
df.sort_ v alues(by=’column_name’,ascending=True) :Sorts a dataframe by a column
df.describe() : R eturns statistics about a dataframe ( count, mean, std, min, max, q uartiles )
df.info() : R eturns information about a dataframe ( data types, non - null values )
[page 8]
Python Advanced Integration with ChatGPT
Importing Library
import openai
import request
import json
Setup the OpenAI API Key
open.api_key= ”YOUR_API_KEY”
Paramaters for GPT-3 API Call
Import Necessary Libraries
Setup the OpenAI API Key
Make the API Call
Parameters ={
“model”:”text-davinci-002”,
“prompt”:prompt,
“temperature”: ,
“max_tokens”: ,
“n”: ,
“st
0.5
100
1
op”:”\n”
}
Define Your Prompt
prompt = “Python code to sort a list of
numbers in ascending order”
Parse the response and extract the generated code:
E xec u te the generated code u s i ng exec () fu nct i on:
generated_code =
response. ch oi c es [ 0 ] .te x t.strip ()
my_list = [3,4,7,2, 5 ,9,4,2, 5 ,9,9, 5 ]
exec ( generated_code )
print ( my_list )
response =
openai C omp l etion. c reate (** parameters )
[page 9]
Use the OpenAI API to interact with ChatGPT
Incorporate the ChatGPT Response into Your ETL Process
Use the CodeGPT VSCode Extension
Utilizing as a Tool for ETL ChatGPT
import
print
openai
openai.api_key =
prompt =
response = openai.Completion.create(
engine= ,
prompt=prompt,
temperature= ,
max_tokens= ,
top_p= ,
frequency_penalty= ,
presence_penalty= ,
)
(response.choices[ ].text.strip())
"your_openai_api_key"
"Transform a list of dictionaries into a Pandas DataFrame."
"text-davinci-002"
0.5
150
1
0
0
0
import as
print
pandas pd
data = [
{ : , : , : },
{ : , : , : },
{ : , : , : }
]
df = pd.DataFrame(data)
(df)
'name' 'John' 'age' 'city' 'New York'
'name' 'Jane' 'age' 'city' 'San Francisco'
'name' 'Mike' 'age' 'city' 'Los Angeles'
28
24
22
mean_age = df[ ].
(mean_age)
'age' mean()
print
# How to calculate the mean of 'age' column in a Pandas DataFrame?
Once the response from ChatGPT is got, we can use it as a guide for the ETL process using Pandas.
Type your question or task description as a comment, and CodeGPT will suggest code snippets relevant to your query.
CodeGPT may suggest the following snippet:
Transforming a list of dictionaries into a Pandas DataFrame based on ChatGPT's response:
[page 10]
Different Use-Cases for ETL with ChatGPT and Pandas
Read data from a CSV file and filter rows based on a condition:
Merge two DataFrames on a common column:
CodeGPT may suggest:
CodeGPT may suggest:
# Read data from a CSV file and filter rows where age is greater than 25
# Merge two DataFrames on a common column 'id'
df = pd.read_csv( )
filtered_df = df[df[ ] > ]
(filtered_df)
'data.csv'
'age' 25
print
df1 = pd.DataFrame({ : [ ], : [ ]})
df2 = pd.DataFrame({ : [ ], : [ ]})
merged_df = (df1, df2, on='id')
(merged)
'id' 'name' 'A', 'B', 'C'
'id' 'age'
1, 2, 3
1, 2, 3 28, 24, 22
pd.merge
print