Files and Internet Scrapping Python Cheat Sheet

A Programming › Python cheat sheet in English by leenmajz

This is a cheat sheet for programming in python

Web Beginner Random Choices

3,528 views 781 downloads 15 Mar 2024 History

PRs

Download

Unicode

Code point higher than 127
"'\u0915'" or "क"

File Modes

	r	r+	w	w+	a	a+
read	*	*		*		*
write		*	*	*	*	*
create			*	*	*	*
truncate			*	*
position at start	*	*	*	*
position at end					*	*

Reading a File

Opens and closing a file
file = open(file_name, encoding = 'utf-8', 'r')
with open(file_name, "r") as file:
file.close()
Reading a file
Reads entire file as a string	___content=file.read()
Reads the first 10 charcters	___content = file.read(10)
Stores contents as a list of lines	___file.readlines()
Read next line as a string	___file.readline()
Iterating through lines
	for line in lines:
strip to remove newline charc	___print(line.strip())
Writing to a text file
	with open(file_name, 'w') as file:
writing one line	___file.write("Hello, World!\n")
writing list of lines	___file.writelines(list)
Appending to a text file
with open(file_name, 'a') as file:
___file.write("\nAppending a new line.")
Checking file existence
import os
os.path.exists(file_name)
Handling file exceptions
try:
___with open(file) as f:
_____content = f.read()
___print(content)
except FileNotFoundError:
___print(""File not found")
except Exception as e:
___print(f"An error occurred: {e}")
Telling/seeking in a file
	with open(file) as f:
Gives current position	___f.tell()
Move the cursor to the beginning	___f.seek(0)

csv Library

import csv
Reading CSV file
returns a list	content = csv.reader(file, delimiter = ";")
	data = list(content)
Writing to CSV file
	writer = csv.writer(file)
	writer.writerows(list)

Loading JSONs

import json
Loading json file
json_data = json.loads(json_file)
Change json to string
string = json.dumps(data, indent=2)

Random library

	import random
Returns a random float in the range[0,0,1)	random.random()
Return a random float in the range[a,b]	random.uniform(a,b)
Returns an integer in the range [0,b)	random.randrange(b)
Returns an integer in the range [a,b) skipping c steps	random.randrange(a,b,c)
Returns an integer in the range [a,b]	random.randint(a,b)
Randomly change position of elements	print (random.shuffle(luck))

Random Choices

Uniformly randomly picks one item from a list	random.choice(list)
	list[randrange(4)]
Selects k items without repetition	random.sample(list, k=2)
Selects k items with repetition	random.choices(list, k=2)
Selects k items without uniformity may repeat	random.choices(list, weights=[10,150,20], k =2)

These methods also work on strings

Raw Strings

Raw Strings
r-strings	r"A raw string"
only a single backslash not valid
odd number of ending backslash not valid

Regular Expressions- Patterns

Regular - Expression Patterns (continued)

*	Match its preceding element zero or more times.
+	Match its preceding element one or more times.
?	Match its preceding element zero or one time.
{n}	Match its preceding element exactly n times.
{n ,m}	Match its preceding element from n to m times.

re Methods

Searches the string for a match and returns a Match object	re.match(pattern, string)
Searches for the first occurrence of the pattern anywhere in the string	re.search(pattern,string)
Finds all occurrences of the pattern in the string returns a list	re.findall(pattern,string)
Returns an iterator yielding match objects for all matches.	re.finditer(pattern, string)
Replaces matching substrings with new string for all occurrences or a specified number	re.sub(pattern, replacement, string)
Splits the string where there is a match and returns list of strings based on splits	re.split(pattern, string)

Get requests using requests

import requests
url = "https://www.wikipedia.org/"
r = requests.get(url)   
text = r.text

Webscrapping

from bs4 import BeautifulSoup

# Parse HTML stored as a string
# 'html5lib' 'html.parser' or 'lxml'
soup = BeautifulSoup(html, 'html5lib')

# Returns formatted html
soup.prettify()

# Find the first instance of an HTML tag
soup.find(tag, attrs={"class":"__"})

# Find all instances of an HTML tag
soup.find_all(tag, attrs={"class":"__"})

Get requests using urllib

from urllib.request import urlopen, Request
url = "https://www.wikipedia.org/"
request = Request(url)
response = urlopen(request)
html = response.read()
response.close()

Higher Order Functions

# min/max

max(iterable[, default=obj, key=func])
min(iterable[, default=obj, key=func])

a = min([12,"apple",223,"A","B"],key= lambda c: len(str(c)))
# Output: "A"   (minimum value in the list based on len(str) of the list object)


students = [{"name":"Saint", "age":"25"},
{"name":"Watson", "age":"35"},
{"name":"Karlson", "age":"21"},
{"name":"Kenzo", "age":"15"}]

youngest= min(students, key= lambda x:x["age"] ) # Output: {"name":"Kenzo", "age":"15"}]
oldest = max(students, key= lambda x:x["age"] )  # Output: {"name":"Watson", "age":"35"}



#sorted

sorted(iterable, key=func, reverse=reverse)

L =["apple", "ban", "dog", "aeroplane"]
print(sorted(L,key=len))  # Sort based on lenght of string


#map 

map(function, iterable, ...)

applies a function to the iterable and returns a mapped object 

Use print(list(map)) to print the value!

# A function to return the square of n
def addition(n):
   return n**2

# Some iterable
list = [1,2,3,4]

#map the function with the iterable and apply list to the map object

print(list(map(addition,list)))  # Output: [1,4,9,16]


# Filter

map(function, iterable, ...)

The filter runs through each element of iterable and applies function to it.
It filters out list elements for which function doesnt give a True value


seq = [0, 1, 2, 3, 5, 8, 13]
 
# result filters out non odd numbers
result = filter(lambda x: x % 2 != 0, seq)
print(list(result)) # Output: [1,3,5,13]

Badges

Downloaded from Unusual Locations × 1

Comments

No comments yet.