print()
Shows something on the screen.
print("Hello")
print("Total:", 25)
print("a", "b", sep="-")Hello Total: 25 a-b
Quick reference
Every function from the course, grouped by the class it was taught in. Each one has a short explanation and a working example, and the output shown is the real result of running that code.
Sessions D02 to D25. The language itself — types, strings, collections, loops, functions, classes and files.
85 functionsShows something on the screen.
print("Hello")
print("Total:", 25)
print("a", "b", sep="-")Hello Total: 25 a-b
Takes typed text from the user. Always returns a string.
# name = input("Enter name: ")
# Whatever the user types comes back as TEXT
name = "Rahul" # pretend this came from input()
print(name, type(name))Rahul <class 'str'>
Converts the text from input into a number so you can do maths.
age_text = "25" # input() always gives text age = int(age_text) print(age + 5)
30
Tells you what kind of value something is.
print(type(10))
print(type(3.5))
print(type("hi"))
print(type(True))<class 'int'> <class 'float'> <class 'str'> <class 'bool'>
Converts a value into a whole number. Cuts off the decimal.
print(int("42"))
print(int(9.87))42 9
Converts a value into a decimal number.
print(float("3.5"))
print(float(10))3.5 10.0
Converts any value into text.
print(str(99) + " marks")
99 marks
Converts a value into True or False. Empty things are False.
print(bool(1), bool(0))
print(bool("hi"), bool(""))
print(bool([]), bool([1]))True False True False False True
Rounds a decimal number.
print(round(3.7)) print(round(3.14159, 2))
4 3.14
Gives the value without the minus sign.
print(abs(-15)) print(abs(15))
15 15
Basic maths. // gives whole division, % gives remainder, ** is power.
a, b = 17, 5 print(a + b, a - b, a * b) print(a / b) # normal division -> decimal print(a // b) # floor division -> whole number print(a % b) # remainder print(a ** 2) # power
22 12 85 3.4 3 2 289
Compares two values. Always gives True or False.
print(5 > 3, 5 == 5, 5 != 3) print(5 <= 4)
True True True False
Joins conditions together.
age = 22 print(age > 18 and age < 30) print(age < 18 or age > 20) print(not (age == 22))
True True False
Counts how many characters are in the text.
s = "Data Analytics" print(len(s))
14
Changes the case of the text.
s = "Data" print(s.upper()) print(s.lower())
DATA data
title() capitalises every word. capitalize() only the first.
s = "rahul kumar sharma" print(s.title()) print(s.capitalize())
Rahul Kumar Sharma Rahul kumar sharma
Removes spaces from the start and end.
s = " Rahul "
print("[" + s.strip() + "]")
print("[" + s.lstrip() + "]")
print("[" + s.rstrip() + "]")[Rahul] [Rahul ] [ Rahul]
Breaks text into a list. Splits on spaces by default.
s = "Rahul,Priya,Amit"
print(s.split(","))
print("a b c".split())['Rahul', 'Priya', 'Amit'] ['a', 'b', 'c']
Joins a list of text into one string.
names = ["Rahul", "Priya", "Amit"]
print(", ".join(names))
print("-".join("abc"))Rahul, Priya, Amit a-b-c
Swaps one piece of text for another.
s = "I like tea"
print(s.replace("tea", "coffee"))I like coffee
Finds where a piece of text starts. find() gives -1 if missing, index() gives an error.
s = "Data Analytics"
print(s.find("Analy"))
print(s.find("Python"))5 -1
Counts how many times something appears.
s = "banana"
print(s.count("a"))3
Checks how the text begins or ends.
f = "report.csv"
print(f.endswith(".csv"))
print(f.startswith("rep"))True True
Checks what the text is made of.
print("12345".isdigit())
print("abc".isalpha())
print("abc123".isalpha())True True False
Takes a part of the text using [start:stop:step].
s = "Data Analytics" print(s[0]) print(s[0:4]) print(s[5:]) print(s[-1]) print(s[::-1]) # reversed
D Data Analytics s scitylanA ataD
Puts variables inside text. The easiest way to build a message.
name, marks = "Rahul", 87
print(f"{name} scored {marks} marks")
print(f"Average: {87/3:.2f}")Rahul scored 87 marks Average: 29.00
The older way of putting variables inside text.
print("{} scored {}".format("Rahul", 87))Rahul scored 87
A list holds many values in order. It can be changed.
nums = [10, 20, 30] mixed = [1, "two", 3.0, True] empty = [] print(nums, mixed, empty)
[10, 20, 30] [1, 'two', 3.0, True] []
Get one item, or a part of the list.
nums = [10, 20, 30, 40, 50] print(nums[0]) print(nums[-1]) print(nums[1:4]) print(nums[::2])
10 50 [20, 30, 40] [10, 30, 50]
Adds ONE item to the end.
nums = [1, 2] nums.append(3) print(nums) nums.append([4, 5]) # adds the whole list as one item print(nums)
[1, 2, 3] [1, 2, 3, [4, 5]]
Adds MANY items to the end, one by one.
nums = [1, 2] nums.extend([3, 4]) print(nums)
[1, 2, 3, 4]
Adds an item at a chosen position.
nums = [10, 30] nums.insert(1, 20) print(nums)
[10, 20, 30]
Removes the first matching value.
nums = [10, 20, 30, 20] nums.remove(20) print(nums)
[10, 30, 20]
Removes by position and gives the item back. Last item if you give nothing.
nums = [10, 20, 30] last = nums.pop() first = nums.pop(0) print(last, first, nums)
30 10 [20]
Arranges the list in order. Changes the original list.
nums = [30, 10, 20] nums.sort() print(nums) nums.sort(reverse=True) print(nums)
[10, 20, 30] [30, 20, 10]
Same as sort but gives a NEW list. The original stays as it was.
nums = [30, 10, 20] new = sorted(nums) print(new) print(nums) # unchanged
[10, 20, 30] [30, 10, 20]
Flips the order of the list.
nums = [1, 2, 3] nums.reverse() print(nums)
[3, 2, 1]
Counts a value, or finds its position.
nums = [10, 20, 10, 30] print(nums.count(10)) print(nums.index(20))
2 1
Quick summaries of a list of numbers.
nums = [10, 20, 30, 40] print(len(nums), sum(nums)) print(min(nums), max(nums)) print(sum(nums) / len(nums)) # average
4 100 10 40 25.0
Makes a separate copy. Without it, both names point to the same list.
a = [1, 2, 3] b = a # SAME list c = a.copy() # NEW list a.append(4) print(b) print(c)
[1, 2, 3, 4] [1, 2, 3]
Builds a new list in one line. Very common in data work.
nums = [1, 2, 3, 4, 5] squares = [n * n for n in nums] evens = [n for n in nums if n % 2 == 0] print(squares) print(evens)
[1, 4, 9, 16, 25] [2, 4]
Like a list, but it CANNOT be changed after you make it.
t = (10, 20, 30) print(t, len(t), t[0])
(10, 20, 30) 3 10
Trying to change one gives an error. That safety is the point.
t = (1, 2, 3)
try:
t[0] = 99
except TypeError as e:
print("Error:", e)Error: 'tuple' object does not support item assignment
Puts each item into its own variable.
point = (10, 20) x, y = point print(x, y)
10 20
Stores key–value pairs. You look things up by name, not by position.
student = {"name": "Rahul", "marks": 87, "city": "Bangalore"}
print(student)
print(student["name"]){'name': 'Rahul', 'marks': 87, 'city': 'Bangalore'}
RahulAssign to a key to add it or update it.
d = {"a": 1}
d["b"] = 2 # add
d["a"] = 99 # change
print(d){'a': 99, 'b': 2}Safely reads a key. Gives None (or your default) instead of an error.
d = {"a": 1}
print(d.get("a"))
print(d.get("z"))
print(d.get("z", 0))1 None 0
Gets the keys, the values, or both together.
d = {"a": 1, "b": 2}
print(list(d.keys()))
print(list(d.values()))
print(list(d.items()))['a', 'b']
[1, 2]
[('a', 1), ('b', 2)]items() gives you the key and value together.
marks = {"Rahul": 87, "Priya": 92}
for name, m in marks.items():
print(name, "scored", m)Rahul scored 87 Priya scored 92
Removes a key.
d = {"a": 1, "b": 2}
d.pop("a")
print(d){'b': 2}Merges another dictionary in.
d = {"a": 1}
d.update({"b": 2, "c": 3})
print(d){'a': 1, 'b': 2, 'c': 3}Holds only UNIQUE values. Order is not kept.
s = {1, 2, 2, 3, 3, 3}
print(s)
print(set([10, 20, 10])){1, 2, 3}
{10, 20}Adds or removes one value.
s = {1, 2}
s.add(3)
s.remove(1)
print(s){2, 3}Compares two sets.
a = {1, 2, 3}
b = {3, 4, 5}
print(a | b) # union - everything
print(a & b) # intersection - common
print(a - b) # difference - only in a{1, 2, 3, 4, 5}
{3}
{1, 2}The most common real use of a set.
names = ["Rahul", "Priya", "Rahul", "Amit"] print(list(set(names)))
['Amit', 'Priya', 'Rahul']
Runs different code depending on the answer.
marks = 78
if marks >= 90:
print("A grade")
elif marks >= 75:
print("B grade")
else:
print("C grade")B grade
Checks whether a value is present.
names = ["Rahul", "Priya"]
print("Rahul" in names)
print("Amit" in names)
print("ta" in "Data")True False True
Repeats once for every item.
for name in ["Rahul", "Priya"]:
print("Hello", name)Hello Rahul Hello Priya
Makes a sequence of numbers to loop over.
print(list(range(5)))
print(list(range(2, 6)))
print(list(range(0, 10, 2)))
for i in range(3):
print(i)[0, 1, 2, 3, 4] [2, 3, 4, 5] [0, 2, 4, 6, 8] 0 1 2
Keeps repeating while the condition stays true.
count = 1
while count <= 3:
print(count)
count = count + 11 2 3
break stops the loop. continue skips to the next round.
for i in range(6):
if i == 4:
break
if i % 2 == 0:
continue
print(i)1 3
Gives the position AND the item together.
names = ["Rahul", "Priya"]
for i, name in enumerate(names):
print(i, name)0 Rahul 1 Priya
Loops through two lists side by side.
names = ["Rahul", "Priya"]
marks = [87, 92]
for n, m in zip(names, marks):
print(n, m)Rahul 87 Priya 92
Creates your own reusable block of code.
def greet(name):
return "Hello " + name
print(greet("Rahul"))Hello Rahul
Sends a value back. Without it you get None.
def add(a, b):
return a + b
def add_no_return(a, b):
total = a + b # nothing sent back
print(add(2, 3))
print(add_no_return(2, 3))5 None
Gives a parameter a fallback if nothing is passed.
def greet(name, greeting="Hello"):
return greeting + " " + name
print(greet("Rahul"))
print(greet("Priya", "Good morning"))Hello Rahul Good morning Priya
Accepts any number of values.
def total(*nums):
return sum(nums)
print(total(1, 2))
print(total(1, 2, 3, 4, 5))3 15
Accepts any number of named values.
def show(**info):
for k, v in info.items():
print(k, "=", v)
show(name="Rahul", city="Bangalore")name = Rahul city = Bangalore
A tiny one-line function with no name.
square = lambda x: x * x print(square(5)) pairs = [(1, "b"), (3, "a"), (2, "c")] print(sorted(pairs, key=lambda p: p[1]))
25 [(3, 'a'), (1, 'b'), (2, 'c')]
Applies a function to every item.
nums = [1, 2, 3] print(list(map(lambda x: x * 10, nums))) print(list(map(str, nums)))
[10, 20, 30] ['1', '2', '3']
Keeps only the items that pass a test.
nums = [1, 2, 3, 4, 5, 6] print(list(filter(lambda x: x % 2 == 0, nums)))
[2, 4, 6]
Creates a file and writes into it.
with open("demo.txt", "w") as f:
f.write("Line one\n")
f.write("Line two\n")
print("file written")file written
Reads the whole file as one string.
with open("demo.txt", "w") as f:
f.write("Line one\nLine two\n")
with open("demo.txt", "r") as f:
print(f.read())Line one Line two
Reads the file into a list of lines.
with open("demo.txt", "w") as f:
f.write("a\nb\nc\n")
with open("demo.txt") as f:
lines = f.readlines()
print(lines)
print([l.strip() for l in lines])['a\n', 'b\n', 'c\n'] ['a', 'b', 'c']
"a" adds to the end instead of erasing the file.
with open("demo.txt", "w") as f:
f.write("first\n")
with open("demo.txt", "a") as f:
f.write("second\n")
with open("demo.txt") as f:
print(f.read())first second
Catches an error so the program does not stop.
try:
x = 10 / 0
except ZeroDivisionError:
print("Cannot divide by zero")Cannot divide by zero
Shows what actually went wrong.
try:
n = int("abc")
except ValueError as e:
print("Error:", e)Error: invalid literal for int() with base 10: 'abc'
Runs whether there was an error or not.
try:
print("trying")
except:
print("failed")
finally:
print("always runs")trying always runs
A blueprint for making objects. __init__ runs when you create one.
class Student:
def __init__(self, name, marks):
self.name = name
self.marks = marks
s = Student("Rahul", 87)
print(s.name, s.marks)Rahul 87
A function that belongs to the class.
class Student:
def __init__(self, name, marks):
self.name = name
self.marks = marks
def grade(self):
return "Pass" if self.marks >= 40 else "Fail"
s = Student("Rahul", 87)
print(s.grade())Pass
A new class takes everything from an existing one.
class Person:
def __init__(self, name):
self.name = name
def greet(self):
return "Hi, I am " + self.name
class Student(Person):
def study(self):
return self.name + " is studying"
s = Student("Rahul")
print(s.greet())
print(s.study())Hi, I am Rahul Rahul is studying
Brings in code someone else wrote.
import math print(math.sqrt(16)) print(math.pi)
4.0 3.141592653589793
Brings in just one piece.
from math import sqrt, floor print(sqrt(25), floor(3.9))
5.0 3
Gives the module a short name. This is why pandas is pd.
import math as m print(m.sqrt(9))
3.0
Makes random numbers.
import random random.seed(1) # seed makes it repeatable print(random.randint(1, 10)) print(random.choice(["a", "b", "c"]))
3 c
Sessions D25 to D30. Arrays, indexing, aggregation, boolean slicing and reshaping.
50 functionsTurns a list into a NumPy array.
a = np.array([10, 20, 30]) print(a) print(type(a))
[10 20 30] <class 'numpy.ndarray'>
A list of lists becomes a table with rows and columns.
b = np.array([[1, 2, 3],
[4, 5, 6]])
print(b)
print(b.shape) # (rows, columns)[[1 2 3] [4 5 6]] (2, 3)
Makes an array filled with 0.
print(np.zeros(4)) print(np.zeros((2, 3)))
[0. 0. 0. 0.] [[0. 0. 0.] [0. 0. 0.]]
Makes an array filled with 1.
print(np.ones(3)) print(np.ones((2, 2)))
[1. 1. 1.] [[1. 1.] [1. 1.]]
Makes an array filled with any value you choose.
print(np.full(4, 7)) print(np.full((2, 2), 9))
[7 7 7 7] [[9 9] [9 9]]
Makes an identity matrix - 1 on the diagonal, 0 everywhere else.
print(np.eye(3)) print(np.identity(2))
[[1. 0. 0.] [0. 1. 0.] [0. 0. 1.]] [[1. 0.] [0. 1.]]
Makes numbers in a range. Like range(), but gives an array.
print(np.arange(5)) print(np.arange(2, 10, 2))
[0 1 2 3 4] [2 4 6 8]
Makes a fixed COUNT of evenly spaced numbers between two values.
print(np.linspace(0, 10, 5)) print(np.linspace(0, 1, 3))
[ 0. 2.5 5. 7.5 10. ] [0. 0.5 1. ]
Random decimals between 0 and 1.
np.random.seed(1) print(np.random.rand(3)) print(np.random.rand(2, 2))
[4.17022005e-01 7.20324493e-01 1.14374817e-04] [[0.30233257 0.14675589] [0.09233859 0.18626021]]
Random whole numbers in a range.
np.random.seed(1) print(np.random.randint(1, 100, 5))
[38 13 73 10 76]
Fixes the randomness so you get the same numbers every time.
np.random.seed(42) print(np.random.randint(1, 10, 4)) np.random.seed(42) print(np.random.randint(1, 10, 4)) # same again
[7 4 8 5] [7 4 8 5]
Rows and columns.
a = np.array([[1, 2, 3], [4, 5, 6]]) print(a.shape) print(np.array([1,2,3,4]).shape)
(2, 3) (4,)
How many dimensions - 1 for a line, 2 for a table.
print(np.array([1, 2, 3]).ndim) print(np.array([[1, 2], [3, 4]]).ndim)
1 2
Total number of items.
a = np.array([[1, 2, 3], [4, 5, 6]]) print(a.size)
6
What kind of numbers are inside.
print(np.array([1, 2, 3]).dtype) print(np.array([1.5, 2.5]).dtype) print(np.array(["a", "b"]).dtype)
int64 float64 <U1
Changes the data type.
a = np.array([1.7, 2.9, 3.1]) print(a.astype(int)) print(np.array([1, 0, 1]).astype(bool))
[1 2 3] [ True False True]
Get one value by its position. Counting starts at 0.
a = np.array([10, 20, 30, 40]) print(a[0]) print(a[-1])
10 40
Take a part using [start:stop].
a = np.array([10, 20, 30, 40, 50]) print(a[1:4]) print(a[:3]) print(a[::2])
[20 30 40] [10 20 30] [10 30 50]
Use [row, column].
b = np.array([[1, 2, 3],
[4, 5, 6]])
print(b[0, 2])
print(b[1, 0])3 4
A colon means "everything" in that direction.
b = np.array([[1, 2, 3],
[4, 5, 6]])
print(b[0, :]) # first row
print(b[:, 1]) # second column
print(b[:, 0:2]) # first two columns[1 2 3] [2 5] [[1 2] [4 5]]
Keeps only the values where the condition is True.
a = np.array([10, 25, 3, 40, 7]) print(a > 20) print(a[a > 20])
[False True False True False] [25 40]
Use & for and, | for or. Each condition needs brackets.
a = np.array([10, 25, 3, 40, 7]) print(a[(a > 5) & (a < 30)]) print(a[(a < 5) | (a > 35)])
[10 25 7] [ 3 40]
Pick several positions at once using a list.
a = np.array([10, 20, 30, 40, 50]) print(a[[0, 2, 4]])
[10 30 50]
Rearranges the same values into a new shape.
a = np.arange(6) print(a) print(a.reshape(2, 3)) print(a.reshape(3, 2))
[0 1 2 3 4 5] [[0 1 2] [3 4 5]] [[0 1] [2 3] [4 5]]
-1 means "work it out for me".
a = np.arange(6) print(a.reshape(-1, 2)) print(a.reshape(2, -1))
[[0 1] [2 3] [4 5]] [[0 1 2] [3 4 5]]
Turns a table back into a single line.
b = np.array([[1, 2], [3, 4]]) print(b.ravel()) print(b.flatten())
[1 2 3 4] [1 2 3 4]
Flips rows into columns.
b = np.array([[1, 2, 3], [4, 5, 6]]) print(b) print(b.T)
[[1 2 3] [4 5 6]] [[1 4] [2 5] [3 6]]
Joins arrays together.
a = np.array([1, 2]) b = np.array([3, 4]) print(np.concatenate([a, b]))
[1 2 3 4]
Stacks arrays on top of each other (vertically).
a = np.array([1, 2, 3]) b = np.array([4, 5, 6]) print(np.vstack([a, b]))
[[1 2 3] [4 5 6]]
Puts arrays side by side (horizontally).
a = np.array([1, 2, 3]) b = np.array([4, 5, 6]) print(np.hstack([a, b]))
[1 2 3 4 5 6]
Cuts one array into several pieces.
a = np.arange(6) print(np.split(a, 3))
[array([0, 1]), array([2, 3]), array([4, 5])]
Maths happens on every value at once. No loop needed.
a = np.array([10, 20, 30]) print(a + 5) print(a * 2) print(a / 10) print(a ** 2)
[15 25 35] [20 40 60] [1. 2. 3.] [100 400 900]
Two arrays of the same shape work position by position.
a = np.array([1, 2, 3]) b = np.array([10, 20, 30]) print(a + b) print(a * b)
[11 22 33] [10 40 90]
Common maths functions applied to every value.
a = np.array([1, 4, 9, 16]) print(np.sqrt(a)) print(np.round(np.log(np.array([1.0, 2.718])), 2))
[1. 2. 3. 4.] [0. 1.]
Rounding. floor goes down, ceil goes up.
a = np.array([1.2, 2.7, 3.5]) print(np.round(a)) print(np.floor(a)) print(np.ceil(a))
[1. 3. 4.] [1. 2. 3.] [2. 3. 4.]
A small array stretches automatically to fit a bigger one.
b = np.array([[1, 2, 3],
[4, 5, 6]])
print(b + np.array([10, 20, 30]))
print(b * 2)[[11 22 33] [14 25 36]] [[ 2 4 6] [ 8 10 12]]
Adds everything up.
a = np.array([10, 20, 30]) print(a.sum()) print(np.sum(a))
60 60
The average and the middle value.
a = np.array([10, 20, 30, 100]) print(a.mean()) print(np.median(a))
40.0 25.0
How spread out the numbers are.
a = np.array([10, 20, 30, 40]) print(round(a.std(), 2)) print(a.var())
11.18 125.0
The smallest and largest value.
a = np.array([10, 5, 30]) print(a.min(), a.max())
5 30
The POSITION of the smallest or largest value.
a = np.array([10, 5, 30]) print(a.argmin()) print(a.argmax())
1 2
axis=0 goes down the columns. axis=1 goes across the rows.
b = np.array([[1, 2, 3],
[4, 5, 6]])
print(b.sum()) # everything
print(b.sum(axis=0)) # down each column
print(b.sum(axis=1)) # across each row21 [5 7 9] [ 6 15]
A running total.
a = np.array([10, 20, 30]) print(a.cumsum())
[10 30 60]
Multiplies everything together.
a = np.array([1, 2, 3, 4]) print(a.prod())
24
Picks one value or another depending on a condition.
a = np.array([10, 25, 3, 40]) print(np.where(a > 20, "high", "low")) print(np.where(a > 20, a, 0))
['low' 'high' 'low' 'high'] [ 0 25 0 40]
Arranges values in order.
a = np.array([30, 10, 20]) print(np.sort(a))
[10 20 30]
Gives the POSITIONS that would sort the array.
a = np.array([30, 10, 20]) print(np.argsort(a))
[1 2 0]
Gives the distinct values, in order.
a = np.array([3, 1, 2, 3, 1]) print(np.unique(a)) print(np.unique(a, return_counts=True))
[1 2 3] (array([1, 2, 3]), array([2, 1, 2]))
any = at least one True. all = every one True.
a = np.array([10, 25, 3]) print((a > 20).any()) print((a > 20).all())
True False
Finds missing values.
a = np.array([1.0, np.nan, 3.0]) print(np.isnan(a)) print(a[~np.isnan(a)]) # ~ means NOT
[False True False] [1. 3.]
Sessions D31 to D37. Series and DataFrames, filtering, sorting, missing values, duplicates and merging.
77 functionsA Series is ONE column of data with an index.
s = pd.Series([10, 20, 30]) print(s) print(s[0], s.sum())
0 10 1 20 2 30 dtype: int64 10 60
You can give your own labels instead of 0,1,2.
s = pd.Series([87, 92], index=['Rahul', 'Priya']) print(s) print(s['Rahul'])
Rahul 87 Priya 92 dtype: int64 87
A DataFrame is a full table. Each key becomes a column.
data = {'name': ['Rahul', 'Priya'], 'marks': [87, 92]}
d = pd.DataFrame(data)
print(d)name marks 0 Rahul 87 1 Priya 92
Give the column names separately.
rows = [['Rahul', 87], ['Priya', 92]] d = pd.DataFrame(rows, columns=['name', 'marks']) print(d)
name marks 0 Rahul 87 1 Priya 92
Loads a CSV file into a DataFrame. The most used function in pandas.
df.to_csv('demo.csv', index=False) # make a file first
d = pd.read_csv('demo.csv')
print(d.head(3))name dept city age exp salary rating 0 Rahul Tech Bangalore 28.0 4 72000 Good 1 Priya Sales Chennai 34.0 10 95000 Excellent 2 Amit Tech Bangalore 25.0 2 58000 Poor
Common settings you will need.
df.to_csv('demo.csv', index=False)
print(pd.read_csv('demo.csv', nrows=2))
print(pd.read_csv('demo.csv', usecols=['name','salary']).head(2)) name dept city age exp salary rating
0 Rahul Tech Bangalore 28.0 4 72000 Good
1 Priya Sales Chennai 34.0 10 95000 Excellent
name salary
0 Rahul 72000
1 Priya 95000Saves a DataFrame to a CSV file. index=False stops the extra column.
df.head(2).to_csv('out.csv', index=False)
print(open('out.csv').read())name,dept,city,age,exp,salary,rating Rahul,Tech,Bangalore,28.0,4,72000,Good Priya,Sales,Chennai,34.0,10,95000,Excellent
Shows the first or last few rows. Always start here.
print(df.head(3)) print(df.tail(2))
name dept city age exp salary rating
0 Rahul Tech Bangalore 28.0 4 72000 Good
1 Priya Sales Chennai 34.0 10 95000 Excellent
2 Amit Tech Bangalore 25.0 2 58000 Poor
name dept city age exp salary rating
4 Vikram Sales Chennai 30.0 6 68000 Average
5 Divya Tech Pune NaN 8 88000 GoodHow many rows and columns.
print(df.shape)
print("rows:", df.shape[0], "columns:", df.shape[1])(6, 7) rows: 6 columns: 7
A summary: column names, how many values are filled, and the data types.
df.info()
<class 'pandas.core.frame.DataFrame'> RangeIndex: 6 entries, 0 to 5 Data columns (total 7 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 name 6 non-null object 1 dept 6 non-null object 2 city 6 non-null object 3 age 5 non-null float64 4 exp 6 non-null int64 5 salary 6 non-null int64 6 rating 6 non-null object dtypes: float64(1), int64(2), object(4) memory usage: 464.0+ bytes
Summary statistics for the number columns.
print(df.describe())
age exp salary count 5.000000 6.000000 6.000000 mean 31.600000 7.500000 83500.000000 std 6.188699 4.636809 22394.195677 min 25.000000 2.000000 58000.000000 25% 28.000000 4.500000 69000.000000 50% 30.000000 7.000000 80000.000000 75% 34.000000 9.500000 93250.000000 max 41.000000 15.000000 120000.000000
The column names, the row labels, and the data types.
print(list(df.columns)) print(df.dtypes)
['name', 'dept', 'city', 'age', 'exp', 'salary', 'rating'] name object dept object city object age float64 exp int64 salary int64 rating object dtype: object
The data as a plain NumPy array, without labels.
print(df[['exp','salary']].values)
[[ 4 72000] [ 10 95000] [ 2 58000] [ 15 120000] [ 6 68000] [ 8 88000]]
Shows random rows instead of the first few.
print(df.sample(2, random_state=1))
name dept city age exp salary rating 2 Amit Tech Bangalore 25.0 2 58000 Poor 1 Priya Sales Chennai 34.0 10 95000 Excellent
One pair of brackets gives a Series.
print(df['name'])
0 Rahul 1 Priya 2 Amit 3 Sneha 4 Vikram 5 Divya Name: name, dtype: object
Two pairs of brackets give a DataFrame.
print(df[['name', 'salary']])
name salary 0 Rahul 72000 1 Priya 95000 2 Amit 58000 3 Sneha 120000 4 Vikram 68000 5 Divya 88000
Selects by LABEL. Row labels and column names.
print(df.loc[0]) print(df.loc[0:2, ['name','salary']])
name Rahul
dept Tech
city Bangalore
age 28.0
exp 4
salary 72000
rating Good
Name: 0, dtype: object
name salary
0 Rahul 72000
1 Priya 95000
2 Amit 58000Selects by POSITION. Numbers only.
print(df.iloc[0]) print(df.iloc[0:3, 0:2]) print(df.iloc[-1])
name Rahul
dept Tech
city Bangalore
age 28.0
exp 4
salary 72000
rating Good
Name: 0, dtype: object
name dept
0 Rahul Tech
1 Priya Sales
2 Amit Tech
name Divya
dept Tech
city Pune
age NaN
exp 8
salary 88000
rating Good
Name: 5, dtype: objectloc uses names and INCLUDES the end. iloc uses numbers and EXCLUDES it.
print(df.loc[0:2, 'name']) # 3 rows - end included print(df.iloc[0:2, 0]) # 2 rows - end excluded
0 Rahul 1 Priya 2 Amit Name: name, dtype: object 0 Rahul 1 Priya Name: name, dtype: object
Gets one single value. Faster than loc for one cell.
print(df.at[0, 'name']) print(df.iat[0, 0])
Rahul Rahul
Keeps only rows where the condition is True.
print(df[df['salary'] > 80000])
name dept city age exp salary rating 1 Priya Sales Chennai 34.0 10 95000 Excellent 3 Sneha HR Pune 41.0 15 120000 Excellent 5 Divya Tech Pune NaN 8 88000 Good
& means and, | means or. Each condition needs brackets.
print(df[(df['dept'] == 'Tech') & (df['salary'] > 60000)]) print(df[(df['dept'] == 'HR') | (df['exp'] > 9)])
name dept city age exp salary rating
0 Rahul Tech Bangalore 28.0 4 72000 Good
5 Divya Tech Pune NaN 8 88000 Good
name dept city age exp salary rating
1 Priya Sales Chennai 34.0 10 95000 Excellent
3 Sneha HR Pune 41.0 15 120000 ExcellentKeeps rows where the value is in your list.
print(df[df['city'].isin(['Chennai', 'Pune'])])
name dept city age exp salary rating 1 Priya Sales Chennai 34.0 10 95000 Excellent 3 Sneha HR Pune 41.0 15 120000 Excellent 4 Vikram Sales Chennai 30.0 6 68000 Average 5 Divya Tech Pune NaN 8 88000 Good
Keeps rows where a number falls in a range.
print(df[df['salary'].between(60000, 90000)])
name dept city age exp salary rating 0 Rahul Tech Bangalore 28.0 4 72000 Good 4 Vikram Sales Chennai 30.0 6 68000 Average 5 Divya Tech Pune NaN 8 88000 Good
Keeps rows where text contains something.
print(df[df['name'].str.contains('a')])name dept city age exp salary rating 0 Rahul Tech Bangalore 28.0 4 72000 Good 1 Priya Sales Chennai 34.0 10 95000 Excellent 3 Sneha HR Pune 41.0 15 120000 Excellent 4 Vikram Sales Chennai 30.0 6 68000 Average 5 Divya Tech Pune NaN 8 88000 Good
Filters using a readable sentence.
print(df.query('salary > 80000 and dept == "Sales"'))name dept city age exp salary rating 1 Priya Sales Chennai 34.0 10 95000 Excellent
Flips the condition around.
print(df[~(df['dept'] == 'Tech')])
name dept city age exp salary rating 1 Priya Sales Chennai 34.0 10 95000 Excellent 3 Sneha HR Pune 41.0 15 120000 Excellent 4 Vikram Sales Chennai 30.0 6 68000 Average
Marks each cell True or False for missing.
print(df['age'].isnull()) print(df.isnull().sum()) # count per column
0 False 1 False 2 False 3 False 4 False 5 True Name: age, dtype: bool name 0 dept 0 city 0 age 1 exp 0 salary 0 rating 0 dtype: int64
Removes rows that have missing values.
print(df.shape) print(df.dropna().shape) print(df.dropna(subset=['age']).shape)
(6, 7) (5, 7) (5, 7)
Fills missing values with something.
print(df['age'].fillna(0).tolist()) print(df['age'].fillna(df['age'].mean()).round(1).tolist())
[28.0, 34.0, 25.0, 41.0, 30.0, 0.0] [28.0, 34.0, 25.0, 41.0, 30.0, 31.6]
Usually safer than the mean when there are extreme values.
print(df['age'].fillna(df['age'].median()).tolist())
[28.0, 34.0, 25.0, 41.0, 30.0, 30.0]
Marks rows that are copies of an earlier row.
d = pd.DataFrame({'a':[1,2,2,3],'b':['x','y','y','z']})
print(d.duplicated())
print(d.duplicated().sum())0 False 1 False 2 True 3 False dtype: bool 1
Removes the repeated rows.
d = pd.DataFrame({'a':[1,2,2,3]})
print(d.drop_duplicates())
print(df.drop_duplicates(subset=['dept'])) a
0 1
1 2
3 3
name dept city age exp salary rating
0 Rahul Tech Bangalore 28.0 4 72000 Good
1 Priya Sales Chennai 34.0 10 95000 Excellent
3 Sneha HR Pune 41.0 15 120000 ExcellentJust assign to a new name.
d = df.copy() d['bonus'] = d['salary'] * 0.10 print(d[['name','salary','bonus']].head(3))
name salary bonus 0 Rahul 72000 7200.0 1 Priya 95000 9500.0 2 Amit 58000 5800.0
Changes column names.
d = df.rename(columns={'exp': 'experience', 'dept': 'department'})
print(list(d.columns))['name', 'department', 'city', 'age', 'experience', 'salary', 'rating']
Removes columns or rows.
print(df.drop(columns=['city','rating']).head(2)) print(df.drop(index=[0,1]).shape)
name dept age exp salary 0 Rahul Tech 28.0 4 72000 1 Priya Sales 34.0 10 95000 (4, 7)
Changes the data type of a column.
d = df.dropna(subset=['age']).copy() d['age'] = d['age'].astype(int) print(d['age'].tolist())
[28, 34, 25, 41, 30]
Swaps values for other values.
print(df['dept'].replace({'Tech':'Engineering'}).tolist())['Engineering', 'Sales', 'Engineering', 'HR', 'Sales', 'Engineering']
Runs your own function on every value.
print(df['salary'].apply(lambda x: x / 1000).tolist())
def band(s):
return 'High' if s > 80000 else 'Low'
print(df['salary'].apply(band).tolist())[72.0, 95.0, 58.0, 120.0, 68.0, 88.0] ['Low', 'High', 'Low', 'High', 'Low', 'High']
Replaces values using a dictionary. Good for encoding.
order = {'Poor':1, 'Average':2, 'Good':3, 'Excellent':4}
print(df['rating'].map(order).tolist())[3, 4, 1, 4, 2, 3]
Puts numbers into named bands.
bands = pd.cut(df['salary'], bins=[0,70000,100000,999999],
labels=['Low','Medium','High'])
print(bands.tolist())['Medium', 'Medium', 'Low', 'High', 'Low', 'Medium']
Arranges rows by a column.
print(df.sort_values('salary')[['name','salary']])name salary 2 Amit 58000 4 Vikram 68000 0 Rahul 72000 5 Divya 88000 1 Priya 95000 3 Sneha 120000
ascending=False puts the largest first.
print(df.sort_values('salary', ascending=False)[['name','salary']].head(3))name salary 3 Sneha 120000 1 Priya 95000 5 Divya 88000
Sorts by the first, then breaks ties with the second.
print(df.sort_values(['dept','salary'])[['dept','name','salary']])
dept name salary 3 HR Sneha 120000 4 Sales Vikram 68000 1 Sales Priya 95000 2 Tech Amit 58000 0 Tech Rahul 72000 5 Tech Divya 88000
The top or bottom few rows, without sorting everything.
print(df.nlargest(3, 'salary')[['name','salary']]) print(df.nsmallest(2, 'salary')[['name','salary']])
name salary
3 Sneha 120000
1 Priya 95000
5 Divya 88000
name salary
2 Amit 58000
4 Vikram 68000Sorts by the row labels instead of a column.
print(df.sort_values('salary').sort_index().head(3)[['name','salary']])name salary 0 Rahul 72000 1 Priya 95000 2 Amit 58000
The average and the middle value of a column.
print(df['salary'].mean()) print(df['salary'].median())
83500.0 80000.0
The most common value. Works on text too.
print(df['dept'].mode()[0]) print(df['city'].mode().tolist())
Tech ['Bangalore', 'Chennai', 'Pune']
How spread out the values are.
print(round(df['salary'].std(), 2)) print(round(df['salary'].var(), 2))
22394.2 501500000.0
The smallest, largest and total.
print(df['salary'].min(), df['salary'].max()) print(df['salary'].sum())
58000 120000 501000
How many values are filled, and how many are distinct.
print(df['age'].count()) # missing not counted print(df['dept'].nunique())
5 3
Counts how many times each value appears. Extremely useful.
print(df['dept'].value_counts()) print(df['dept'].value_counts(normalize=True).round(2))
dept Tech 3 Sales 2 HR 1 Name: count, dtype: int64 dept Tech 0.50 Sales 0.33 HR 0.17 Name: proportion, dtype: float64
Lists the distinct values.
print(df['city'].unique())
['Bangalore' 'Chennai' 'Pune']
The value below which a share of the data falls.
print(df['salary'].quantile(0.25)) print(df['salary'].quantile([0.25, 0.5, 0.75]).tolist())
69000.0 [69000.0, 80000.0, 93250.0]
How strongly two number columns move together. Between -1 and 1.
print(round(df['exp'].corr(df['salary']), 3)) print(df[['exp','salary']].corr().round(3))
0.979
exp salary
exp 1.000 0.979
salary 0.979 1.000A running total down the column.
print(df['salary'].cumsum().tolist())
[72000, 167000, 225000, 345000, 413000, 501000]
Splits into groups, then summarises each group.
print(df.groupby('dept')['salary'].mean())dept HR 120000.000000 Sales 81500.000000 Tech 72666.666667 Name: salary, dtype: float64
How many rows in each group.
print(df.groupby('dept').size())
print(df.groupby('dept')['salary'].count())dept HR 1 Sales 2 Tech 3 dtype: int64 dept HR 1 Sales 2 Tech 3 Name: salary, dtype: int64
Several summaries at once.
print(df.groupby('dept')['salary'].agg(['count','mean','max']))count mean max dept HR 1 120000.000000 120000 Sales 2 81500.000000 95000 Tech 3 72666.666667 88000
Different summary for each column.
print(df.groupby('dept').agg({'salary':'mean', 'exp':'max'}))salary exp dept HR 120000.000000 15 Sales 81500.000000 10 Tech 72666.666667 8
Groups within groups.
print(df.groupby(['dept','city'])['salary'].mean())
dept city
HR Pune 120000.0
Sales Chennai 81500.0
Tech Bangalore 65000.0
Pune 88000.0
Name: salary, dtype: float64Turns the group labels back into a normal column.
g = df.groupby('dept')['salary'].mean().reset_index()
print(g)dept salary 0 HR 120000.000000 1 Sales 81500.000000 2 Tech 72666.666667
Summarises data into a grid. Like a pivot table in Excel.
print(pd.pivot_table(df, values='salary', index='dept',
columns='city', aggfunc='mean'))city Bangalore Chennai Pune dept HR NaN NaN 120000.0 Sales NaN 81500.0 NaN Tech 65000.0 NaN 88000.0
Counts how often combinations appear.
print(pd.crosstab(df['dept'], df['city']))
city Bangalore Chennai Pune dept HR 0 0 1 Sales 0 2 0 Tech 2 0 1
Turns wide data into long data.
w = pd.DataFrame({'name':['Rahul','Priya'],'maths':[87,92],'science':[78,85]})
print(pd.melt(w, id_vars='name', var_name='subject', value_name='marks'))name subject marks 0 Rahul maths 87 1 Priya maths 92 2 Rahul science 78 3 Priya science 85
Makes a column the row label, or puts it back.
d = df.set_index('name')
print(d.loc['Rahul'])
print(d.reset_index().head(2)[['name','dept']])dept Tech
city Bangalore
age 28.0
exp 4
salary 72000
rating Good
Name: Rahul, dtype: object
name dept
0 Rahul Tech
1 Priya SalesStacks tables on top of each other.
a = df.head(2)[['name','dept']] b = df.tail(2)[['name','dept']] print(pd.concat([a, b]))
name dept 0 Rahul Tech 1 Priya Sales 4 Vikram Sales 5 Divya Tech
axis=1 joins columns instead of rows.
a = df[['name']].head(3) b = df[['salary']].head(3) print(pd.concat([a, b], axis=1))
name salary 0 Rahul 72000 1 Priya 95000 2 Amit 58000
Joins two tables on a shared column. Like a JOIN in SQL.
emp = pd.DataFrame({'name':['Rahul','Priya'], 'dept_id':[1, 2]})
dept = pd.DataFrame({'dept_id':[1, 2], 'dept_name':['Tech','Sales']})
print(pd.merge(emp, dept, on='dept_id'))name dept_id dept_name 0 Rahul 1 Tech 1 Priya 2 Sales
how= decides which rows are kept.
a = pd.DataFrame({'id':[1,2,3], 'x':['a','b','c']})
b = pd.DataFrame({'id':[2,3,4], 'y':['B','C','D']})
print(pd.merge(a, b, on='id', how='inner'))
print(pd.merge(a, b, on='id', how='left'))id x y 0 2 b B 1 3 c C id x y 0 1 a NaN 1 2 b B 2 3 c C
Changes the case of every value in a text column.
print(df['name'].str.upper().tolist())
['RAHUL', 'PRIYA', 'AMIT', 'SNEHA', 'VIKRAM', 'DIVYA']
How many characters in each value.
print(df['name'].str.len().tolist())
[5, 5, 4, 5, 6, 5]
Cleans text values.
messy = pd.Series([' Rahul ', ' Priya'])
print(messy.str.strip().tolist())
print(df['dept'].str.replace('Tech', 'IT').tolist())['Rahul', 'Priya'] ['IT', 'Sales', 'IT', 'HR', 'Sales', 'IT']
Splits text and picks a part.
full = pd.Series(['Rahul Sharma', 'Priya Nair'])
print(full.str.split(' ').tolist())
print(full.str.split(' ').str[0].tolist())[['Rahul', 'Sharma'], ['Priya', 'Nair']] ['Rahul', 'Priya']
Turns text into real dates so you can work with them.
d = pd.DataFrame({'joined':['2022-01-15','2023-06-30']})
d['joined'] = pd.to_datetime(d['joined'])
print(d.dtypes)
print(d)joined datetime64[ns]
dtype: object
joined
0 2022-01-15
1 2023-06-30Pulls parts out of a date column.
d = pd.DataFrame({'joined': pd.to_datetime(['2022-01-15','2023-06-30'])})
print(d['joined'].dt.year.tolist())
print(d['joined'].dt.month.tolist())
print(d['joined'].dt.day_name().tolist())[2022, 2023] [1, 6] ['Saturday', 'Friday']
Subtract two dates to get the gap.
d = pd.to_datetime(['2022-01-01','2024-01-01']) print((d[1] - d[0]).days)
730
Sessions D01 to D23. Databases, queries, functions, subqueries, joins and window functions.
48 functionsCreates a new table and defines its columns.
SELECT sql FROM sqlite_master WHERE name='employees';
sql ----------------------------------------------------------------------------------------------------------------------------------------------------- CREATE TABLE employees ( emp_id INTEGER PRIMARY KEY, name TEXT, dept_id INTEGER, salary INTEGER, experience INTEGER, join_date TEXT, rating TEXT)
The column that uniquely identifies each row.
SELECT emp_id, name FROM employees LIMIT 3;
emp_id | name -------+------------- 101 | Rahul Sharma 102 | Priya Nair 103 | Amit Kumar
A column that points to the primary key of another table.
SELECT e.name, d.dept_name FROM employees e JOIN departments d ON e.dept_id=d.dept_id LIMIT 3;
name | dept_name -------------+---------- Rahul Sharma | Tech Priya Nair | Sales Amit Kumar | Tech
Fetches columns from a table.
SELECT name, salary FROM employees LIMIT 3;
name | salary -------------+------- Rahul Sharma | 72000 Priya Nair | 95000 Amit Kumar | 58000
Keeps only the rows that match a condition.
SELECT name, salary FROM employees WHERE salary > 70000;
name | salary -------------+------- Rahul Sharma | 72000 Priya Nair | 95000 Sneha Reddy | 120000 Divya Menon | 88000
Sorts the result.
SELECT name, salary FROM employees ORDER BY salary DESC LIMIT 3;
name | salary ------------+------- Sneha Reddy | 120000 Priya Nair | 95000 Divya Menon | 88000
Restricts how many rows come back.
SELECT name, salary FROM employees ORDER BY salary DESC LIMIT 2;
name | salary ------------+------- Sneha Reddy | 120000 Priya Nair | 95000
Removes duplicate rows from the result.
SELECT DISTINCT dept_id FROM employees ORDER BY dept_id;
dept_id ------- 1 2 3 4
Gives a column or table a temporary name.
SELECT name AS employee, salary AS monthly_pay FROM employees LIMIT 2;
employee | monthly_pay -------------+------------ Rahul Sharma | 72000 Priya Nair | 95000
Checks a value against a list.
SELECT name, dept_id FROM employees WHERE dept_id IN (1,3);
name | dept_id -------------+-------- Rahul Sharma | 1 Amit Kumar | 1 Sneha Reddy | 3 Divya Menon | 1
Checks a value inside a range.
SELECT name, salary FROM employees WHERE salary BETWEEN 60000 AND 90000;
name | salary -------------+------- Rahul Sharma | 72000 Vikram Rao | 68000 Divya Menon | 88000
Pattern matching on text.
SELECT name FROM employees WHERE name LIKE '%Kumar%';
name ---------- Amit Kumar
Finds missing values.
SELECT name, rating FROM employees WHERE rating IS NULL;
name | rating ----------+------- Arjun Das | NULL
Adds a new row.
SELECT COUNT(*) AS rows_now FROM employees;
rows_now -------- 7
Changes values in existing rows.
SELECT name, salary FROM employees WHERE emp_id=101;
name | salary -------------+------- Rahul Sharma | 72000
Removes rows.
SELECT COUNT(*) AS total FROM employees;
total ----- 7
Changes the structure of an existing table.
SELECT name FROM pragma_table_info('employees') LIMIT 4;name ------- emp_id name dept_id salary
Changes the case of text.
SELECT UPPER(name) AS caps, LOWER(name) AS small FROM employees LIMIT 2;
caps | small -------------+------------- RAHUL SHARMA | rahul sharma PRIYA NAIR | priya nair
How many characters in the text.
SELECT name, LENGTH(name) AS chars FROM employees LIMIT 3;
name | chars -------------+------ Rahul Sharma | 12 Priya Nair | 10 Amit Kumar | 10
Takes part of a string.
SELECT name, SUBSTR(name,1,5) AS first_five FROM employees LIMIT 3;
name | first_five -------------+----------- Rahul Sharma | Rahul Priya Nair | Priya Amit Kumar | Amit
Joins strings together.
SELECT name || ' works in dept ' || dept_id AS description FROM employees LIMIT 2;
description ---------------------------- Rahul Sharma works in dept 1 Priya Nair works in dept 2
Removes spaces from both ends.
SELECT '[' || TRIM(' Rahul ') || ']' AS trimmed;trimmed ------- [Rahul]
Swaps one piece of text for another.
SELECT REPLACE(name,'Kumar','K.') AS shortened FROM employees WHERE name LIKE '%Kumar%';
shortened --------- Amit K.
Pulls the year, month or day out of a date.
SELECT name, STRFTIME('%Y',join_date) AS yr, STRFTIME('%m',join_date) AS mth FROM employees LIMIT 3;name | yr | mth -------------+------+---- Rahul Sharma | 2022 | 01 Priya Nair | 2019 | 06 Amit Kumar | 2023 | 03
Returns today's date.
SELECT DATE('2026-09-04') AS today;today ---------- 2026-09-04
How far apart two dates are.
SELECT name, 2026 - CAST(STRFTIME('%Y',join_date) AS INTEGER) AS years_here FROM employees LIMIT 3;name | years_here -------------+----------- Rahul Sharma | 4 Priya Nair | 7 Amit Kumar | 3
If-then-else logic inside a query.
SELECT name, salary,
CASE WHEN salary > 100000 THEN 'High'
WHEN salary > 65000 THEN 'Medium'
ELSE 'Low' END AS band
FROM employees ORDER BY salary DESC;name | salary | band -------------+--------+------- Sneha Reddy | 120000 | High Priya Nair | 95000 | Medium Divya Menon | 88000 | Medium Rahul Sharma | 72000 | Medium Vikram Rao | 68000 | Medium Amit Kumar | 58000 | Low Arjun Das | 54000 | Low
Returns the first value that is not NULL.
SELECT name, COALESCE(rating,'Not rated') AS rating FROM employees;
name | rating -------------+---------- Rahul Sharma | Good Priya Nair | Excellent Amit Kumar | Poor Sneha Reddy | Excellent Vikram Rao | Average Divya Menon | Good Arjun Das | Not rated
Counts rows.
SELECT COUNT(*) AS all_rows, COUNT(rating) AS with_rating FROM employees;
all_rows | with_rating ---------+------------ 7 | 6
The total and the average of a column.
SELECT SUM(salary) AS total, ROUND(AVG(salary),0) AS average FROM employees;
total | average -------+-------- 555000 | 79286.0
The smallest and largest value.
SELECT MIN(salary) AS lowest, MAX(salary) AS highest FROM employees;
lowest | highest -------+-------- 54000 | 120000
Splits rows into groups and summarises each one.
SELECT dept_id, COUNT(*) AS headcount, ROUND(AVG(salary),0) AS avg_sal FROM employees GROUP BY dept_id;
dept_id | headcount | avg_sal --------+-----------+--------- 1 | 3 | 72667.0 2 | 2 | 81500.0 3 | 1 | 120000.0 4 | 1 | 54000.0
Filters the groups after aggregation.
SELECT dept_id, COUNT(*) AS n FROM employees GROUP BY dept_id HAVING COUNT(*) > 1;
dept_id | n --------+-- 1 | 3 2 | 2
A query inside another query.
SELECT name, salary FROM employees WHERE salary > (SELECT AVG(salary) FROM employees);
name | salary ------------+------- Priya Nair | 95000 Sneha Reddy | 120000 Divya Menon | 88000
The classic interview question.
SELECT MAX(salary) AS second_highest FROM employees WHERE salary < (SELECT MAX(salary) FROM employees);
second_highest -------------- 95000
Matches against a list produced by another query.
SELECT name FROM employees WHERE dept_id IN (SELECT dept_id FROM departments WHERE city='Bangalore');
name ------------ Rahul Sharma Amit Kumar Divya Menon Arjun Das
Keeps only rows that match in both tables.
SELECT e.name, d.dept_name FROM employees e INNER JOIN departments d ON e.dept_id=d.dept_id LIMIT 4;
name | dept_name -------------+---------- Rahul Sharma | Tech Priya Nair | Sales Amit Kumar | Tech Sneha Reddy | HR
Keeps every row from the left table.
SELECT d.dept_name, COUNT(e.emp_id) AS headcount FROM departments d LEFT JOIN employees e ON d.dept_id=e.dept_id GROUP BY d.dept_name;
dept_name | headcount ----------+---------- HR | 1 Marketing | 1 Sales | 2 Tech | 3
Rows in one table that have nothing in the other.
SELECT d.dept_name FROM departments d LEFT JOIN employees e ON d.dept_id=e.dept_id WHERE e.emp_id IS NULL;
dept_name ---------
A table joined to itself.
SELECT a.name, b.name AS same_dept_colleague FROM employees a JOIN employees b ON a.dept_id=b.dept_id AND a.emp_id<b.emp_id;
name | same_dept_colleague -------------+-------------------- Rahul Sharma | Amit Kumar Rahul Sharma | Divya Menon Priya Nair | Vikram Rao Amit Kumar | Divya Menon
Stacks the results of two queries.
SELECT name FROM employees WHERE salary>100000 UNION SELECT name FROM employees WHERE experience>9;
name ----------- Priya Nair Sneha Reddy
Numbers the rows in order, with no ties.
SELECT name, salary, ROW_NUMBER() OVER (ORDER BY salary DESC) AS rn FROM employees LIMIT 4;
name | salary | rn -------------+--------+--- Sneha Reddy | 120000 | 1 Priya Nair | 95000 | 2 Divya Menon | 88000 | 3 Rahul Sharma | 72000 | 4
Ranks rows, handling ties differently.
SELECT name, salary, RANK() OVER (ORDER BY salary DESC) AS rnk FROM employees LIMIT 4;
name | salary | rnk -------------+--------+---- Sneha Reddy | 120000 | 1 Priya Nair | 95000 | 2 Divya Menon | 88000 | 3 Rahul Sharma | 72000 | 4
Restarts the window calculation for each group.
SELECT name, dept_id, salary, RANK() OVER (PARTITION BY dept_id ORDER BY salary DESC) AS rnk FROM employees;
name | dept_id | salary | rnk -------------+---------+--------+---- Divya Menon | 1 | 88000 | 1 Rahul Sharma | 1 | 72000 | 2 Amit Kumar | 1 | 58000 | 3 Priya Nair | 2 | 95000 | 1 Vikram Rao | 2 | 68000 | 2 Sneha Reddy | 3 | 120000 | 1 Arjun Das | 4 | 54000 | 1
Adds a group summary without collapsing the rows.
SELECT name, dept_id, salary, ROUND(AVG(salary) OVER (PARTITION BY dept_id),0) AS dept_avg FROM employees;
name | dept_id | salary | dept_avg -------------+---------+--------+--------- Rahul Sharma | 1 | 72000 | 72667.0 Amit Kumar | 1 | 58000 | 72667.0 Divya Menon | 1 | 88000 | 72667.0 Priya Nair | 2 | 95000 | 81500.0 Vikram Rao | 2 | 68000 | 81500.0 Sneha Reddy | 3 | 120000 | 120000.0 Arjun Das | 4 | 54000 | 54000.0
Saves a query so it behaves like a virtual table.
SELECT name, salary FROM employees WHERE salary > 80000;
name | salary ------------+------- Priya Nair | 95000 Sneha Reddy | 120000 Divya Menon | 88000
A named block of SQL saved in the database.
SELECT 'CALL get_dept_summary(1);' AS mysql_syntax;
mysql_syntax ------------------------- CALL get_dept_summary(1);
SQL that runs automatically when data changes.
SELECT 'CREATE TRIGGER log_change AFTER UPDATE ON employees ...' AS mysql_syntax;
mysql_syntax ------------------------------------------------------- CREATE TRIGGER log_change AFTER UPDATE ON employees ...