instructions.ipynb

Kernel: Python 3 (Anaconda 5)

Using AES encryption in Python

Below you will see a simple example of how to use AES for encryption in Python.

The Advanced Encryption Standard (AES) is one of the most widely used and secure ciphers.

AES is a block cipher meaning that it encrypts data block by block. A block for AES is 128 bits of information (16 bytes). This is about 16 keyboard characters (or about 5 pixels) worth of information.

The details of what AES does to a block of data to encrypt it is a little complicated, but it basically involves repeatedly making substitutions in the bytes (like our substitution cipher) and then mixing the bytes around. Exactly what comes out also depends on the key which is mixed into the data during the encryption process.

A block cipher like AES usually requires a mode of operation. Again this is beyond the scope of our project but the curious can read the Wikipedia page below.

https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation

The goals of the block cipher modes are to blur the statistical relationship between the plaintext and the ciphertext, and also to ensure that if a message is encrypted twice with the same key then different ciphertexts come out.

That's possible because in addition to the key the block cipher takes an initialization vector (IV) which is a random number associated with a single encryption. It does not need to be kept private.

There are a lot of libraries for doing cryptography in Python.

The library we will use is called pycrypto.

The basic usage is demonstrated below. Ciphertext is not printable (because it involves random bits which do not render as normal characters.) We print the ciphertext out in a few different ways to give you a feel for what it is. In a Python byte string '\xAA' is the hexadecimal number AA, '\x10' is the hexadecimal number 10, etc.

#https://pypi.org/project/pycrypto/

from Crypto.Cipher import AES
obj = AES.new('This is a key123', AES.MODE_CBC, 'This is an IV456')
message = "The answer is noThe answer is noBye mom, love you.**************"
ciphertext = obj.encrypt(message)

print("Encrypting the message M='The answer is no' using the key 'This is a key123' and the IV 'This is an IV456'")

print("\nHere is the ciphertext as a byte string:")

print( ciphertext)

print("\nHere is the ciphertext as a list of hexadecimal numbers:")
print( [hex(x) for x in list(ciphertext)])

print("\nHere is the ciphertext as a sequence of bits:")
print("".join( [bin(x)[2:] for x in list(ciphertext)]))

Encrypting the message M='The answer is no' using the key 'This is a key123' and the IV 'This is an IV456'

Here is the ciphertext as a byte string:
b'\xd6\x83\x8dd!VT\x92\xaa`A\x05\xe0\x9b\x8b\xf1)\xc8\x80\x90p\xe2\xa4X\x9d\xe7\xf71\xeb= \xcb\xf0\xb6\xdfs\x8a\xa6\x188CzN\xc5\x83:\x98\xb0lH\xde\xc9E\x8b\xdc\x91\x1e\xee\xb1\\U\xb0;\x05'

Here is the ciphertext as a list of hexadecimal numbers:
['0xd6', '0x83', '0x8d', '0x64', '0x21', '0x56', '0x54', '0x92', '0xaa', '0x60', '0x41', '0x5', '0xe0', '0x9b', '0x8b', '0xf1', '0x29', '0xc8', '0x80', '0x90', '0x70', '0xe2', '0xa4', '0x58', '0x9d', '0xe7', '0xf7', '0x31', '0xeb', '0x3d', '0x20', '0xcb', '0xf0', '0xb6', '0xdf', '0x73', '0x8a', '0xa6', '0x18', '0x38', '0x43', '0x7a', '0x4e', '0xc5', '0x83', '0x3a', '0x98', '0xb0', '0x6c', '0x48', '0xde', '0xc9', '0x45', '0x8b', '0xdc', '0x91', '0x1e', '0xee', '0xb1', '0x5c', '0x55', '0xb0', '0x3b', '0x5']

Here is the ciphertext as a sequence of bits:
11010110100000111000110111001001000011010110101010010010010101010101100000100000110111100000100110111000101111110001101001110010001000000010010000111000011100010101001001011000100111011110011111110111110001111010111111011000001100101111110000101101101101111111100111000101010100110110001110001000011111101010011101100010110000011111010100110001011000011011001001000110111101100100110001011000101111011100100100011111011101110101100011011100101010110110000111011101

Decryption

Decryption works like this.

obj2 = AES.new('This is a key123', AES.MODE_CBC, 'This is an IV456')
obj2.decrypt(ciphertext)

b'The answer is noThe answer is noBye mom, love you.**************'

# Exercise:
## What happens if the IV or the key is wrong in the decryption (try it)?

# You might want to read the documentation for AES.new, below. 

AES.new?

An irritating detail (part 1)

You might notice that the message encrypted above is exactly 16 bytes (characters) long.

If you change the message to be shorter or longer (but not an exact multiple of 16 characters) you'll find that you get an error message.

This happens because the block cipher mode CBC requires the message to divide evenly into blocks. Because a block is 16 bytes, messages must have lengths that are multiples of 16.

You can get around this with "padding" which rounds out the length of a message.

Here is an example of how it works.

# Much of this code is borrowed from here:  https://gist.github.com/crmccreary/5610068
BS = 16
pad = lambda s: s + (BS - len(s) % BS) * chr(BS - len(s) % BS) 
unpad = lambda s : s[0:-ord(s[-1])]

#The above lambdas do the padding and unpadding

padded_message = pad("squ")
padded_message  

'squ\r\r\r\r\r\r\r\r\r\r\r\r\r'

ps = "squ" + 13*chr(13)
print(repr( ps))
ps[0:-13]

'squ\r\r\r\r\r\r\r\r\r\r\r\r\r'

'squ'

Because the message "squ" is length 3, it lacks 13 characters to make a full block.

The padding scheme works by filling out 13 more bytes in the message with the value 13 (which happens to represent the ASCII carriage return character '\r').

#Another example
padded_message = pad("Class of 2023")
padded_message

'Class of 2023\x03\x03\x03'

#Now we encrypt the padded message as usual...

obj3 = AES.new('This is a key123', AES.MODE_CBC, 'This is an IV456')
ciphertext = obj3.encrypt(padded_message)
print(ciphertext)

b'_\xbf\xe8"\x86*\xc6\xf3\xc2\n\xf5\x0c\x98\xcc\x01z'

#Aside:  The decode function turns a byte array into a python string
type(b'hello'), type(b'hello'.decode())

(bytes, str)

type( b'Nicholas Cage'.decode())

str

# Now we decrypt the ciphertext and "unpad" the result to get the original message.

obj4 = AES.new('This is a key123', AES.MODE_CBC, 'This is an IV456')
msg_with_pad = obj4.decrypt(ciphertext)
print(msg_with_pad)
message = unpad(msg_with_pad.decode())
message

b'Class of 2023\x03\x03\x03'

'Class of 2023'

Exercise:

Encrypt and then decrypt a message of your choice using a key and IV of your choice. The key must be 16,24 or 32 characters, exactly, and the IV must be 16 characters.

Be sure to use padding so that you don't need to worry about the length of the message.

# Solution to exercise goes here

# What happens if the message is already a multiple of 16 in length
# and you pad it?

S = "Even God doesn't work on Sundays"
print( len(S))
pad(S)

32

"Even God doesn't work on Sundays\x10\x10\x10\x10\x10\x10\x10\x10\x10\x10\x10\x10\x10\x10\x10\x10"

An irritating detail (part 2)

Notice that the "key" must be exactly 16, 24, or 32 bytes long. This means that we can't use ordinary passwords like "swordfish" as a key.

We can get around this and use natural passphrases as keys, we just need to be sure to use the hash of a passphrase as a key. Below is an example of how this can work.

We use SHA256 as a hash function.

It returns a 256 bit = 32 byte hash digest. We use the first 16 bytes for the key, and the last 16 bytes for the IV. Now we can relax and use regular passwords, plus we don't have to remember the IV because it's built into the password.

# A quick example of hashing ...

import hashlib
m = hashlib.sha256()
message_to_hash = "I liked you the whole time".encode()
D = m.digest()
print("Here's the digest of the message:")
print(D)
print("\nHere are the first 16 bytes")
print(D[:16])
print("\nHere are the last 16 bytes")
print(D[16:])

Here's the digest of the message:
b"\xe3\xb0\xc4B\x98\xfc\x1c\x14\x9a\xfb\xf4\xc8\x99o\xb9$'\xaeA\xe4d\x9b\x93L\xa4\x95\x99\x1bxR\xb8U"

Here are the first 16 bytes
b'\xe3\xb0\xc4B\x98\xfc\x1c\x14\x9a\xfb\xf4\xc8\x99o\xb9$'

Here are the last 16 bytes
b"'\xaeA\xe4d\x9b\x93L\xa4\x95\x99\x1bxR\xb8U"

"""
This code turns a natural passphrase into a 16 byte key, a 16 byte IV, and encrypts a given message.
"""

import hashlib
m = hashlib.sha256()
passphrase = input("Please enter a passphrase.")
passphrase = passphrase.encode() #turns a string into a byte string
m.update(passphrase)
digest = m.digest() #this "digest" is the hash
key = digest[:16]  # The first 16 bytes of the 32 byte hash are the key
IV = digest[16:]   # The last 16 bytes of the 32 byte hash are the IV

message = input("Enter a short message to encrypt")
obj = AES.new(key, AES.MODE_CBC,IV)
message = pad(message)
ciphertext = obj.encrypt(message)

print("This produced the ciphertext below.")
print(ciphertext)

Please enter a passphrase.

Enter a short message to encrypt

This produced the ciphertext below.
b'\xe9\x9e\xfb\xaa^\xf8\x10\xa5j\x14\x80,\xdfo\xd3\x85\xd6\xe5\xaf\xd4\xa1\x96u\xda~\xf2\xfbU\xd3\xc0\x99\x86'

# Now we get the passphrase once again for decryption.

import hashlib
m = hashlib.sha256()
passphrase = input("Please enter a passphrase for decryption (same as encryption).")
passphrase = passphrase.encode() #turns a string into a byte string
m.update(passphrase)
digest = m.digest() #this "digest" is the hash
key = digest[:16]  # The first 16 bytes of the 32 byte hash are the key
IV = digest[16:]   # The last 16 bytes of the 32 byte hash are the IV

obj = AES.new(key, AES.MODE_CBC,IV)

try:
    message = obj.decrypt(ciphertext)
    message = unpad(message.decode())
    print("The plaintext was:")
    print(message)
except:
    print("Decryption failed.  Was the passphrase correct?")

Please enter a passphrase for decryption (same as encryption).

The plaintext was:
the bats are in the belfry

#Excercise: 
## Complete the function below so that it works as described.
import hashlib

def get_AES_obj(passphrase):
    """Input: a passphrase string
       Output: the function returns AES.new(key, AES.MODE_CBC,IV), where key and IV are derived from the passphrase as above.
    """


## Complete the function below so that it works as described.    
def encrypt(message,passphrase):
    """
    Input:  a message string to encrypt and an arbitrary passphrase string.
    Output: a byte string corresponding to the ciphertext produced by encrypting
    "message" in the manner shown in the above cells of this notebook.  
    Be sure to use the hash trick and padding. 
    """

    
    
    
    
    
## Complete the function below so that it works as described.
def decrypt(ciphertext,passphrase):
    """
    Input:  a ciphertext byte string to decrypt and an arbitrary passphrase string.
    Output: a string corresponding to the original plaintext

    Be sure to unpad the message and use try/except structures for robustness.
    """    

# Test your encryption and decryption functions...

CT = encrypt("prof johnson is in his kitchen","stalker666")
PT = decrypt(CT,"stalker666")
print(PT)

## Exercise:

###  Read in and encrypt a file.
###  Write the ciphertext to a new file.
###  Read in the ciphertext and decrypt it.
###  Print out the plaintext

### I've done some of the rigamorole for you to get you started.

m = hashlib.sha256()
passphrase = input("Please enter a passphrase.")
passphrase = passphrase.encode() #turns a string into a byte string
m.update(passphrase)
digest = m.digest() #this "digest" is the hash
key = digest[:16]  # The first 16 bytes of the 32 byte hash are the key
IV = digest[16:]   # The last 16 bytes of the 32 byte hash are the IV
obj = AES.new(key, AES.MODE_CBC,IV)


# Solution:  (Instructors can delete this)

file_to_encrypt = open("fte.txt","w")
file_to_encrypt.write("Luxembourg is a country in central Europe.")
file_to_encrypt.close()

fp = open("fte.txt","r")
plaintext = fp.read()
fp.close()
plaintext = pad(plaintext)
ct = obj.encrypt(plaintext)

fp = open("fte.enc","wb")
fp.write(ct)   ### Write the ciphertext to the fte.enc file
fp.close()

fp = open("fte.enc","rb")
obj = AES.new(key, AES.MODE_CBC,IV)
ct = fp.read()
fp.close()

pt = obj.decrypt(ct)
pt = unpad(pt.decode())
pt

The Project

The goal of this project is to upgrade the password manager program we wrote earlier so that it stores passwords in encrypted form. The original password manager is included in this folder in the file pwd_manager.py. You might want to make a safe copy of this file in case you make some changes that you later want to revert.

Here is the basic flow of the program pwd_manager.py as it currently works:

1. The file passwords.json is opened.

2. The file is read by the json module and converted to a dictionary object called pwds.

3. The program interacts with the user to read and write to pwds.

4. The json module converts pwds back to a string of JSON formatted text and saves the result in the file pwd_manager.py

The logic of the new program will be very similar. You only need to modify the beginning and the end of the program to introduce a cryptographic "layer".

Here is how the new program should work:

1. An attempt is made to open passwords.dat. If it fails, the user is notified and we exit from the program with the exit(1) command. If it succeeds then we read and store the file as a string called ciphertext.

2. The user is prompted for a passphrase, which is saved in a variable called passphrase.

3. An attempt is made to decrypt the ciphertext using the passphrase. Success can be tested by trying to load the resulting plaintext string into the json module (using json.loads()) for conversion to a dictionary. If this fails, notify the user and exit the program.

4. We now have the pwd dictionary, and the program operates as pwd_manager.py did. You might want to add a new option to let the user change the passphrase.

5. The user exits the program by typing 'Q'.

6. The json module is used to dump pwd to a string (not a file) using json.dumps(). Let the string be called data.

7. Encrypt the data using the passphrase and write it to a file.

Notice that the encrypt() and decrypt() functions you wrote as solutions to the exercises in this notebook can do most of the work here. You can literally cut and pasted these functions into pwd_manager.py. Be sure to include the pad and unpad lambdas, as well as import all the necessary modules.

# Some experiments with JSON

import json

# # end of pwd_manager.py 

pwds = {'1':"huey",'2':'dewey','3':'lewey'}

s = json.dumps(pwds)
# pad s
# encrypt s
# write s to a file using "wb" (write binary) mode

'{"1": "huey", "2": "dewey", "3": "lewey"}'

pad(s)
#encrypt and save to password.dat

'{"1": "huey", "2": "dewey", "3": "lewey"}\x07\x07\x07\x07\x07\x07\x07'

J = '{"1": "huey", "2": "dewey", "3": "lewey"}'
J

'{"1": "huey", "2": "dewey", "3": "lewey"}'

## The beginning of pwd_manager.py

#open passwords.dat (ciphertext)
# decrypt
# decode and unpad
# J will be the json
pwds = json.loads(J)

({'1': 'huey', '2': 'dewey', '3': 'lewey'}, dict)