Final Exam
Machine Learning 2015-2
After solving all the questions in the exam save your notebook with the name username.ipynb
and submit it to: https://www.dropbox.com/request/KN8GwdAIi0Hl2jk2mg2E
The following code implements a simple one-neuron neural network:
1. (1.0)
Find a weight vector such that the neural network calculates the NOR function:
Use the following function to test your answer:
2. (1.0)
The following function calculates the loss function of the neural network
Write a function that calculates the gradient of the loss with respect to the weights:
Use the following functions to test your code:
Now, we can use the gradient function to train the neural network using gradient descent
3. (1.0)
Now we will modify the loss function to include a regularization term:
where is the prediction calculated by the neural network.
To accomplish this you must modify the following functions:
You can use the following functions to test your code:
4. (1.0)
Now train the neural network using regularization:
What is the effect of regularization? Discuss.
5. (1.0)
Here, we will build a kernel version of the previous neural network, i.e., a neural network able to work in a feature space induced by a kernel. To do this we will express the weight vector as a linear combination of vectors in a set :
Now, implement this modifying the following functions:
Test your functions with the following code:
6. (optional, extra credit)
Train the kernel neural network using gradient descent.