Progress Report 1

Summary

Found a pre-trained VGG-19 network in MatConvNet format
Learned how to access outputs from convolutional layers in MatConvNet
Wrote a function to calculate a gram matrix based on feature maps

Pre-Trained Network

The paper which our work is based on, "A Neural Algorithm of Artistic Style", uses a pre-trained VGG-19 neural network. A critical step in our project, therefore, is to find a pre-trained network of the same type which is compatible with the MatConvNet library which we are using. We found the network at http://www.vlfeat.org/matconvnet/pretrained/.

Convolutional Layers

The convolutional layers of our network are based on 2-D convolution, like we learned about in class. We used the demo below to understand and test out the convolutional layers of our neural network. This demo was taken from http://cs231n.github.io/convolutional-networks/

The demo above is a single convolutional layer consisting of two filters, or nodes. The inputs have a zero-padding of 1. Each filter has a depth of 3, since the input has a depth of 3. Filter W0 has a bias of 1 and filter W1 has a bias of 0. The filter size is 3x3 and stride is 2, meaning that our output size is 3x3. This comes from the equation below:

Output Size = (W−F+2P)/S+1

In this equation, W is the size of the input, F is the size of the filters, P is the zero-padding width, and S is the stride. Therefore, Output Size = (5-3+2*1)/2 +1 = 3, and we have a 3x3 output. As we learned in class, the zero-padding does not effect the information contained in our output, but it can change the size of our output.

Feature Maps

One important step of our overall algorithm is to access the output of each convolutional layer, which is referred to as a feature map in the paper that our work is based on. In order to confirm that we can do this using the MatConvNet library, we created a network using MatConvNet that mirrors the example above and confirmed that the results of our network match the results of the example. Our results are shown below.

Gram Matrix

The feature maps are used to calculate a gram matrix for each layer, which are used to represent the texture of our "Style" images.

After we confirmed that we can access the feature maps, we created a function to calculate the gram matrix of each layer.

All of the code described above has been published and included below.

Click the symbol above to view our simple convolutional network example.

Click the symbol above to view our gram matrix function.

Our Plan: Three Weeks to Go!

1. Finish writing the algorithm

Although we have made progress on parts of the algorithm, we still have a long way to go before we have a working project.

2. GPU Computing

We must figure our how to use GPU and parallel computing to make our code run faster.

3. Optimize Our Functions

We have a working Gram matrix function, however for loops are very slow in matlab and we are attempting to eradicate them from our code if possible. Generally making our code faster will be a main focus in the future.