Commit 6af8efc0 authored by Ronghang Hu's avatar Ronghang Hu
Browse files

Add MatCaffe docs to docs/tutorial/interfaces.md

parent 9735f4b3
......@@ -67,10 +67,211 @@ Compile pycaffe by `make pycaffe`. The module dir caffe/python/caffe should be i
## MATLAB
The MATLAB interface -- matcaffe -- is the `caffe` mex and its helper m-files in caffe/matlab. Load models, do forward and backward, extract output and read-only model weights, and load the binaryproto format mean as a matrix.
The MATLAB interface -- matcaffe -- is a `caffe` package in caffe/matlab in which you can integrate Caffe in your Matlab code.
A MATLAB demo is in caffe/matlab/caffe/matcaffe_demo.m
In MatCaffe, you can
Note that MATLAB matrices and memory are in column-major layout counter to Caffe's row-major layout! Double-check your work accordingly.
* Creating multiple Nets in Matlab
* Do forward and backward computation
* Access any layer within a network, and any parameter blob in a layer
* Get and set data or diff to any blob within a network, not restricting to input blobs or output blobs
* Save a network's parameters to file, and load parameters from file
* Reshape a blob and reshape a network
* Edit network parameter and do network surgery
* Create multiple Solvers in Matlab for training
* Resume training from solver snapshots
* Access train net and test nets in a solver
* Run for a certain number of iterations and give back control to Matlab
* Intermingle arbitrary Matlab code to with gradient steps
Compile matcaffe by `make matcaffe`.
A MATLAB demo is in caffe/matlab/matcaffe_demo.m
### Build MatCaffe
Build MatCaffe with `make all matcaffe`. After that, you may test it using `make mattest`.
Common issue: if you run into error messages `libstdc++.so.6:version 'GLIBCXX_3.4.15' not found` during `make mattest`, then it means that your Matlab's runtime libraries does not match your compile-time libraries. You may need to do the following before you start matlab:
export LD_LIBRARY_PATH=/opt/intel/mkl/lib/intel64:/usr/local/cuda/lib64
export LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libstdc++.so.6
Or the equivalent based on where things are installed on your system, and do `make mattest` again to see if the issue is fixed. Note: this issue is sometimes more complicated since during its startup Matlab may overwrite your `LD_LIBRARY_PATH` environment variable. You can run `!ldd ./matlab/+caffe/private/caffe_.mexa64` in Matlab to see its runtime libraries, and preload your compile-time libraries by exporting them to your `LD_PRELOAD` environment variable.
After successful building and testing, add this package to Matlab search PATH by starting `matlab` from caffe root folder and running the following commands in Matlab command window.
addpath ./matlab
You can save your Matlab search PATH by running `savepath` so that you don't have to run the command above again every time you use MatCaffe.
### Use MatCaffe
MatCaffe is very similar to PyCaffe in usage.
Examples below shows detailed usages and assumes you have downloaded BVLC CaffeNet from [Model Zoo](http://caffe.berkeleyvision.org/model_zoo.html) and started `matlab` from caffe root folder.
model = './models/bvlc_reference_caffenet/deploy.prototxt';
weights = './models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel';
#### Set mode and device
**Mode and device should always be set BEFORE you create a net or a solver.**
Use CPU:
caffe.set_mode_cpu();
Use GPU and specify its gpu_id:
caffe.set_mode_gpu();
caffe.set_device(gpu_id);
#### Create a network and access its layers and blobs
Create a network:
net = caffe.Net(model, weights, 'test'); % create net and load weights
Or
net = caffe.Net(model, 'test'); % create net but not load weights
net.copy_from(weights); % load weights
which creates `net` object as
Net with properties:
layer_vec: [1x23 caffe.Layer]
blob_vec: [1x15 caffe.Blob]
inputs: {'data'}
outputs: {'prob'}
name2layer_index: [23x1 containers.Map]
name2blob_index: [15x1 containers.Map]
layer_names: {23x1 cell}
blob_names: {15x1 cell}
The two `containers.Map` objects are useful to find the index of a layer or a blob by its name.
You have access to every blob in this network. To fill blob 'data' with all ones:
net.blobs('data').set_data(ones(net.blobs('data').shape));
To multiply all values in blob 'data' by 10:
net.blobs('data').set_data(net.blobs('data').get_data() * 10);
**Be aware that since Matlab is 1-indexed and column-major, the usual 4 blob dimensions in Matlab are `[width, height, channels, num]`, and `width` is the fastest dimension. Also be aware that images are in BGR channels.** Also, Caffe uses single-precision float data. If your data is not single, `set_data` will automatically convert it to single.
You also have access to every layer, so you can do network surgery. For example, to multiply conv1 parameters by 10:
net.params('conv1', 1).set_data(net.params('conv1', 1).get_data() * 10); % set weights
net.params('conv1', 2).set_data(net.params('conv1', 2).get_data() * 10); % set bias
Alternatively, you can use
net.layers('conv1').params(1).set_data(net.layers('conv1').params(1).get_data() * 10);
net.layers('conv1').params(2).set_data(net.layers('conv1').params(2).get_data() * 10);
To save the network you just modified:
net.save('my_net.caffemodel');
To get a layer's type (string):
layer_type = net.layers('conv1').type;
#### Forward and backward
Forward pass can be done using `net.forward` or `net.forward_prefilled`. After creating some data for input blobs like `data = rand(net.blobs('data').shape);` you can run
res = net.forward({data});
prob = res{1};
Or
net.blobs('data').set_data(data);
net.forward_prefilled();
prob = net.blobs('prob').get_data();
Backward is similar using `net.backward` or `net.backward_prefilled` and replacing `get_data` and `set_data` with `get_diff` and `set_diff`. After creating some gradients for output blobs like `prob_diff = rand(net.blobs('prob').shape);` you can run
res = net.backward({prob_diff});
data_diff = res{1};
Or
net.blobs('prob').set_diff(prob_diff);
net.backward_prefilled();
data_diff = net.blobs('data').get_diff();
**However, the backward computation above doesn't get correct results, because Caffe decides that the network does not need backward computation. To get correct backward results, you need to set `'force_backward: true'` in your network prototxt.**
After performing forward or backward pass, you can also get the data or diff in internal blobs. For example, to extract pool5 features after forward pass:
pool5_feat = net.blobs('pool5').get_data();
#### Reshape
Assume you want to run 1 image at a time instead of 10:
net.blobs('data').reshape([227 227 3 1]); % reshape blob 'data'
net.reshape();
Then the whole network is reshaped, and now `net.blobs('prob').shape` should be `[1000 1]`;
#### Training
Assume you have created training and validation lmdbs following our [ImageNET Tutorial](http://caffe.berkeleyvision.org/gathered/examples/imagenet.html), to create a solver and train on ILSVRC 2012 classification dataset:
solver = caffe.Solver('./models/bvlc_reference_caffenet/solver.prototxt');
which creates `solver` object as
Solver with properties:
net: [1x1 caffe.Net]
test_nets: [1x1 caffe.Net]
To train:
solver.solve();
Or train for only 1000 iterations (so that you can do something to its net before training more iterations)
solver.step(1000);
To get iteration number:
iter = solver.iter();
To get its network:
train_net = solver.net;
test_net = solver.test_nets(1);
To resume from a snapshot "your_snapshot.solverstate":
solver.restore('your_snapshot.solverstate');
#### Input and output
`caffe.io` class provides basic input functions `load_image` and `read_mean`. For example, to read ILSVRC 2012 mean file (assume you have downloaded imagenet example auxiliary files by running `./data/ilsvrc12/get_ilsvrc_aux.sh`):
mean_data = caffe.io.read_mean('./data/ilsvrc12/imagenet_mean.binaryproto');
To read Caffe's example image and resize to `[width, height]` and suppose we want `width = 256; height = 256;`
im_data = caffe.io.load_image('./examples/images/cat.jpg');
im_data = imresize(im_data, [width, height]); % resize using Matlab's imresize
**Keep in mind that `width` is the fastest dimension and channels are BGR, which is different from the usual way that Matlab stores an image.** If you don't want to use `caffe.io.load_image` and prefer to load an image by yourself, you can do
im_data = imread('./examples/images/cat.jpg'); % read image
im_data = im_data(:, :, [3, 2, 1]); % convert from RGB to BGR
im_data = permute(im_data, [2, 1, 3]); % permute width and height
im_data = single(im_data); % convert to single precision
We do not provide extra functions for data output as Matlab itself is already quite powerful in output.
#### Clear nets and solvers
Call `caffe.reset()` to clear all solvers and stand-alone nets you have created.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment