person-re-ranking/caffe/examples/00-classification.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Classification: Instant Recognition with Caffe\n",
    "\n",
    "In this example we'll classify an image with the bundled CaffeNet model (which is based on the network architecture of Krizhevsky et al. for ImageNet).\n",
    "\n",
    "We'll compare CPU and GPU modes and then dig into the model to inspect features and the output."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Setup\n",
    "\n",
    "* First, set up Python, `numpy`, and `matplotlib`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# set up Python environment: numpy for numerical routines, and matplotlib for plotting\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "# display plots in this notebook\n",
    "%matplotlib inline\n",
    "\n",
    "# set display defaults\n",
    "plt.rcParams['figure.figsize'] = (10, 10)        # large images\n",
    "plt.rcParams['image.interpolation'] = 'nearest'  # don't interpolate: show square pixels\n",
    "plt.rcParams['image.cmap'] = 'gray'  # use grayscale output rather than a (potentially misleading) color heatmap"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Load `caffe`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# The caffe module needs to be on the Python path;\n",
    "#  we'll add it here explicitly.\n",
    "import sys\n",
    "caffe_root = '../'  # this file should be run from {caffe_root}/examples (otherwise change this line)\n",
    "sys.path.insert(0, caffe_root + 'python')\n",
    "\n",
    "import caffe\n",
    "# If you get \"No module named _caffe\", either you have not built pycaffe or you have the wrong path."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* If needed, download the reference model (\"CaffeNet\", a variant of AlexNet)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CaffeNet found.\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "if os.path.isfile(caffe_root + 'models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel'):\n",
    "    print 'CaffeNet found.'\n",
    "else:\n",
    "    print 'Downloading pre-trained CaffeNet model...'\n",
    "    !../scripts/download_model_binary.py ../models/bvlc_reference_caffenet"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Load net and set up input preprocessing\n",
    "\n",
    "* Set Caffe to CPU mode and load the net from disk."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "caffe.set_mode_cpu()\n",
    "\n",
    "model_def = caffe_root + 'models/bvlc_reference_caffenet/deploy.prototxt'\n",
    "model_weights = caffe_root + 'models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel'\n",
    "\n",
    "net = caffe.Net(model_def,      # defines the structure of the model\n",
    "                model_weights,  # contains the trained weights\n",
    "                caffe.TEST)     # use test mode (e.g., don't perform dropout)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Set up input preprocessing. (We'll use Caffe's `caffe.io.Transformer` to do this, but this step is independent of other parts of Caffe, so any custom preprocessing code may be used).\n",
    "\n",
    "    Our default CaffeNet is configured to take images in BGR format. Values are expected to start in the range [0, 255] and then have the mean ImageNet pixel value subtracted from them. In addition, the channel dimension is expected as the first (_outermost_) dimension.\n",
    "    \n",
    "    As matplotlib will load images with values in the range [0, 1] in RGB format with the channel as the _innermost_ dimension, we are arranging for the needed transformations here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "mean-subtracted values: [('B', 104.0069879317889), ('G', 116.66876761696767), ('R', 122.6789143406786)]\n"
     ]
    }
   ],
   "source": [
    "# load the mean ImageNet image (as distributed with Caffe) for subtraction\n",
    "mu = np.load(caffe_root + 'python/caffe/imagenet/ilsvrc_2012_mean.npy')\n",
    "mu = mu.mean(1).mean(1)  # average over pixels to obtain the mean (BGR) pixel values\n",
    "print 'mean-subtracted values:', zip('BGR', mu)\n",
    "\n",
    "# create transformer for the input called 'data'\n",
    "transformer = caffe.io.Transformer({'data': net.blobs['data'].data.shape})\n",
    "\n",
    "transformer.set_transpose('data', (2,0,1))  # move image channels to outermost dimension\n",
    "transformer.set_mean('data', mu)            # subtract the dataset-mean value in each channel\n",
    "transformer.set_raw_scale('data', 255)      # rescale from [0, 1] to [0, 255]\n",
    "transformer.set_channel_swap('data', (2,1,0))  # swap channels from RGB to BGR"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. CPU classification\n",
    "\n",
    "* Now we're ready to perform classification. Even though we'll only classify one image, we'll set a batch size of 50 to demonstrate batching."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# set the size of the input (we can skip this if we're happy\n",
    "#  with the default; we can also change it later, e.g., for different batch sizes)\n",
    "net.blobs['data'].reshape(50,        # batch size\n",
    "                          3,         # 3-channel (BGR) images\n",
    "                          227, 227)  # image size is 227x227"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Load an image (that comes with Caffe) and perform the preprocessing we've set up."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7f09693a8c90>"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlIAAAHDCAYAAADvDfQIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsvcuvbUly3veLyMy1z7m3urrZZDfJ5kMkRdrQA4YgyBY8\n8MwDAx4Y8MTQ0CNPPPc/Y8/9F1iwYcCGAUOQH5DtiWRYoiXwoRbJZndX1b3n7LUyIzyIyFzrVFfT\nNqF2ScZOgmTds89Ze61cmZERX3zxhbg7j/EYj/EYj/EYj/EYj/H/fujXfQOP8RiP8RiP8RiP8Rj/\nso6HI/UYj/EYj/EYj/EYj/HnHA9H6jEe4zEe4zEe4zEe4885Ho7UYzzGYzzGYzzGYzzGn3M8HKnH\neIzHeIzHeIzHeIw/53g4Uo/xGI/xGI/xGI/xGH/O8TNxpETk3xGRfyAi/4eI/Cc/i+94jMd4jMd4\njMd4jMf4uof889aREpEC/O/Avw38AfA/An/L3f/+P9cveozHeIzHeIzHeIzH+JrHzwKR+jeAf+ju\n/9jdD+A/B/69n8H3PMZjPMZjPMZjPMZjfK3jZ+FI/Qrwe5d//37+7DEe4zEe4zEe4zEe4/9Xo/4M\nrvl/mysUkUdfmsd4jMd4jMd4jMf4l2a4u3zVz38WjtQfAL92+fevEajUm/HJr/4y4ODO7dNPePrW\np4gIlpwtFQH5ynsGFSQ/M4FSCi7g6qC+/m6o4VWoQwJ7K4qU8zKGowju/uaaAKrK5I8VOf/bzDAz\ncKeYgXXMBgACiAPWKQaiDnb6jD46RcjvMYoGIKg2MHeGKIiBOcUtniF/391xd1T1hBHNY87Mvnqa\nVOMp3amtYGbrGef14rmMMQalNAB67/Rj0Hv8d/xePOMY8TzzGUScYZ2P3x988r2GW3ymqqjquuac\nv1orqlBboZVYflpj7lWcWjZKU0qJF1VU0Sq01vJ6hVrz7xBKKet/RQTJ+R5jYGb03nl9feV+P7gf\n8dnr/sK+xzOawb7vjDHy/pRSlaoFF8OK5zzminEDFVwlPif+rhRBAMUppaBVKQrzT0UdIZ7JGev7\nAI67cgzDfTB8xLyqX94hmAwKCl7XO8gr4yPeq4263pNqBRTF0dJBHMmVs5UNLXNdGyIKnmsRsO78\n4T/4nF/6nW8yhjG6r3UxhmMGNpzeWZ+5n2tUdMQ85jusValN0Cq4GvVWaK2tOQVDS65JE8bIfeiK\nmyKunPbrXL9y3VvusSIuU+PC2jNzD83fXft5xL3rBZvvvSOuGGOtYwCREn+rjhQw64y8HxFB3Ric\n+2Pen7vg1uFwlMYwWfPt1nFx9FZ4qoVSK9ris7YV6qaUm3C7Vd69f0LyXkoVWiuUorn2TxvVe0cR\n2vOGbcJte+bv/Rd/n7/+7/5lVJW2FZ6eNm7vb3Cxc4ax7zv7vocN6J2XDy8AfPz8I/eXnY8f7/z4\nR5/z2Y8+sH8cYYCBegBFGeQacMH7nFSFIRzHgU8DPGy+OPAa//awx733mEMbFAWR833NNQWa66oQ\n+3Kg+ZFqfKWIQhEUu6wLxdxRLfEzFXreyhiD0rZYgw6tFcZlf+utUG9pI1pFWvxhabBtjdtToxSn\nVE4bpRUbjlvYYQd6fqH1WLPHS6d32F9sffbyxZ1CY+zQX3deXzo+TcZwfvQHP+Ib3/0Um+eCCF/G\nJ0aeISKsdYnGmSHiaCHPj2n7GlKE50+feP7WM+35idu7DYDbU+Hd0y3/Nt7lPHfmfjIzxIX9OPh4\n3wF4/dELP/7+5+hQaq30MdZ+0iLr3UoVhvd1PhWpy6aApT1jfV8rTmtKa+NiS+J5anGQA3XiPVdd\nf1dqnGXuBmKMPv+u8Ho/wrZK2Nkf/9ELn//xHQporfzB//pDftr4WThS/xPwOyLyG8AfAv8B8Le+\n/Euf/tr3gNg8j/EYj/EYj/EYj/EY/6KMT7/zxKffeYLNKben/28dKXfvIvIfA/8lUID/7Ksr9uxE\ncIg4c3h47SKCqL7JEa5oV0p63zNCDU84PO0SF7pE8w4R8YnmzzOCVEftjCZx8Es0OSMgufw8PitU\nLWA9PnPF060VERTBXBkMikkE3BBIExpgmRlxO/PDQAUEA5G4t3kvCMxoRgAf2LD1fSBvotHr/4/o\nYUYd1795O6+gqF6eEaUUDXSpaETe+Z2ujltDZEQkIxEBzu8hUQDzQTzOjFok78kjCPXOKIme9Igu\nW2uMMRhutJbvpgkMYYjndxueEVSrupDD9Xx2IodX1C2ikJybcUZSvY+fmBcfxhBBcBiGX9ba/F1R\nZfgZ6ZZELsY44j0hIIqvKDEinbgvxZw3Ed3I6BQdic619f6FjqCYkc8+I/3LPbuDdOaSMgtU18Rw\nh1raiua1CkVG7AkX+jEWslQQfOQcjEE/BMt9YqaM0RndGMMZXRZStPaLCCo1o+N8J6r5b0eJPTWf\nXfFYv+NASsH8RGtEYmG5xb4NRDWf3ZXh4D4CvRMoGBNyjj19rgW52IxrpbIJIMJYG/VEPYq0sAV6\nzrMoSBG0KmKCXp4bCuqOFA1USM7vH92hBHqolBNFdsGGYcM5/KAfhrzG9x+lU7cCzfl4q+x3Z3u6\nAdBaw56EUjulDFq7rUi/tMJWN8rzRn1uPL+7cXve+Oa3P0FLoIFba0gLlGAup310ytYoe2Pfd/Q4\nOPI+n3Ao0OXgNgrv9YntnWFHro175zgG4zBsQHGNyQLcIiNQb4U+nDFszQ1z/aiiwxkTIpjvwQci\nhVLyfMhXUVVAHLOOKoGwlvhQizCRzjBJhq41FbtYRXAcR+Ja8z0dHZGCoIwxIssBOIb6hLwEqYVa\nJ7JitNaoVRE1tPhCxxgjbLgIVZTXY2ccaXNM6H1gHfpujDFRqji7rBuOozNbcgVlRQJ1TzvvImsu\nA1v0zAxI7JF5hklcLwxXnA8zM9C2FtunGKpCuWRw2q3l9WI2WiuQCPvM0rhXeu80GrdEzXfdaduG\n9ELvsb64omMaLyTsxon+1kSgXIiztCjHPtF25RhHnC+DmO+W+15LZBFyDlwKde17gxLvVnyg6ixz\nYoWtKmO977rQSLaO1q/O+szxs0CkcPe/DfztP+t3Vroq333hcph5pBqc8aW/KT9pDPMA16K4COaG\nL6ehBIxbC+h0h2Ya7sTydaYU8xAWPRdtOC9+plryf6V4bMqh69Ab+84wi8WWqa950ooTzgUajtLV\nQZNY+iQkLurpZob/5blxuDiZ+WfhKF5SFW8dJcMy/aeiP+FEXd+Daj2N+zz0AlfHuuAjl4pbOqgj\nrxcOQPuEdYi7h0MJekkFglBwE2w5pnO+E9k/Bq3G3/aZLu1CKdO41NzIMw0Vm9CGI2LUmPi8dskN\nHmkhkYLZnvfn6Sg4vYfDcKZ2Cu4Ds45IHIqS9+J6piCHGFVPR0Ly4WutZ9rZjPMMltjIrpnOONeg\nyLis/XhX81z3aRDS2Ihf8oUzzczcDxJrF8A75gYOVQutKSWNgSqUkimTPnLe8p2MeJ/PP1fpfeBe\nFmg8ukU6tEukp0zWn5kbKhUtEgZK7E2qJVJkkRKL9PR5KniuKbH5/i6pPZ+/Q6zl+VdS0nkvuB+R\nLhU5P/+S83TOEW9syEz7zUNhphrMDNFpV+YVDGmRajZ3tMkKoiAO/sPCsYvMTn7PcFpR+gTgzfGV\n2hUYIIdzN4d+4DO1hSFFaLdKl48cL+94/kb83e12o/eN7RYpPrfO9hR26Pn5idutIc8bZStstxv/\nyt/4LT75xjeotxpOoAi1KVLKOS8Wz1a3Qrkr+gLdWi61QcN48sHeDxxl2yIoADj2QX3ZqR8OjrvR\nD19rXyTSKN7PgG+luWcggkSaXIWap7d6OBJFgypwtdmlBG1BJA7DcKZkrbdMUkcKSwuXlbHWgHsE\nSvOjJkr3gbjE0h4s+kVrNTI+F1s7D/1W5z4+g9je97VOS7mhDmNYOK5zzl6PXGseaU+vDDttXTfD\ne9hwVb2k0Y3t3cYYY63
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f09b97eab50>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "image = caffe.io.load_image(caffe_root + 'examples/images/cat.jpg')\n",
    "transformed_image = transformer.preprocess('data', image)\n",
    "plt.imshow(image)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Adorable! Let's classify it!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "predicted class is: 281\n"
     ]
    }
   ],
   "source": [
    "# copy the image data into the memory allocated for the net\n",
    "net.blobs['data'].data[...] = transformed_image\n",
    "\n",
    "### perform classification\n",
    "output = net.forward()\n",
    "\n",
    "output_prob = output['prob'][0]  # the output probability vector for the first image in the batch\n",
    "\n",
    "print 'predicted class is:', output_prob.argmax()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* The net gives us a vector of probabilities; the most probable class was the 281st one. But is that correct? Let's check the ImageNet labels..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "output label: n02123045 tabby, tabby cat\n"
     ]
    }
   ],
   "source": [
    "# load ImageNet labels\n",
    "labels_file = caffe_root + 'data/ilsvrc12/synset_words.txt'\n",
    "if not os.path.exists(labels_file):\n",
    "    !../data/ilsvrc12/get_ilsvrc_aux.sh\n",
    "    \n",
    "labels = np.loadtxt(labels_file, str, delimiter='\\t')\n",
    "\n",
    "print 'output label:', labels[output_prob.argmax()]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* \"Tabby cat\" is correct! But let's also look at other top (but less confident predictions)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "probabilities and labels:\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[(0.31243637, 'n02123045 tabby, tabby cat'),\n",
       " (0.2379719, 'n02123159 tiger cat'),\n",
       " (0.12387239, 'n02124075 Egyptian cat'),\n",
       " (0.10075711, 'n02119022 red fox, Vulpes vulpes'),\n",
       " (0.070957087, 'n02127052 lynx, catamount')]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# sort top five predictions from softmax output\n",
    "top_inds = output_prob.argsort()[::-1][:5]  # reverse sort and take five largest items\n",
    "\n",
    "print 'probabilities and labels:'\n",
    "zip(output_prob[top_inds], labels[top_inds])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* We see that less confident predictions are sensible."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. Switching to GPU mode\n",
    "\n",
    "* Let's see how long classification took, and compare it to GPU mode."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1 loop, best of 3: 1.42 s per loop\n"
     ]
    }
   ],
   "source": [
    "%timeit net.forward()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* That's a while, even for a batch of 50 images. Let's switch to GPU mode."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10 loops, best of 3: 70.2 ms per loop\n"
     ]
    }
   ],
   "source": [
    "caffe.set_device(0)  # if we have multiple GPUs, pick the first one\n",
    "caffe.set_mode_gpu()\n",
    "net.forward()  # run once before timing to set up memory\n",
    "%timeit net.forward()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* That should be much faster!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5. Examining intermediate output\n",
    "\n",
    "* A net is not just a black box; let's take a look at some of the parameters and intermediate activations.\n",
    "\n",
    "First we'll see how to read out the structure of the net in terms of activation and parameter shapes.\n",
    "\n",
    "* For each layer, let's look at the activation shapes, which typically have the form `(batch_size, channel_dim, height, width)`.\n",
    "\n",
    "    The activations are exposed as an `OrderedDict`, `net.blobs`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "data\t(50, 3, 227, 227)\n",
      "conv1\t(50, 96, 55, 55)\n",
      "pool1\t(50, 96, 27, 27)\n",
      "norm1\t(50, 96, 27, 27)\n",
      "conv2\t(50, 256, 27, 27)\n",
      "pool2\t(50, 256, 13, 13)\n",
      "norm2\t(50, 256, 13, 13)\n",
      "conv3\t(50, 384, 13, 13)\n",
      "conv4\t(50, 384, 13, 13)\n",
      "conv5\t(50, 256, 13, 13)\n",
      "pool5\t(50, 256, 6, 6)\n",
      "fc6\t(50, 4096)\n",
      "fc7\t(50, 4096)\n",
      "fc8\t(50, 1000)\n",
      "prob\t(50, 1000)\n"
     ]
    }
   ],
   "source": [
    "# for each layer, show the output shape\n",
    "for layer_name, blob in net.blobs.iteritems():\n",
    "    print layer_name + '\\t' + str(blob.data.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Now look at the parameter shapes. The parameters are exposed as another `OrderedDict`, `net.params`. We need to index the resulting values with either `[0]` for weights or `[1]` for biases.\n",
    "\n",
    "    The param shapes typically have the form `(output_channels, input_channels, filter_height, filter_width)` (for the weights) and the 1-dimensional shape `(output_channels,)` (for the biases)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "conv1\t(96, 3, 11, 11) (96,)\n",
      "conv2\t(256, 48, 5, 5) (256,)\n",
      "conv3\t(384, 256, 3, 3) (384,)\n",
      "conv4\t(384, 192, 3, 3) (384,)\n",
      "conv5\t(256, 192, 3, 3) (256,)\n",
      "fc6\t(4096, 9216) (4096,)\n",
      "fc7\t(4096, 4096) (4096,)\n",
      "fc8\t(1000, 4096) (1000,)\n"
     ]
    }
   ],
   "source": [
    "for layer_name, param in net.params.iteritems():\n",
    "    print layer_name + '\\t' + str(param[0].data.shape), str(param[1].data.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Since we're dealing with four-dimensional data here, we'll define a helper function for visualizing sets of rectangular heatmaps."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def vis_square(data):\n",
    "    \"\"\"Take an array of shape (n, height, width) or (n, height, width, 3)\n",
    "       and visualize each (height, width) thing in a grid of size approx. sqrt(n) by sqrt(n)\"\"\"\n",
    "    \n",
    "    # normalize data for display\n",
    "    data = (data - data.min()) / (data.max() - data.min())\n",
    "    \n",
    "    # force the number of filters to be square\n",
    "    n = int(np.ceil(np.sqrt(data.shape[0])))\n",
    "    padding = (((0, n ** 2 - data.shape[0]),\n",
    "               (0, 1), (0, 1))                 # add some space between filters\n",
    "               + ((0, 0),) * (data.ndim - 3))  # don't pad the last dimension (if there is one)\n",
    "    data = np.pad(data, padding, mode='constant', constant_values=1)  # pad with ones (white)\n",
    "    \n",
    "    # tile the filters into an image\n",
    "    data = data.reshape((n, n) + data.shape[1:]).transpose((0, 2, 1, 3) + tuple(range(4, data.ndim + 1)))\n",
    "    data = data.reshape((n * data.shape[1], n * data.shape[3]) + data.shape[4:])\n",
    "    \n",
    "    plt.imshow(data); plt.axis('off')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* First we'll look at the first layer filters, `conv1`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlEAAAJNCAYAAAARaCA+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsvWmQbdd5nrf2dOaeu++Mi4sZIAiCIgnKFCWKkkmJiRWX\nklhlp+xyVSouy3Ek2dZEABxAkAQoUaKHuOKqxLbsKs+uisqSIkWWLFOkbHOeQZCYL+489O3x9Jn2\nlB+Mf3zf+6L6aMtuVyrv82+tWmefPay99u4+z3pXVNd1EEIIIYQQfzji/9I7IIQQQgjx/0X0EiWE\nEEII0QC9RAkhhBBCNEAvUUIIIYQQDdBLlBBCCCFEA/QSJYQQQgjRAL1ECSGEEEI0QC9RQgghhBAN\n0EuUEEIIIUQD0qP+wvc99vMQkR61OracZPC5Ok6wLrF1eT6FNsWU1I0ntqLENuztMk3tfv7yJ34Z\n2nzog4/hB90RTw5G0GRvOMT9rEpTjlK8XK1W25TbnQ60icm5a2V2W1GIoM1HPvIU1P3M//ITpjwr\ncmgzHuHxRZHdfpziNY4iPOuZ6xtpu41tMtxW8NvCwwsf/0U8vic/+iFTrqsK2tRYFerSXuQyL6FN\nUZK6orDbyXHjRY7nOI7tAfnzG0IIf+Pv/K9Q9/ijj5tymmGfIpchtNx5768sQ5vF5SWom7p7cvPK\nZWhz9fx5qBvu7dvvb+E1/nv/8J+a8gc+9GFoU1d4zoM7V1m7BU2yDvazdtv2xSwlY1Jx+DWekjFp\n6sekEMKTT3zQlD/+ceyvSYT7kLXcvZ2QcYPeM/a81KRPBTKWzMYzW57OoE1R4coYj/7cz5jyL//S\nx6HNvusHIYRw6/qmKR+QsXNlfQXqFldsXbvbhTZJixzfzF6/8cEY2lQl3rcfe/qjpvzzP//z0Kbb\nwb7XymxdQq5flNibtCY37WiI4/Do4MCUqxKvS0KucRTZffj4J7AvPv74+6Audv2zxk2HyT5ev8Gg\nZ8rLZ45Bmxe+/AzUtUr7BQ++4xFo87nP/geou+30aVN+4n1P4o7Sp4j+EyWEEEII0Qi9RAkhhBBC\nNEAvUUIIIYQQDThyJyqQ347DxPoAcYZtsn4f6npLi7ZND3/jLojT4n8X3t/chDY18asC2XXPbIZO\nxP7Oninv7e7iB4lf0V+13snKxga06bnf9dlbcVzj7975zHoLBXE5GGVhz0tMfotvE8ckdb/zZ50e\ntGEnOHF+XJLhtr0fEEIIlTvmOJ7v7wXvCBUF7hPziGrXr+OkgDYVcUWC86TyEj/n3bgQQojcJZ33\nr6GR8zmSGH/m975VCCFMEueBkOu+uIweyvKadRlS4jZNxuiYjJ03xDw7T4tc47SN1ypt2T7kfa8Q\n+HnxbtNwewvaDHfw3p7m9rqXxI2L5riCBRk78xq3lbsxL07xc9Mc+5nfhyjBMSkmdfnMOnuzGTp8\n+Qy/D7ZNPMnJiDitbvtt4q91+wNSZ8fKinhakzF+H4yVOd7HzImCNmQcLsm9PXMOZELaJE4uSsi5\nS1Lsw7UTdCsieCZE/anDHM8H0oVLd+46bXyOt9dXoe65z3zVlL/7zBlo85YffifU/f2/9gumfP8b\nH4Y2r3vkzVD37c98GermRf+JEkIIIYRogF6ihBBCCCEaoJcoIYQQQogGHLkTlZPMm9r9xs0ydqry\nBtQtrtnfUlfOnII2C8fRI+ovLZjyYBF/P98nvkNOfp+Hz+0fQN3ByDofbedyhRDC+mnc9xPnzpry\nKsnmKZ07crC9B20OiKdRFe63ceL+MLz7k7bxt3i2pdr9zl4RV435AT7jhqhjIU6I11O7XCriTTE6\nLmeL+R1pil6WzyMaj0jeE/HOcudXTGbEySAuVeoOpyZ+ByNxHyyJq+LdnxDQC2H36PLqGtQtbayb\n8mrvJLSZjNF3Grosta3ZNWjjGe9jP4+IhzIe2Vya2QTP+d4u3kfbm3ZMmJB8IpZL1Vu09/tgFR2Q\n/uIC1HlSku3kfaQQ2L2G22K5cLXzq2LWhmQIlewLfJsS9xO/H+sOSIbQdGod2oUVHE97C+hcJt7H\nIx4TUeHC7MDuO/Wf5rj9WCYT0eNC5ITHlLh+NbhMuO2U5Ev5PjQriGvIxhLi3nkiMjinrg/dvIA5\ncW/4wbdB3f71W6b8r/63fwRt/vqn/znUffGHP2nK//gjfwvafPi3fgXqvvXlr0DdvOg/UUIIIYQQ\nDdBLlBBCCCFEA/QSJYQQQgjRAL1ECSGEEEI04MjF8owskDt1YmJOFuMcbaE0euPVV025+/yL0Ob4\nHeegbv3cbaacEgmx10fRcxqhUOxhct2x2+3ihqfvuQvanLnrDqhbGNh92L9+E9pcOX/FlG/dwDYj\nIrsXzuL0izm/FoVbcDglQXcVC4dz5wXFyBBqInX6OhbWyBZYzgsrQqet+bp6zwW2ZiTcMyXBdoUL\nL5xMSLAmoXSSajFjQX4sgNO3mS8sNXPBkv77QwhhMsJ92HWBtJvXcKIHWwx30U2GOO4mS4QQwqnb\nse/7xVNzsmgvfIYI6pMh9v3Rvl3UdkqCPKfk+iVuQd6lFRTEB0RyXjt1wpQXyOd6g8PFcub7VjX5\nO7iy++nv2RBCiGIiVfspIUy8Jv0sn7rtE9E8JxMmPLMRjvszdt2TwxeQ7g9wslDl5GsfahlCCDU7\nPi/vE6+8lc0zfpIAVzaZpnTtiHGfuvGUTQIIZDJNr2ufv1VFJpGQINaSSPHwORJUunDCTjYZbu9A\nm69++jNQ96N/4X8w5c/9xiehza888Qmo+8n/42lT/rF/+gZo8x/++b+Guvsewnbzov9ECSGEEEI0\nQC9RQgghhBAN0EuUEEIIIUQD9BIlhBBCCNGAIxfLe8soXg5c3XSEq7qnLRT3Zhdtu+uvnoc2W5cu\nQd3GeSt6b9yJYmtnFdPBkzYK6J4Tt92GdedOuzYo18YVioEXv/ZtU/7253Gl6cvnL7oN4T71SCJ7\nZ8Ge85pFgTNcmm5EIn4TmpRr5U8meqcR7nya2nYJS+9lu+lOhJeCXwu/Inya4dZrYsD7lG/2bQkR\nPWN3PplEylKUfVVEc+KRXt9N7PDfH0KYTvH+K51we/PKFWgzPkCJuzuwq7YvLON9tbi2AnUnz9r7\naLi7DW08bTIZpNXCiSwr6zZFvdvHleWzLt7rHSd/95bw+/pkVYF+326LydLjPZIc7cjJqgIV6S9+\nVQHmBMdkYkfp6qpAJojEeN/6RP2I3JEFkbhxO3heWNJ5x4nkvQFev3YPr/vMrQ5QHuC2R+TZUzqj\nPyVjCbu3PTW5kVnYuz/vVOn2+0BWbahy3HjipfwMrydbsaAmAjrsEjmWm9fshJS734IC96d+9beg\n7iuftc+6H//oz0GbJ/8C1r3nL/9ZU/6zT/4VaPPZf/LbUPfOO38M6uZF/4kSQgghhGiAXqKEEEII\nIRqglyghhBBCiAYcuRM1neJvq92+DThcO41ewfKxDahbO3nclH34ZgghbJJVo3c3bVBgVeBv/0sn\nT0BdexF9Ls/GCfxcK7G/4V/4+rPQ5qWvfBPrXLu9LQwq66/afVq/7Qy0WVjDcL+Wc39yEujIiN1v\n/zHxmKKEhMO1bUBlq4UhnSxs0686HhMnitlAsfMPWCAnI2s7l2KC3kTBJBN3HrKMhYLiAcYu5DRJ\nmXOC18afh4hcB0ar4/aLrbxO6ioXArp76xa0uXbhAtQ987kvmPLy+hq0uevh10Ndv2t9o5W1Y9DG\n0+p2oS7podvUdnULa7hPGQlrHCwvmXJ3AV2cmPxdOh3acM+bVzEQd+vadajzsEBV1jci19eZN1Uy\nt9CNARFxfyoS0jmd2pDMhIVK+sBKwpiENVZEGuo4r2+BeGitNo4vpQ91JuP+iISsem0wnStYE2H3\naE3CNsva1pXEUarcOMyyktl+ztz3sX2KqT96eNjmwtIS1F09b53kKxfRpfyeH/5+qPvsH3zOlE/+\nd38S2rzjj78d6v7N3/+
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f09692b9ed0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# the parameters are a list of [weights, biases]\n",
    "filters = net.params['conv1'][0].data\n",
    "vis_square(filters.transpose(0, 2, 3, 1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* The first layer output, `conv1` (rectified responses of the filters above, first 36 only)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlIAAAJOCAYAAAB8y+mTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsvVmsnWd5Nnyvedh77cnb23biKY5xppI4sRNCEigmFNLS\nMpSWqQWaFlGplTipkHrQg6JKPekkVepZUfVBUaqoFQFBCxQKgRA5NJMz4STO4Nixt2Nv73HtNa/1\nH6z/utf1Ps+93vXunfzf5pee62Tb652e+bmf655SvV5PAgICAgICAgICNo70VhcgICAgICAgIOD/\nrwiCVEBAQEBAQEDAJhEEqYCAgICAgICATSIIUgEBAQEBAQEBm0QQpAICAgICAgICNokgSAUEBAQE\nBAQEbBJBkAoICAgICAgI2CSCIBUQEBAQEBAQsEkEQSogICAgICAgYJMIglRAQEBAQEBAwCaR3YqP\nplKp2Lw02Wy/WN1uN/KXMT4+Lq1WC+8TEZF6va7Xc7mcXms2myIikk735cZer+d9Q0Sk0+lsojZ9\njI2NRcq+vLzs3ZPNZrWsKDsjk8l45cD9o1L58H2oZ7fb1edwPSlSqZQ+i3Kl02mz3MViUUSi7Q9w\nWfL5vIiI9od7Hf+3+tuC1V5o//HxcRERWVlZMd/ntmsmk3lT/e/Wg8cYrlnvH1ZXjKeZmRm9b2Vl\nRUQGY6der2ubTkxMiIjI2tqa9gPqeODAAf33qVOn9Bt4pt1ui4jI+vq6lhVl7/V63tjJ5XKyb98+\nfUZE5NVXX/XqkEqltB6oZ6PRkNnZWRERKZVKQ5+dnZ2Vm266SURELly4ICIizzzzjF5HOXO5nPb1\npUuXYsc52mp6elpWV1cj5eKxWy6XRaTfPvPz8957du/eLSKDfqjVatq3GJPoKxGRe+65R0RETpw4\nIefPn4+8a9S4Q/tVq1Xv2l133SWHDx8WEZHXX39dRERee+01eeyxx4a+D9i/f7+84x3vEBHR+3ls\nAF/4whf0+vPPPy8i/TEGYP5ks1n9N9o0l8uZ6wXGFvoDY0hk0PblclkmJydFROSll17yno1bR0UG\nYwvX0+l0ZN0ZVibMBcaHP/xh+dGPfiQi0X4FpqamRKS/5qP/8Z5sNqu/NRoN73uFQkHrw+3qYnJy\nUveUUWs6vgfwPrAZ4Dv4m3R9HvU+lGnYmo/xgb28Xq9veI3Gt3iecX3c37hcDPrNbPQtEaRGwR3M\nlUpFFxI0+Nramk4WDJxCoaCD1Zpg3Fm4fuDAAREROXPmjNnQVge7m6bIYKFDp3NZuF54FmCBhQcJ\nBhG+wc9aZXozAkE6nfbebQmY/F4WYlwBCouhSHSDshYyty7DJikWHqDdbnv1HB8f1036zJkzWk73\nnZlMRtuL2z5OMOMxicUP7+U+5G+xUBVXN7wP32o0Girk7N27V0RElpaWvE2fDwmXLl2K1E9ksMAf\nPXpU78OmlE6nI4s4fkNZrb7Ce4vFoi7q2MAZLDBjk+RFGHMkbnHPZrO6saAvGWiLbDYr27ZtG/oe\nBuZ8tVrVcqGdx8bGZGFhQUQGG/vb3/52ue6660RE5MEHH9Tvnj17VkREbrjhBhERuXz5sgpcXCfU\n+Qtf+IKIiPzt3/6tJ0gVi0VPSGIBxBKggMcee0weeughEen3sYjIddddp4Ip6mMBwqeIv/EylpeX\ndZxMT0+LiJgbfqfTiQjfIv1xgH/z/MG/ISQePHhQhTi0fa1Wi1wX6Qt6eJbLbB3QrrjiChERWVxc\n1HoA6JdsNqvtjPE0PT2tzwAPPvigvq9SqYhIdNwvLS2JSF94q9VqIiIqBNZqNW8fQHtx2bdt22YK\nsoC7/g0D7y/uGsfgNcsCysVr4ZsVoNxyxb03m81qf8YJwKlUyjuo8rhD//L4s+bom805vOWClCvt\nlstlb6Kurq5qY2EBWFtb00GLa8ViUTdxPuW4kjuzBS+//LKI9Ac+GhOnDus03uv1YgeUezJlFItF\nHaD4Ri6X08nJC587eHiTw0ZkCYsWm+UC7cVMkzXZ48DfgUDLAoZ1cnMxOTmpfW0JfyhnKpXSiWDV\nCfVIpVIeu2EJf8P6kE/XIv3JZ51Q0e5xC9Ww69bERT150mMDhdBUrVaVRYk7uabTaRUOMK46nY5u\n4Fwm9BcwaqHkxdUSoN7//veLiKhQ8dRTT2mb79q1y/uuu2GJRBlF1NNid4EdO3bI3NxcbLkBCAJY\nN0QG8/DQoUOe4PHII4/I5z//eREZsFBf+9rX9DqEVzBmLlxG9w//8A/l4Ycfjtyzc+dObUv0dbFY\nHMq0iAwOf1i7REQuXrwoIiJ/8id/Ir/4xS9EZLCe8AYPsJAb963Tp0/rs6NYbXeOVqtVHds4EDSb\nTR0f6Nft27fLr/zKr4jIgHXs9Xpy+vRpERG5+uqrRaTPzl6+fFlEoussxjsLhzgwHDt2TET6fYn6\n8hxwD0/1el2FIJRvaWnJO5xYmJqa0rbCs4cOHVLBm/cklAFzYHJyMpZ9jBOoGdYczmQyXj15TbXu\nw7VhbD7KDwFzbW3NW5fiSIhh92HscNni2MJer+ftHXEaD/5Gr9cbKmTielIEG6mAgICAgICAgE1i\nyxkp99S+tramJwswL4uLixGVnkhUr4q/6+vrKr2yBO6eDAuFgp4OcApYXl7Wk431LMCnIktyxb+L\nxaKn8qrX6/obnz5wcgSD0G639WSDsmQyGT05xp0gR8FqN/e6SNSey1L34L6pqSltD/fUOwrLy8vm\nKddV/bAu2zoloM+HUcBxNkqMJNR1NpuNqJXwXBJ16tjYmDKmKGutVtPyWUwJmIb19XXvxMfvAzsy\nPT2tzBFOz9VqVU/FXFecmrnsGJdQl83Pz5s2hug3fOvmm29W5uCpp56KfEdkoEITGcxhHsfoczBX\n7XY7oq50gTVi9+7diVUOsOWbm5uTkydPRq5BPeMCbBMYk23btul8BUM4Cn/1V38lIiJ///d/rwwJ\nVIWTk5MRGyCRPgPzyiuviEiUjUP5rVM9TuulUkn27NkjIgPbp1qtJtu3bxeRwXhiezJrTQVeeeUV\nUz1iwTr949+vvfaaiIi84x3v0HmDMXnq1Cm59957RSRqBwegfcrlsmeP2el0lNXBOMVYEhG1bdq7\nd6+WAeD6XHnllSLSV9mBYWegz2+88UYR6a9JripuZWVFbRqxJp47d07LzIwUwAyQ1a9ALpczbVCT\ngNWu+Ea73dbfWG3ofmN8fNxTsfEafO7cORHpjzvUhddHd8yMmqvW+s7vcNto1PviNA8M1n5sRoUZ\nGKmAgICAgICAgE1iyxkpC+7J+8CBA3r6w4mKpWhmIlwJ2DLmZuYKp4RMJqMsAE7ZrEPF38uXL+tJ\nHwxCs9n07ltaWtJTDr7BNiGWzts64bKhN6RmsAbr6+sRg2eR/qktzqZhlLSNb1jsDrMx+IuTF67j\nGnvhud9lnbfLFvZ6PZNxYwNRkSibEWffVS6X9Ru4j081zMZYRuGubUGn01Ebn42eENkRAO2bSqVk\n586dIjI4STNbgrKyAT+XHfMBWF1d1fGB+qyuruoYBOu5urpqjjcwRzhZX7x4UU/U+Ntut9VmCCzU\n7OysMkjs6Yb6wm5nZmbGK3OhUNA5h7nfaDTMOQLDY8zBSqVishgWMP8mJia0D7FevP3tb5ef/OQn\nIhIdW7AtQ7lWVla0v+Js1RiPPvqoiPTtdN773veKyMC+yZqjN9xwg7b18ePH9XeMN8so+dprrxUR\nkW984xvmuATDiP5/+eWXde6CgdmzZ4+WB2vhMPsvF7lczrNlsebxI488ov+GV+a5c+fkgQceEJGo\nYbkLZnT2798vIn2PT7ZvEumvrR//+MdFROT+++8XEXv+sJE77NRKpZKORbRLsVjUsQOG64477tDv\ngVmrVqvyvve9T0REvvnNb4pIVMNi2fqg7EtLS9o3WKd4DR61bru2xu4z1hqJclj2SC5byWBHFWbW\nk2CYJ2cc8K18Pu+xYr1
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f09692c7350>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "feat = net.blobs['conv1'].data[0, :36]\n",
    "vis_square(feat)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* The fifth layer after pooling, `pool5`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlEAAAJMCAYAAADaNPObAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3UtsHed1OPB7RfFNSRSp98NWHCdWbDexHQdJ0wRBijYN\n2gJtV+0m6KpAgXbRoECALrrsOosuuyjaTdAuUgRdGC0cGAlcN02c1DXk+Blb0Yt6kBQpUiRFUsx/\n0aBo9R398+mbuXOHV7/fzgffzBzOnXvv0fjMud2f/exnHQAAHsyeficAALAbKaIAAAooogAACiii\nAAAKKKIAAAooogAACiiiAAAKKKIAAArsbfqAe/bsSaZ7jo+PJ+vW1taSWLfbTWK5w0KjddH+mlAl\nl5mZmax10fnb2NioNZe6DUoun/zkJ7PW/fCHP+x5LnVrSy79ymPv3vQjc2trq9ZcomNsb29nbVv3\nedmzJ/139s7OTs9z+dznPpe17s0330xiCwsLteZSt9xcRkZGktjY2FgSm5qaSmI3btzI2t/q6mpW\nLpOTk0kscvv27ax1kba/Rp1OJ0zGnSgAgAKKKACAAoooAIACjfdERf+vMerfyd02Mjw8/EA5tVX0\n/4OjHoWJiYkkNjQ0lMSinihoyhNPPJG17u233+5xJuVye5PafozII488ksSiz6Co3yb3MzzXs88+\nm7XuypUrSSzqidqNNjc3s2K3bt3K2l/Uu5erSq/ToHMnCgCggCIKAKCAIgoAoIAiCgCgQOON5U2I\nhtXtRtEQtWjoWTQwrYnmytOnT2etu3jxYo8ziRvuI7mDAqt4/fXXk9hHPvKRJPbkk08msR//+Mc9\nyalpo6OjSezUqVNZ27a5sZxmBiC+9NJLWevm5uZ6nEn8mRt9xywvL/c8l36Jvot++Zd/OWvbF198\nMYnlPiS2G7gTBQBQQBEFAFBAEQUAUEARBQBQYDA6sO/RRPMw7WoOjBofI7nTfav41Kc+lcQ+9KEP\nJbGo+X9QGsuja6MtU49zH4hYWlpKYisrK3Wn0xoXLlzodwr/49y5cz0/RvQwSvTdEf36Q/QrEevr\n60ksmjC+G+U+uPMwcmYAAAooogAACiiiAAAKKKIAAAoMZGP5yMhIv1OoRdTEGjXn9quR/tKlS305\nbqSJhvFcUZN7NOU5akQdFFFD7fe+970+ZDK4NPs2I/rM3djYSGKD0kQe2draSmLRLzNE52B6ejqJ\n3bx5s57EWsC7EACggCIKAKCAIgoAoIAiCgCgwEA2lu/GqcJnz55NYpOTk1nbLi8vJ7G1tbUkduLE\niQdPjAcWNWFGjZRzc3NNpNMXo6OjSSya5B6JHhKoc5r2xYsXa9tXP+3dO5Af342JpupH08kPHDiQ\nte0gN5Zvb28nsePHjyex6FxFD9B8//vfryexFnAnCgCggCIKAKCAIgoAoIAiCgCgQDdqkOuxxg8I\nAFBB+rMTHXeiAACKKKIAAAooogAACiiiAAAKND7yttsNe7MSe/ak9d3Ozk7xcaMG+txc6tb2XMbG\nxpJYNIn2Ix/5SNYxPvjggyQWTVlfXV1NYm06L9PT00ks+jtyRdd4NIX6zp07SSz3vHz4wx/OWveT\nn/wka110XqKc7969m7W/3Knb905Mbvt7KDeX/fv3J7FoYnsTudQtymV4eDiJRdOwm8ilTedFLnEu\n0fdONBl+Y2Mj6xjRd1s0pT76Lrofd6IAAAooogAACiiiAAAKKKIAAAo03liea3x8PIlFjbhR023U\neEa+6JxOTEwksZmZmaz9RU3LfZiUX9mDNBvmiB6U6Ne1Ozo6msSi6yCS20QeNVFHomtjZWUla9vd\nJvccD4ommsgZHNGDJyMjI0kst7E82l/UWP4g3IkCACigiAIAKKCIAgAooIgCACjQisbyqFHs8OHD\nSSxqwmyiMfPs2bNZ6+bn57NibRdNrL169WoSu337dtb+ogbq3EnVuaLG98ja2lrxMXIbqNskauqP\nmsjpj9yHOCJVruVI3dPtqV/ue3dQHlhYWlpKYtEDZrmiz/Cqn+vuRAEAFFBEAQAUUEQBABRQRAEA\nFGhFY3kVbZpOPijNfNHE6GgibO6U2NnZ2eJtd6No2n5kfX29x5l0Oo8++mgSm5ycTGLRNPYLFy7U\nmsutW7ey1kUPmtRpamoqidU9jZ7BkTvRuokHT6KHfgbZvn37klh0nnPfv8PDw0ks9/P6ftyJAgAo\noIgCACigiAIAKKCIAgAo0IrG8qg5vE1N2ufPn09iUbNh7gTvh83CwkLPj1H39OYqDh06lLXu8uXL\nSSya7l7F2NhYrfurW5Xpw6Xa9FBD1NQaTaVuIuf333+/58fYjXLfQ018/m9tbfX8GG0SPQSS+4BK\nU9yJAgAooIgCACigiAIAKKCIAgAo0HhjedQoNjExkcSiqaRRU93MzEwSq7sJM2r2jRrLo4bQvXtb\n0bv/QKK/re5pvI888kit+2vCgQMHktjy8nISqzJVOJrQG4maXaPXKHcqetTkfvjw4axto+bwaCr6\n8ePHs/Z3/fr1JBa9t0ptb2/Xtq+qoteniUn2keiXCmjXA0NNTEWPPlv69TDG0tJSEqvyekRN6VXf\nb+5EAQAUUEQBABRQRAEAFFBEAQAU6PahmVD3IgCwm4RPDLkTBQBQQBEFAFBAEQUAUEARBQBQoPFx\n2s8++2wSW11dTWLRZPPIpUuXklg01XVxcTGJVZksXUXUzN+mXA4ePJjEokna0ST3O3fuJLFoIn20\n7urVq0msTefl5MmTSSyaqLu2tpZ1jGgK97Fjx5LY+fPnk1jueZmdnU1iCwsLWdtG2nLttiWPTkcu\n91Mll2eeeSZr3WuvvdbzXOoml1iVXHJ/GST6tYLx8fEklvsZ3um4EwUAUEQRBQBQQBEFAFBAEQUA\nUKDxxvIPPvggiS0vLzedxkAZGRlJYpubm8X7i5qlp6enk9jMzEwSm5+fT2IXLlwozqVNrly5Uuv+\noub6ixcvFu8vav6PHuSIvPjii8XHhbpFn2lRk3F0za+srPQkJ9oreiAnEj2IduDAgUrHdicKAKCA\nIgoAoIAiCgCggCIKAKBA443lDzIJlNTx48ez1s3NzdV63KhZM2r+jKZw04zoNXrjjTeSWDTRn/6I\nGqMjD1uzdO7DKNEE6kH22c9+NmvdK6+80uNMmhE1fY+NjSWx9fX14mNMTk4Wb9vpuBMFAFBEEQUA\nUEARBQBQQBEFAFCg8cbyn/3sZ00fcqBE08SbOKd79qT1dtRY3q9Gz8OHDyex6CGG27dvFx9jaGgo\nidXdpL2zs1Pr/qLXLYpBm9y4cSOJDQ8PJ7Eqv8zA7rR3b1q2XLt2rXh/GxsbVdJxJwoAoIQiCgCg\ngCIKAKCAIgoAoEDjjeUP24TZulWZzFrFrVu3ktjExEQfMolFU2yjWJXG8t3o4MGDWeuuXr2axEw2\n772HbRJ5ruiXD/bv35/EFhYWkljdD2e0yaBMIs8Vfe9MTU3Veoz5+flK27sTBQBQQBEFAFBAEQUA\nUEARBQBQoPHG8odNE9Oho8nhUXNllab+aEpsriYmfTchOs/RFOVINFU+aswfHx/P2t/k5GQSiya0\nr66uZu1vN74epQbleqxb1LCbe/3kyj33VR5aiZrS2Z2iz9zoVztyzczMJLGqU+/diQIAKKCIAgAo\noIgCACigiAIAKNCNGl57rPEDAgBU0I2C7kQBABRQRAEAFFBEAQAUUEQBABRofGJ5txv2ZvVc1ECf\nm8uhQ4eSWDQ1NZoIHk3ojdbl5hJN471z507WtpEq56VuUS5Hjx5NYtevX6/1uNGU8Gj6d7RuY2Oj\n1lwibX+Nolymp6eT2Mc//vEk9sEHHySxy5cvJ7F7J/DvxnPShLon40cWFxeTWDRBP5p23qbzIpc4\nlyNHjiSxGzduFB8jmjoefQdGk+uj8xLFnnrqqaxczp8/n8Si6/RBHrhzJwoAoIAiCgCggCIKAKCA\nIgoAoEDjjeWR2dnZJHb
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f09691fce50>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "feat = net.blobs['pool5'].data[0]\n",
    "vis_square(feat)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* The first fully connected layer, `fc6` (rectified)\n",
    "\n",
    "    We show the output values and the histogram of the positive values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlgAAAJPCAYAAACgtar/AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XncHFWd7/Hvj8WLKwwjBlRmcFS8LjMD6IBeXB4UHEAH\ncZzrFWeUwXV8KeCGgqIEFQFX3McFMSKyCIowgiQCD4sIYUkgEDCABBIhT4iEhBCWhJz7R3eTTqe6\nu5Zzqk5Vfd6vV17pp7v6nFOntl+fc+qUOecEAAAAfzapugAAAABNQ4AFAADgGQEWAACAZwRYAAAA\nnhFgAQAAeEaABQAA4FmqAMvMtjKzM83sZjObb2a7mdnWZjbLzBaY2Uwz2yp0YQEAAOogbQvWNySd\n55x7oaR/kHSLpMMlzXLO7Sjpwu7fAAAArWfjJho1sy0lzXHO/d3A+7dIeo1zbsrMtpU06Zz73+GK\nCgAAUA9pWrCeI+leMzvJzK4zsx+a2ZMlTXPOTXWXmZI0LVgpAQAAaiRNgLWZpF0kfdc5t4ukBzXQ\nHeg6zWA8cwcAAECd4GmcxZIWO+eu7v59pqQjJC0xs22dc0vMbDtJSwe/aGYEXQAAoDacc+YjnbEt\nWM65JZIWmdmO3bf2lHSTpHMlHdh970BJZw/5fuv+HXXUUZWXgfVmvVlv1pv1Zr1Z72z/fErTgiVJ\nB0s6xcyeIOl2SQdJ2lTSGWb2bkkLJb3Va8kAAABqKlWA5Zy7XtI/JXy0p9/iAAAA1B8zuQcwMTFR\ndREqwXq3C+vdLqx3u7R1vX0aOw9WocTNXMj0AQAAfDEzubIGuQMAACAbAiwAAADPCLAAAAA8I8AC\nAADwjAALAADAMwIsAAAAzwiwAAAAPCPAAgAA8IwACwCgqamqSwA0CwEWALSAc9KcOcmfLV0qbbtt\nueUBmo4ACwBa4MorpV12Sf7skUfKLQvQBgRYANACjz5adQmAdiHAAgAA8IwACwAAwDMCLAAAAM8I\nsAAAADwjwAIAAPCMAAsAAMAzAiwAAADPCLAAAAA8I8ACAADwjAALAADAMwIsAAAAzwiwAAAAPCPA\nAgAA8IwACwAAwDMCLACtddFF0re/XXUpADQRARaA1vrkJ6WDD666FACaiAALAADAMwIsAK1lVnUJ\nADQVARYAAIBnm6VZyMwWSlop6TFJa5xzu5rZ1pJOl/S3khZKeqtz7v5A5QQA72jB6qAeAP/StmA5\nSRPOuZ2dc7t23ztc0izn3I6SLuz+DQCoGeeqLgHQPFm6CAd/4+wnaUb39QxJ+3spEYDUnJNe+cqq\nS1FftNwACCVLC9bvzOwaM3tv971pzrmp7uspSdO8lw7AWL//fdUlAAAMSjUGS9Luzrl7zGwbSbPM\n7Jb+D51zzsxoZAYAAFDKAMs5d0/3/3vN7FeSdpU0ZWbbOueWmNl2kpYmfXf69OmPv56YmNDExETR\nMgOAF3QRAu02OTmpycnJIGmbGzO60cyeJGlT59wDZvZkSTMlHS1pT0l/cc4db2aHS9rKOXf4wHfd\nuPQB5OectMkmDFLO6xWvkK68csP6++EPpY99TFq5srpyhXDJJdLERPK+smiR9Dd/w34EmJmcc15+\neqVpwZom6VfW+am3maRTnHMzzewaSWeY2bvVnabBR4EAoCxJLVhXXCE98ED5ZakSLXmAf2MDLOfc\nHZJ2Snj/PnVasQCglggsOmi5AvxjJnegxnoXRi6QABAXAiwAAADPCLAAAAA8I8AC0FqMwQIQCgEW\nAACAZwRYAFqLFqyOOtTDRRdVXQIgGwIsAK1Vh8ACnbtkX/e6qksBZEOABdQY0zS0z0MPSV//ut80\n2X8A/wiwGsZMWriw6lIACOXyy6WPfrTqUgAYhwCrge64o+oSAAilqd2aZu17RBGajQALQGslBSux\nd5c1NcCSpPvuq7oEgD8EWA0U+wUCQH5NDrCAJiHAAtBaScFK7AFMiPLFvs7czIE6IsACaowLTjGx\nBxZJ6lhmoI0IsIAGINBqDwIsoB4IsACg5QjQAf8IsACgRjbhrA3UAodqA/FrFEinjt1tdSwz0EYE\nWBG46y7p9tv9pffoo9LKlf7SAxCPNgZY/GhEHRFgRWC33aTnPc9feu97n7Tllv7SA5qqjsFKG6dp\n6CHQQp0QYEVg9Wq/6S1e7Dc9xIsLTjF1CSz61bHMQBsRYEXoP/+z8w9Ii0ALAOJCgBWhGTM6/wBg\nUIgWLAJ0wD8CLADoE3uwQRchUA8EWBGI/YQONFUdg5U6lrkozpGoIwIsAK1Vx4c9t3miUQIt1EmL\nD9V41PEkD6AanBuAeiDAihS/1JAG+0kxBCsd1APgHwEW0ABZA61LL5W+9KUwZUFYBENAPRBgAS30\n+c9Ln/xk1aVAHgRYQD2UGmD94hfSZz9bZo71kNT6wEm0XI89Jk1NVV0KxCD2blfmwQLqodQA6wtf\n6PxyRn0dfrj05jdXXQr/vvc9adttqy4FylbHHzJ1LHNRvQCQQBB1QhdhBOp0wjzlFOnss6suhX+0\nXrVTHe/gjb18oX3rW9Ls2VWXAhiPAAuosby/6GkJQF0dcoh09NFVlwIYL1WAZWabmtkcMzu3+/fW\nZjbLzBaY2Uwz2ypdOkWKCgB+1fGcFKLMdawHIHZpW7AOlTRfUu937+GSZjnndpR0YfdvtAAn4jhl\nbZFiO9ZXm2dyB+pk7KFqZs+WtK+kH0nqnZb3kzSj+3qGpP2DlK4l6K4B4hH78UhwDNRDmt9CX5d0\nmKR1fe9Nc871hgVPSZrmu2AAwok9iMBwbQyw2F9RR5uN+tDM3ihpqXNujplNJC3jnHNmNnT3nz59\n+uOvV62akJSYTKvV8U4moAk4zjrqEsDUpZyoj8nJSU1OTgZJe2SAJen/SNrPzPaVtIWkp5nZyZKm\nzGxb59wSM9tO0tJhCfQHWE28vT8UTiRAeHX8ccO5AfBnYmJCExMTj/99tMdbVEd2ETrnPuWc2945\n9xxJb5N0kXPuHZLOkXRgd7EDJaUKnWI/cWE8tmFc8l5s2Y4AEFbW+1F6p/PjJO1lZgskvbb7N4CK\n0KqRD4FmB/UA+Deui/BxzrlLJF3SfX2fpD1DFapteBYhykZABgBhMaMKCjOTbrih6lKgrR5+2O8P\nktiDzzb9+LrwQun88+PfJnVnJl13XdWlaB4CLHV2rgcfrDb/ulu0qOoSoK0eeaTqEiCUN75R2nff\n9X8TaIVz661Vl6B5Sg2wYg4kVq+uugTt8NOfSi9+cdWlKJdz0n33VV0KJOEuwnBpAm1HCxYyKXrx\nueACaf58P2UpapttpD/9KXw+v/mN9Nd/HT6fLGIPIspCPQAIhQALrbVsmXTTTeHzmZoav0xeeVse\naLEAxlu2TLr33qpLUQ5+bPiX+i5ChFP1xW7tWmkz9oRaq3ofqqs6XlRClDnWeqh6v37pSzvDR9oS\nZMEvWrAqtP/+0uLFVZdC2nxz6Zprysmr6hMmMA77KHruvrvTigXkQYBVoV//WrrssjgG2t5zT7n5\nATGIteUGHb3tU1XQS7CNIlp7F+G8edKznlV1KTb0kpdIv/td53VTD+z+feDqq6WFC6spx9VXV5Nv\nHZlJDzxQdSnQ09RzQxptXnfUT2tbsGbP7jT/xuSmmzoT68WsaJDcO0H++c/Srrt2ukmrsCfPIchk\n5cqqS1CemH4IAmVhv/evtQEWqvXsZ3f+56Auhl/0xdR5//O57dmPAP8IsCLAswjrb/XqTqsogHis\nXMlM/6hOawOsLAHMypXSFVeEKwvq78tflnbbrfP6scekI48sN39aIPLhh0xHU+thyy2l97wn//c5\nrlBEawe5Z/H5z0u77x4u/VD18uCD0rXXhkkbG3r00fWv779fOuaY6sqCYrioxqfINrntNn/lALJo\nbQvWOOvWSRdd1Hm9dm21ZcnrmGOkl73Mb5p1DZLLxAW6Poruz3PmbBhcl6HN+1eb1z00zu3+EWAN\nceml0ute13ld14M6xNi
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f096914f6d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "feat = net.blobs['fc6'].data[0]\n",
    "plt.subplot(2, 1, 1)\n",
    "plt.plot(feat.flat)\n",
    "plt.subplot(2, 1, 2)\n",
    "_ = plt.hist(feat.flat[feat.flat > 0], bins=100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* The final probability output, `prob`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7f09587dfb50>]"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA3gAAADICAYAAAC6TEOmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGzVJREFUeJzt3X2MXPV97/HPh12c4AdCHgiktiMTYh4chSb0yvheQjKN\nKHJpGkdBKkEhVCnlIvU6yb2qIje3Uu5at1Xa6qJLUxTq5LoNAW5olRjqSDgGGqZCCWCbEh53wRti\nsI1tzJO54Kddz/f+cc6wZ2dn58x6Z3bOWb9fkjXn6Tf7G/u3M/7M93fOcUQIAAAAAFB+J/W6AwAA\nAACAziDgAQAAAMAsQcADAAAAgFmCgAcAAAAAswQBDwAAAABmCQIeAAAAAMwSuQHP9krbQ7a3217T\nZP8q24/ZftT2I7Y/ndm3w/bj6b4tne48AAAAAGCMW90Hz3afpGckXSppt6Stkq6KiMHMMfMi4q10\n+aOS7oyID6frv5b0WxHxavdeAgAAAABAyq/gLZc0HBE7ImJE0h2SVmUPqIe71HxJLzc8h6fdSwAA\nAABArryAt1DSzsz6rnTbOLY/Z3tQ0iZJX83sCkn32d5m+7rpdhYAAAAAMLn+nP2Tz9/MHhRxl6S7\nbF8i6VZJ56a7Lo6IPbZPl3Sv7aGIeCDb1nZbPwMAAAAAZquI6MjMx7yAt1vS4sz6YiVVvMk69YDt\nftvvjYhXImJPun2/7TuVTPl8oEm7qfccmAEDAwMaGBjodTeACRibKCrGJoqM8Ymisjt3VlveFM1t\nkpbaXmJ7jqQrJW1s6MzZTntk+0JJiohXbM+1vSDdPk/SZZKe6FjPAQAAAADjtKzgRcSo7dWSNkvq\nk7Q+IgZtX5/uXyfpCknX2B6R9KakL6TNz5S0Ic1+/ZJuj4h7uvMyAAAAAAB5UzQVEZuUXDwlu21d\nZvlvJP1Nk3bPSfpYB/oI9EylUul1F4CmGJsoKsYmiozxiRNBy/vgzUgH7Oh1HwAAAACgV2x37CIr\neefgAQAAAABKgoAHAAAAALMEAQ8AAAAAZgkCHgAAAADMEgQ8AAAAAJglCHgAAAAAMEsQ8AAAAABg\nliDgAQAAAMAsQcADAAAAgFmCgAcAAAAAswQBDwAAAABmCQIegLYcPNjrHgAAACAPAQ9Arpdfln7z\nN3vdCwAAAOQh4AHIdfiw9Nprve4FAAAA8uQGPNsrbQ/Z3m57TZP9q2w/ZvtR24/Y/nS7bQGUQ60m\nHT3a614AAAAgjyNi8p12n6RnJF0qabekrZKuiojBzDHzIuKtdPmjku6MiA+30zZtE636AKD3nn9e\nOvfcpJIHAACAzrKtiHAnniuvgrdc0nBE7IiIEUl3SFqVPaAe7lLzJb3cblsA5VCv4PFdDAAAQLHl\nBbyFknZm1nel28ax/Tnbg5I2SfrqVNoCKL6I5M+xY73uCQAAAFrpz9nf1vf1EXGXpLtsXyLpVtvn\nTaUTAwMDby9XKhVVKpWpNAfQZbVa8nj0qNSf964BAACAlqrVqqrValeeO+8cvBWSBiJiZbr+DUm1\niPjrFm1+pWR65tJ22nIOHlB827dL55yTXEnztNN63RsAAIDZZSbPwdsmaantJbbnSLpS0saGzpxt\n2+nyhZIUEa+00xZAOWQreAAAACiulpOtImLU9mpJmyX1SVofEYO2r0/3r5N0haRrbI9IelPSF1q1\n7d5LAdAt9SI7AQ8AAKDYWk7RnJEOMEUTKLynn5Y+8hHpV7+SPvShXvcGAABgdpnJKZoAQAUPAACg\nJAh4AHJxDh4AAEA5EPAA5KKCBwAAUA4EPAC5qOABAACUAwEPQC4qeAAAAOVAwAOQqx7wjhzpbT8A\nAADQGgEPQC6maAIAAJQDAQ9ALqZoAgAAlAMBD0AuKngAAADlQMADkIsKHgAAQDkQ8ADkooIHAABQ\nDgQ8ALmo4AEAAJQDAQ9ALip4AAAA5UDAA5CLCh4AAEA5EPAA5KKCBwAAUA65Ac/2SttDtrfbXtNk\n/xdtP2b7cds/t31BZt+OdPujtrd0uvMAZgYVPAAAgHLob7XTdp+kmyRdKmm3pK22N0bEYOaw5yR9\nMiIO2F4p6buSVqT7QlIlIl7tfNcBzBQqeAAAAOWQV8FbLmk4InZExIikOyStyh4QEQ9GxIF09WFJ\nixqewx3pKYCeoYIHAABQDnkBb6GknZn1Xem2yVwr6e7Meki6z/Y229cdXxcB9BoVPAAAgHJoOUVT\nSUBri+3flvRHki7ObL44IvbYPl3SvbaHIuKBxrYDAwNvL1cqFVUqlXZ/LIAZQAUPAACgc6rVqqrV\nalee2xGTZzjbKyQNRMTKdP0bkmoR8dcNx10gaYOklRExPMlz/Q9Jb0bEDQ3bo1UfAPTepk3S5ZdL\n11wj3XJLr3sDAAAwu9hWRHTk1La8KZrbJC21vcT2HElXStrY0JkPKgl3V2fDne25theky/MkXSbp\niU50GsDMooIHAABQDi2naEbEqO3VkjZL6pO0PiIGbV+f7l8n6ZuS3i3pZtuSNBIRyyWdKWlDuq1f\n0u0RcU/XXgmAriHgAQAAlEPeOXiKiE2SNjVsW5dZ/mNJf9yk3XOSPtaBPgLoMS6yAgAAUA65NzoH\ngAhpzhzpyJFe9wQAAACtEPAA5KrVpJNPlo4d63VPAAAA0AoBD0CuCKm/f2yqJgAAAIqJgAcgV60m\n9fUR8AAAAIqOgAcgVwQBDwAAoAwIeABy1WpM0QQAACgDAh6AXFTwAAAAyoGAByAXFTwAAIByIOAB\nyEUFDwAAoBwIeAByUcEDAAAoBwIegFxU8AAAAMqBgAcgFxU8AACAciDgAchFBQ8AAKAcCHgAckVQ\nwQMAACgDAh6AXLUaFTwAAIAyyA14tlfaHrK93faaJvu/aPsx24/b/rntC9ptC6AcqOABAACUQ8uA\nZ7tP0k2SVkpaJukq2+c3HPacpE9GxAWS/qek706hLYASoIIHAABQDnkVvOWShiNiR0SMSLpD0qrs\nARHxYEQcSFcflrSo3bYAyoEKHgAAQDnkBbyFknZm1nel2yZzraS7j7MtgIKiggcAAFAO/Tn7o90n\nsv3bkv5I0sVTbTswMPD2cqVSUaVSabcpgBlABQ8AAKBzqtWqqtVqV547L+DtlrQ4s75YSSVunPTC\nKt+TtDIiXptKW2l8wANQPFTwAAAAOqexqLV27dqOPXfeFM1tkpbaXmJ7jqQrJW3MHmD7g5I2SLo6\nIoan0hZAOXCjcwAAgHJoWcGLiFHbqyVtltQnaX1EDNq+Pt2/TtI3Jb1b0s22JWkkIpZP1raLrwVA\nl9RqTNEEAAAoA0e0fapcdzpgR6/7AKC1G2+UHnpIuv9+ad++XvcGAABgdrGtiHAnniv3RucAQAUP\nAACgHAh4AHJxDh4AAEA5EPAA5KKCBwAAUA4EPAC5qOABAACUAwEPQC4qeAAAAOVAwAOQiwoeAABA\nORDwAOSKoIIHAABQBgQ8ALlqNSp4AAAAZUDAA5CLCh4AAEA5EPAA5KKCBwAAUA4EPAC5qOABAACU\nAwEPQK5aTTopfbeI6G1fAAAAMDkCHoBcEUnAO+kkqngAAABFRsADkKtWk2wCHgAAQNER8ADkiiDg\nAQAAlEFuwLO90vaQ7e221zTZf57tB20ftv2nDft22H7c9qO2t3Sy4wBmTv0cPAIeAABAsfW32mm7\nT9JNki6VtFvSVtsbI2Iwc9grkr4i6XNNniIkVSLi1Q71F0APUMEDAAAoh7wK3nJJwxGxIyJGJN0h\naVX2gIjYHxHbJI1M8hyefjcB9BIVPAAAgHLIC3gLJe3MrO9Kt7UrJN1ne5vt66baOQDFQAUPAACg\nHFpO0VQS0Kbj4ojYY/t0SffaHoqIBxoPGhgYeHu5UqmoUqlM88cC6CQqeAAAAJ1TrVZVrVa78tx5\nAW+3pMWZ9cVKqnhtiYg96eN+23cqmfLZMuABKB4qeAAAAJ3TWNRau3Ztx547b4rmNklLbS+xPUfS\nlZI2TnLsuHPtbM+1vSB
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f0969270290>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "feat = net.blobs['prob'].data[0]\n",
    "plt.figure(figsize=(15, 3))\n",
    "plt.plot(feat.flat)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note the cluster of strong predictions; the labels are sorted semantically. The top peaks correspond to the top predicted labels, as shown above."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6. Try your own image\n",
    "\n",
    "Now we'll grab an image from the web and classify it using the steps above.\n",
    "\n",
    "* Try setting `my_image_url` to any JPEG image URL."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# download an image\n",
    "my_image_url = \"...\"  # paste your URL here\n",
    "# for example:\n",
    "# my_image_url = \"https://upload.wikimedia.org/wikipedia/commons/b/be/Orang_Utan%2C_Semenggok_Forest_Reserve%2C_Sarawak%2C_Borneo%2C_Malaysia.JPG\"\n",
    "!wget -O image.jpg $my_image_url\n",
    "\n",
    "# transform it and copy it into the net\n",
    "image = caffe.io.load_image('image.jpg')\n",
    "net.blobs['data'].data[...] = transformer.preprocess('data', image)\n",
    "\n",
    "# perform classification\n",
    "net.forward()\n",
    "\n",
    "# obtain the output probabilities\n",
    "output_prob = net.blobs['prob'].data[0]\n",
    "\n",
    "# sort top five predictions from softmax output\n",
    "top_inds = output_prob.argsort()[::-1][:5]\n",
    "\n",
    "plt.imshow(image)\n",
    "\n",
    "print 'probabilities and labels:'\n",
    "zip(output_prob[top_inds], labels[top_inds])"
   ]
  }
 ],
 "metadata": {
  "description": "Instant recognition with a pre-trained model and a tour of the net interface for visualizing features and parameters layer-by-layer.",
  "example_name": "Image Classification and Filter Visualization",
  "include_in_docs": true,
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  },
  "priority": 1
 },
 "nbformat": 4,
 "nbformat_minor": 0
}