Complete model builder guide

MinPy’s model builder simplifies the declaration of networks, in styles similar to Keras. Networks expressed with model builder is compatible with examples listed elsewehre in other parts of the tutorial.

Get started

MinPy’s model builder consists of classes describing various neural network architectures.

  • Sequence class enables users to create feedforward networks.
  • Other classes representing layers of neural networks, such as Affine and Convolution.

The following code snippet demonstrates how a few lines of specification with model builder removes the need of defining a complete network from scratch. The full code is here

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
'''
  Minpy model builder example.

  For details about how to train a model with solver, please refer to:
    http://minpy.readthedocs.io/en/latest/tutorial/complete.html

  More models are available in minpy.nn.model_gallery.
'''

#... other import modules
import minpy.nn.model_builder as builder

class TwoLayerNet(ModelBase):
    def __init__(self):
        super(TwoLayerNet, self).__init__()
        # Define model parameters.
        self.add_param(name='w1', shape=(flattened_input_size, hidden_size)) \
            .add_param(name='b1', shape=(hidden_size,)) \
            .add_param(name='w2', shape=(hidden_size, num_classes)) \
            .add_param(name='b2', shape=(num_classes,))

    def forward(self, X, mode):
        # Flatten the input data to matrix.
        X = np.reshape(X, (batch_size, 3 * 32 * 32))
        # First affine layer (fully-connected layer).
        y1 = layers.affine(X, self.params['w1'], self.params['b1'])
        # ReLU activation.
        y2 = layers.relu(y1)
        # Second affine layer.
        y3 = layers.affine(y2, self.params['w2'], self.params['b2'])
        return y3

    def loss(self, predict, y):
        # Compute softmax loss between the output and the label.
        return layers.softmax_loss(predict, y)

def main(args):
    # Define a 2-layer perceptron
    MLP = builder.Sequential(
        builder.Affine(512),
        builder.ReLU(),
        builder.Affine(10)
    )

    # Cast the definition to a model compatible with minpy solver
    model = builder.Model(MLP, 'softmax', (3 * 32 * 32,))

    # ....
    # the above is equivalent to use the TwoLayerNet, as below (uncomment )
    # model = TwoLayerNet()

    solver = Solver(model,
    # ...

Arbitrarily complex networks could be constructed by combining these classes in a nested way. Please refer to the model gallery to discover how to easily declare complex networks such as ResNet with model builder.

Customize model builder layers

MinPy’s model builder is designed to be extended easily. To create customized layers, one only needs to inherit classes from minpy.nn.model_builder.Module class and implement several inherited functions. These functions are forward, output_shape, parameter_shape and parameter_settings.

  • forward receives input data and a dictionary containing the parameters of the network, and generates output data. It can be implemented by Numpy syntax, layers provided in minpy.nn.layers, MXNet symbols or their combination as described in Complete solver and optimizer guide.
  • output_shape returns the layer’s output shape given the input shape. Please be aware that the shapes should be tuples specifying shape of one training sample, i.e. the shape of CIFAR-10 data is either (3072,) or (3, 32, 32).
  • parameter_shape returns a dictionary that includes the shapes of all parameters of the layer. One could pass more information to MinPy’s solver in parameter_settings.

It is optional to implement parameter_settings. The model builder script should be self-explanatory.