# Logistic regression tutorial¶

This part of tutorial is derived from its step-by-step notebook version multinomial logistic regression example, the emphasis is to showcase the basic capacity of MinPy.

We will work on a classification problem of a synthetic data set. Each point is a high-dimentional data in one of the five clusters. We will build a one-layer multinomial logistic regression model. The goal is to learn a weight matrix weight, such that for a data point x the probability that it is assigned to its class (cluster) is the largest.

The data is generated with the following code.

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 import numpy as np """ Generates several clusters of Gaussian points """ def gaussian_cluster_generator(num_samples=10000, num_features=500, num_classes=5): mu = np.random.rand(num_classes, num_features) sigma = np.ones((num_classes, num_features)) * 0.1 num_cls_samples = num_samples / num_classes x = np.zeros((num_samples, num_features)) y = np.zeros((num_samples, num_classes)) for i in range(num_classes): cls_samples = np.random.normal(mu[i,:], sigma[i,:], (num_cls_samples, num_features)) x[i*num_cls_samples:(i+1)*num_cls_samples] = cls_samples y[i*num_cls_samples:(i+1)*num_cls_samples,i] = 1 return x, y 

The visualization of the data:

The following is the numpy version. The function predict outputs the probability, the train function iterates over the data, computes the loss, the gradients, and updates the parameters w with a fixed learning rate.

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 import numpy as np import numpy.random as random from examples.utils.data_utils import gaussian_cluster_generator as make_data # Predict the class using multinomial logistic regression (softmax regression). def predict(w, x): a = np.exp(np.dot(x, w)) a_sum = np.sum(a, axis=1, keepdims=True) prob = a / a_sum return prob # Using gradient descent to fit the correct classes. def train(w, x, loops): for i in range(loops): prob = predict(w, x) loss = -np.sum(label * np.log(prob)) / num_samples if i % 10 == 0: print('Iter {}, training loss {}'.format(i, loss)) # gradient descent dy = prob - label dw = np.dot(data.T, dy) / num_samples # update parameters; fixed learning rate of 0.1 w -= 0.1 * dw # Initialize training data. num_samples = 10000 num_features = 500 num_classes = 5 data, label = make_data(num_samples, num_features, num_classes) # Initialize training weight and train weight = random.randn(num_features, num_classes) train(weight, data, 100) 

The minpy version is very similar, except a few lines that are highlighted:

• Among some new imported libraries, minpy.numpy replaces numpy. This lightweight library is fully numpy compatible, but it allows us to add small instrumentations in the style of autograd
• Defines loss explicitly with the function train_loss
• MinPy then derives a function to compute gradients automatically (line 24)
  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 import minpy.numpy as np import minpy.numpy.random as random from minpy.core import grad_and_loss from examples.utils.data_utils import gaussian_cluster_generator as make_data from minpy.context import set_context, gpu # Please uncomment following if you have GPU-enabled MXNet installed. # This single line of code will run MXNet operations on GPU 0. # set_context(gpu(0)) # set the global context as gpu(0) # Predict the class using multinomial logistic regression (softmax regression). def predict(w, x): a = np.exp(np.dot(x, w)) a_sum = np.sum(a, axis=1, keepdims=True) prob = a / a_sum return prob def train_loss(w, x): prob = predict(w, x) loss = -np.sum(label * np.log(prob)) / num_samples return loss """Use Minpy's auto-grad to derive a gradient function off loss""" grad_function = grad_and_loss(train_loss) # Using gradient descent to fit the correct classes. def train(w, x, loops): for i in range(loops): dw, loss = grad_function(w, x) if i % 10 == 0: print('Iter {}, training loss {}'.format(i, loss)) # gradient descent w -= 0.1 * dw # Initialize training data. num_samples = 10000 num_features = 500 num_classes = 5 data, label = make_data(num_samples, num_features, num_classes) # Initialize training weight and train weight = random.randn(num_features, num_classes) train(weight, data, 100) 

Now, if you uncomment line 9 to set MXNet context on GPU 0, this one line change (set_context(gpu(0))) will enable the same code to run on GPU!

For more functionality of MinPy/MXNet, we invite you to read later sections of this tutorial.