【mnist】How to create a learning and prediction model by softmax regression [numpy]

【mnist】How to create a prediction model with softmax regression [numpy]

【mnist】How to create a prediction model with softmax regression [numpy]

For the dataset mnist, we summarize how to learn using numpy as the main to create a predictive model.

You only need to run the program in the order shown below.

The execution environment is Google Colaboratory.

Import library

Import the library you want to use in the following code (for example, dataset or numpy):

from sklearn.utils import shuffle
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split
from keras.datasets import mnist

import numpy as np

np.random.seed(34)

Implement a function to learn in Softmax regression

First, implement np_log function.

# Prevent log contents from becoming 0
def np_log(x):
    return np.log(np.clip(a=x, a_min=1e-10, a_max=1e+10))

Next, we implement a function train_mnist for learning.

def train_mnist(x, y, eps=1.0):
    """
    :p aram x: np.ndarray, input data, shape=(batch_size, number of dimensions of input)
    :p aram y: np.ndarray, teacher label, shape=(batch_size, number of dimensions in output)
    :p aram eps: float, learning rate
    """
    global W_mnist, b_mnist
    
batch_size = x.shape[0]
    
# Forecast
    y_hat = softmax(np.matmul(x, W_mnist) + b_mnist) # shape: (batch_size, number of dimensions in output)
    
# Evaluating objective functions
    cost = (- y * np_log(y_hat)).sum(axis=1).mean()
    delta = y_hat - y # shape: (batch_size, number of dimensions in output)
    
# Update parameters
    dW = np.matmul(x.T, delta) / batch_size # shape: (number of dimensions of input, number of dimensions in output)
    db = np.matmul(np.ones=(batch_size,)), delta) / batch_size # shape: (number of dimensions in output,)
    W_mnist -= eps * dW
    b_mnist -= eps * db

return cost

def valid_mnist(x, y):
    y_hat = softmax(np.matmul(x, W_mnist) + b_mnist)
    cost = (- y * np_log(y_hat)).sum(axis=1).mean()
    
return cost, y_hat

See below for more information on Softmax functions.

Learn predictive models

Learn the model in the following code.

The number of epochs is 100.

for epoch in range(100):
    x_train_mnist, y_train_mnist = shuffle(x_train_mnist, y_train_mnist)
    cost = train_mnist(x_train_mnist, y_train_mnist)
    cost, y_pred = valid_mnist(x_valid_mnist, y_valid_mnist)
    if epoch % 10 == 9 or epoch == 0:
        print('EPOCH: {}, Valid Cost: {:.3f}, Valid Accuracy: {:.3f}'.format(
            epoch + 1,
            cost,
            accuracy_score(y_valid_mnist.argmax(axis=1), y_pred.argmax(axis=1))
        ))