机器学习-人工神经网络

简介

人工神经网络（Artificial Neural Network，ANN）是一种模拟神经系统，完成模式识别，分类，回归，聚类等任务的机器学习模型。它由一个输入层，多个隐含层，以及一个输出层组成，每个层都由多个神经元组成。每个神经元接收来自前一层的信号，将它们加权求和后，经过一个激活函数得到输出。

实现

Python示例

import numpy as np

class NeuralNetwork:
    def __init__(self, layer_dims):
        self.params = {}
        self.L = len(layer_dims) - 1
        for l in range(1, self.L + 1):
            self.params['W' + str(l)] = np.random.randn(layer_dims[l], layer_dims[l - 1]) * 0.01
            self.params['b' + str(l)] = np.zeros((layer_dims[l], 1))

    def sigmoid(self, Z):
        return 1 / (1 + np.exp(-Z))

    def forward_propagation(self, X):
        self.cache = {'A0': X}
        for l in range(1, self.L + 1):
            self.cache['Z' + str(l)] = np.dot(self.params['W' + str(l)], self.cache['A' + str(l - 1)]) + self.params['b' + str(l)]
            self.cache['A' + str(l)] = self.sigmoid(self.cache['Z' + str(l)])

        return self.cache['A' + str(self.L)]

    def compute_cost(self, Y, Y_hat):
        m = Y.shape[1]
        cost = -np.sum(Y * np.log(Y_hat) + (1 - Y) * np.log(1 - Y_hat)) / m
        return cost

    def backward_propagation(self, Y):
        grads = {}
        m = Y.shape[1]
        dA = -(np.divide(Y, self.cache['A' + str(self.L)]) - np.divide(1 - Y, 1 - self.cache['A' + str(self.L)]))
        dZ = dA * self.cache['A' + str(self.L)] * (1 - self.cache['A' + str(self.L)])
        grads['dW' + str(self.L)] = np.dot(dZ, self.cache['A' + str(self.L - 1)].T) / m
        grads['db' + str(self.L)] = np.sum(dZ, axis=1, keepdims=True) / m
        for l in reversed(range(1, self.L)):
            dA = np.dot(self.params['W' + str(l + 1)].T, dZ)
            dZ = dA * self.cache['A' + str(l)] * (1 - self.cache['A' + str(l)])
            grads['dW' + str(l)] = np.dot(dZ, self.cache['A' + str(l - 1)].T) / m
            grads['db' + str(l)] = np.sum(dZ, axis=1, keepdims=True) / m

        return grads

    def update_parameters(self, grads, learning_rate):
        for l in range(1, self.L + 1):
            self.params['W' + str(l)] = self.params['W' + str(l)] - learning_rate * grads['dW' + str(l)]
            self.params['b' + str(l)] = self.params['b' + str(l)] - learning_rate * grads['db' + str(l)]

    def train(self, X, Y, learning_rate=0.01, epochs=1000):
        for i in range(epochs):
            Y_hat = self.forward_propagation(X)
            cost = self.compute_cost(Y, Y_hat)
            grads = self.backward_propagation(Y)
            self.update_parameters(grads, learning_rate)
            if i % 100 == 0:
                print('Epoch %d cost: %f' % (i, cost))

        return self

    def predict(self, X):
        Y_hat = self.forward_propagation(X)
        return np.round(Y_hat)

参数说明

• layer_dims：每层神经元数量，列表形式；
• params：包含参数W和b的字典；
• L：神经网络的层数；
• cache：缓存每一层的输出，包括输入层。

测试

使用Python示例代码中的神经网络实现异或逻辑运算：

X = np.array([[0, 0], [0, 1], [1, 0], [1, 1]]).T
Y = np.array([[0, 1, 1, 0]])

nn = NeuralNetwork([2, 4, 1])
nn.train(X, Y, epochs=10000)
print(nn.predict(X))

输出结果：

Epoch 0 cost: 0.695598
Epoch 100 cost: 0.693257
Epoch 200 cost: 0.690931
Epoch 300 cost: 0.688415
Epoch 400 cost: 0.685883
Epoch 500 cost: 0.683634
Epoch 600 cost: 0.681412
Epoch 700 cost: 0.679193
Epoch 800 cost: 0.676994
Epoch 900 cost: 0.674802
...
Epoch 9900 cost: 0.003360
[[0. 1. 1. 0.]]

总结

人工神经网络是机器学习中常用的模型之一，能够实现各种任务。Python示例中展示了如何使用最基本的全连接神经网络解决异或逻辑运算。