Keras.layers.SimpleRNN

The keras.layers.SimpleRNN layer is a recurrent neural network layer in Keras that implements a simple RNN architecture.

Syntax

tf.keras.layers.SimpleRNN(units, activation='tanh', use_bias=True, 
                          kernel_initializer='glorot_uniform', 
                          recurrent_initializer='orthogonal', 
                          bias_initializer='zeros', 
                          kernel_regularizer=None, 
                          recurrent_regularizer=None, 
                          bias_regularizer=None, 
                          activity_regularizer=None, 
                          kernel_constraint=None, 
                          recurrent_constraint=None, 
                          bias_constraint=None, 
                          dropout=0.0, recurrent_dropout=0.0, 
                          return_sequences=False, return_state=False, 
                          go_backwards=False, stateful=False, unroll=False, 
                          **kwargs)

Parameters

• units: Integer, the dimensionality of the output space (i.e. the number of output nodes).
• activation: Activation function to use (defaults to 'tanh').
• use_bias: Boolean, whether the layer uses a bias vector (defaults to True).
• kernel_initializer: Initializer for the kernel weights matrix (defaults to 'glorot_uniform').
• recurrent_initializer: Initializer for the recurrent weights matrix (defaults to 'orthogonal').
• bias_initializer: Initializer for the bias vector (defaults to 'zeros').
• kernel_regularizer: Regularizer function applied to the kernel weights matrix (defaults to None).
• recurrent_regularizer: Regularizer function applied to the recurrent weights matrix (defaults to None).
• bias_regularizer: Regularizer function applied to the bias vector (defaults to None).
• activity_regularizer: Regularizer function applied to the output (defaults to None).
• kernel_constraint: Constraint function applied to the kernel weights matrix (defaults to None).
• recurrent_constraint: Constraint function applied to the recurrent weights matrix (defaults to None).
• bias_constraint: Constraint function applied to the bias vector (defaults to None).
• dropout: Float between 0 and 1, fraction of the input units to drop for the linear transformation of the inputs (defaults to 0.0).
• recurrent_dropout: Float between 0 and 1, fraction of the input units to drop for the linear transformation of the recurrent state (defaults to 0.0).
• return_sequences: Boolean, whether to return the full sequence or only the last output (defaults to False).
• return_state: Boolean, whether to return the last state in addition to the output (defaults to False).
• go_backwards: Boolean, whether to process the input sequence in reverse (defaults to False).
• stateful: Boolean, whether to use a stateful RNN (defaults to False).
• unroll: Boolean, whether to unroll the input sequence, which can speed up computation but may increase memory usage (defaults to False).
• **kwargs: Additional keyword arguments that may be passed to the constructor of the parent tf.keras.layers.Layer class.

Usage

import tensorflow as tf

model = tf.keras.Sequential()
model.add(tf.keras.layers.Embedding(input_dim=vocab_size, 
                                     output_dim=embedding_dim, 
                                     input_length=max_length))
model.add(tf.keras.layers.SimpleRNN(units=64))
model.add(tf.keras.layers.Dense(units=1, activation='sigmoid'))

model.summary()

Return Value

This layer returns a tensor with shape (batch_size, units) if return_sequences=False or (batch_size, timesteps, units) if return_sequences=True and return_state=False, where timesteps is the length of the input sequence. If return_state=True, then an additional tensor representing the last state of the RNN is returned, with shape (batch_size, units).

Example

import tensorflow as tf

# define model architecture
model = tf.keras.Sequential()
model.add(tf.keras.layers.Embedding(input_dim=vocab_size, 
                                     output_dim=embedding_dim, 
                                     input_length=max_length))
model.add(tf.keras.layers.SimpleRNN(units=64))
model.add(tf.keras.layers.Dense(units=1, activation='sigmoid'))

# compile model
model.compile(optimizer='adam', loss='binary_crossentropy', 
              metrics=['accuracy'])

# train model
model.fit(X_train, y_train, epochs=10, batch_size=64, validation_split=0.2)

Conclusion

In summary, the keras.layers.SimpleRNN layer is a simple implementation of a recurrent neural network architecture in Keras that can be used to solve a wide variety of sequence-based machine learning problems. The layer is highly customizable and can be easily integrated into more complex model architectures.