Using an RNN to generate a sequence with a static end point using TF and Keras
I’m currently learning about how to use neural networks while working on a project of mine. In the project I’m attempting to have a neural network create a path from a starting point (0,0) to an end point (xn, yn).
I have data composed of a list of coordinates starting from (0,0) all the way to the end point: [(0,1), (0,2), (-1,2)…, (xn, yn)], where each subsequent coordinate is one step further along the path towards the goal.
I’m using a simple LSTM neural network built using the Keras sequential model as follows, where the goal is to be able to predict a path given only the end coordinates.
import numpy as np import tensorflow as tf from tensorflow.keras.wrappers.scikit_learn import KerasClassifier from tensorflow.keras.models import Sequential from tensorflow.keras.layers import Dense, LSTM, Activation from tensorflow.keras import optimizers #Sequence end point, each tuple is the end point for one output sequence input_data = [(x1n, y1n), (x2n, y2n)] #Sequences from (0,0) to the end point (xn, yn) output_data = [[(0,1), (0,2), (-1,2)..., (x1n, y1n)], [(0,1), (0,2), (-1,2)..., (x2n, y2n)]] input = np.array(input_data).reshape((input_data.shape[0], input_data.shape[1], 1)) model = Sequential() model.add(LSTM(50, input_shape = (2,1), return_sequences = True)) model.add(LSTM(50, return_sequences = False)) model.add(Dense(input.shape[0])) model.add(Activation('softmax')) adam = optimizers.Adam(lr = 0.001) model.compile(loss = 'categorical_crossentropy', optimizer = adam, metrics = ['accuracy']) model.fit(input, output_data) A lot of the code used here is from this incredibly useful beginners guide to neural networks.
My questions are as follows:
-
Is there a better layout (e.g. using another kind of layer or adding different layers) for a neural network that generates sequence data? I was really intrigued by the way it was done in Alex Graves’ paper to generate sequences of handwritten text. He did use quite a more complicated version than this, but as I’m still just a beginner I’ll stick to the easy stuff.
-
Is it possible to force the neural network to create a sequence that always ends at a set point? In my case I’m interested in the path between the start and end point, so how can I program the network to always use the end point as the end of the generated sequence?
-
Is there any preprocessing I could do to improve the results? The only thing I’ve done so far is to reshape the input into three dimensions so it is accepted by the LSTM layer. I know that people usually try to standardize the input, but I’m not sure how to go about it with my data. Does this improve the neural network and if so, why?
