mydata.describe()
3. Split the data into train, validation, test datasets: We use the train_test_split from ScikitLearn to split the data into train and test sets. Then, we split the training set into training and validation datasets. To achieve a split of 20%, 20%, 60% test, validation and training sets, we first split the data into 20% test and 80% training, and then split the training data into 25% validation and 75% training sets.
X_train, X_test, y_train, y_test = train_test_split(X,y, test_size=0.2, random_state=42)
X_train, X_valid, y_train, y_valid = train_test_split(X_train, y_train, test_size = 0.25,
random_state=2018)
4. Define model: Deep learning models in Keras are defined as a sequence of layers. As we learnt in previous chapters, any deep learning model is comprised of an input layer, one or more hidden layers, and an output layer. In Keras, we add these layers in sequence, hence it's called a Sequential model. For the first layers, we need to make sure that we have the correct number of inputs. This is normally the number of attributes in data and can be found as the first dimension of the dataset:
input_dim = X_train.shape[0]
Since our data has 18 attributes, the input dimension is 18. We do not need to define the input dimension for the subsequent layers, since the network can automatically calculate that. The difficult question is, how do we decided number of hidden layers, and their parameters such as number of nodes in each layer, and activation function. Deep learning experts and practitioners suggest the process of trial and error experimentation [ref]. The network should be large enough to capture the structure of the problem, and small enough not to overfit the data. In Keras, fully connected layers are defined using Dense class. You can specify the number of neurons in the layers using for the first argument, and specify the initialization argument and activation function as the subsequent arguments using init and activation. We will discuss the possible choices for initialization argument and activation function in the following sections. For now, let's start with Keras default option for initialization, i.e. 'uniform' which chooses the initial weights from a uniform distribution between 0 and 0.05. We use 'relu' activation function for both input and hidden layers in this model.
kmodel = Sequential()
kmodel.add(Dense(100, activation="relu", input_dim = 18))
kmodel.add(Dense(50, activation="relu"))
kmodel.add(Dense(1))
print(kmodel.summary())