model1_EDLSTM.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 653,
     "status": "ok",
     "timestamp": 1607938388303,
     "user": {
      "displayName": "Zr Xiang",
      "photoUrl": "",
      "userId": "10347344196072452629"
     },
     "user_tz": 360
    },
    "id": "qDKMog461J_l",
    "outputId": "0dedc7d4-3213-4a91-be23-9de5e19e9431"
   },
   "outputs": [],
   "source": [
    "from google.colab import drive\n",
    "drive.mount('/content/drive')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 1114,
     "status": "ok",
     "timestamp": 1607938388788,
     "user": {
      "displayName": "Zr Xiang",
      "photoUrl": "",
      "userId": "10347344196072452629"
     },
     "user_tz": 360
    },
    "id": "u631zTBG1MFo",
    "outputId": "e6d2380e-4405-4a93-824b-a1737c0bbba6"
   },
   "outputs": [],
   "source": [
    "%cd '/content/drive/My Drive/ModelSharing'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "executionInfo": {
     "elapsed": 1099,
     "status": "ok",
     "timestamp": 1607938388789,
     "user": {
      "displayName": "Zr Xiang",
      "photoUrl": "",
      "userId": "10347344196072452629"
     },
     "user_tz": 360
    },
    "id": "gPPqYh3U1VL4"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "from tensorflow.keras.models import *\n",
    "from tensorflow.keras.layers import *\n",
    "from tensorflow.keras.callbacks import ModelCheckpoint, ReduceLROnPlateau, EarlyStopping\n",
    "from tensorflow.keras.optimizers import *\n",
    "import tensorflow.keras.backend as K\n",
    "\n",
    "from sklearn.preprocessing import MinMaxScaler\n",
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The function series_to_supervised takes several arguments: data (the input dataset), n_in (the number of lag observations as input), n_out (the number of future observations to predict), and dropnan (a flag indicating whether to drop rows with missing values or not).\n",
    "\n",
    "- It determines the number of variables in the dataset (n_vars) based on whether the data is a list or a DataFrame.\n",
    "\n",
    "- It creates a DataFrame (df) from the input dataset.\n",
    "\n",
    "- Two empty lists, cols and names, are initialized. These lists will store the columns and column names for the transformed dataset.\n",
    "\n",
    "- The next block of code handles the input sequence, which consists of lag observations. It iterates over the range of n_in to 0 (exclusive) in reverse order. For each iteration, it appends a shifted version of the DataFrame by i time steps to the cols list. It also adds the corresponding column names to the names list.\n",
    "\n",
    "The following block handles the forecast sequence, which represents future observations to predict. It iterates over the range of 0 to n_out. For each iteration, it appends a shifted version of the DataFrame in the negative direction (to predict future values). If it's the first iteration (i.e., i == 0), it adds column names indicating the current time step. Otherwise, it adds column names indicating future time steps.\n",
    "\n",
    "- The cols list is concatenated along the columns axis (axis=1) to create a new DataFrame called agg.\n",
    "\n",
    "- The column names of agg are assigned from the names list.\n",
    "\n",
    "- If the dropnan flag is set to True, the function drops rows with missing values from agg.\n",
    "\n",
    "- Finally, the transformed dataset (agg) is converted to a DataFrame of type 'float32' and returned.\n",
    "\n",
    "In summary, this function takes a dataset and transforms it into a supervised learning format by creating lag observations as input and future observations as output. It can be useful for tasks such as time series forecasting, where we want to predict future values based on past observations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "executionInfo": {
     "elapsed": 1089,
     "status": "ok",
     "timestamp": 1607938388790,
     "user": {
      "displayName": "Zr Xiang",
      "photoUrl": "",
      "userId": "10347344196072452629"
     },
     "user_tz": 360
    },
    "id": "GGyMOPlH1ihI"
   },
   "outputs": [],
   "source": [
    "def series_to_supervised(data, n_in=1, n_out=1, dropnan=False):\n",
    "    n_vars = 1 if type(data) is list else data.shape[1]\n",
    "    df = pd.DataFrame(data)\n",
    "    cols, names = list(), list()\n",
    "  # input sequence (t-n, ... t-1)\n",
    "    for i in range(n_in, 0, -1):\n",
    "        cols.append(df.shift(i))\n",
    "        names += [('var%d(t-%d)' % (j+1, i)) for j in range(n_vars)]\n",
    "  # forecast sequence (t, t+1, ... t+n)\n",
    "    for i in range(0, n_out):\n",
    "        cols.append(df.shift(-i))\n",
    "    if i == 0:\n",
    "        names += [('var%d(t)' % (j+1)) for j in range(n_vars)]\n",
    "    else:\n",
    "        names += [('var%d(t+%d)' % (j+1, i)) for j in range(n_vars)]\n",
    "  # put it all together\n",
    "    agg = pd.concat(cols, axis=1)\n",
    "    agg.columns = names\n",
    "  # drop rows with NaN values\n",
    "    if dropnan:\n",
    "        agg.dropna(inplace=True)\n",
    "    return pd.DataFrame(agg.astype('float32'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The function train_valid_test_split takes four arguments: data (the input dataset), hours_of_history (the number of hours of historical data to include as input), hours_to_predict (the number of hours to predict in the future), and parameters_included (a list of parameters/features to include in the dataset).\n",
    "\n",
    "- The first line of code selects the first 52,608 rows of the dataset as data_train_valid, representing the first 6 years of data for training and validation.\n",
    "\n",
    "- The second line of code selects the remaining rows of the dataset as data_test, representing the last 1 year of data for testing.\n",
    "\n",
    "- The next two lines of code remove any rows with missing values from data_train_valid and data_test by calling the dropna method.\n",
    "\n",
    "- The train_test_split function is used to split data_train_valid into two sets: data_valid and data_train. The test_size parameter is set to 0.4, indicating that 40% of the data will be used for validation, while the remaining 60% will be used for training. The shuffle parameter is set to False, indicating that the data will not be randomly shuffled before splitting.\n",
    "\n",
    "Finally, the function returns the values of data_train, data_valid, and data_test as NumPy arrays."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "executionInfo": {
     "elapsed": 1082,
     "status": "ok",
     "timestamp": 1607938388791,
     "user": {
      "displayName": "Zr Xiang",
      "photoUrl": "",
      "userId": "10347344196072452629"
     },
     "user_tz": 360
    },
    "id": "c46oMGAs_Ile"
   },
   "outputs": [],
   "source": [
    "def train_valid_test_split(data, hours_of_history, hours_to_predict, parameters_included):\n",
    "    data_train_valid = data.iloc[:52608,:] # the first 6 years for training/validation\n",
    "    data_test = data.iloc[52608:,:] # the last 1 years for test evaluation\n",
    "    data_train_valid.dropna(inplace=True)\n",
    "    data_test.dropna(inplace=True)\n",
    "\n",
    "    data_valid, data_train = train_test_split(data_train_valid, test_size=0.4, shuffle= False) # the last 60% data in the first 6 years used for training and the first 40% used for validation.\n",
    "\n",
    "    return data_train.values, data_valid.values, data_test.values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This code defines a function called prepare_data that prepares the data for a specific station by performing several preprocessing steps. Let's go through the code:\n",
    "\n",
    "1. The function prepare_data takes four arguments: station_id (the ID of the station), hours_of_history (the number of hours of historical data to include as input), hours_to_predict (the number of hours to predict in the future), and parameters_included (a list of parameters/features to include in the dataset).\n",
    "\n",
    "2. The next line reads the data from a CSV file specific to the station using the pd.read_csv function. It removes the first column (assuming it contains index values) using .iloc[:,1:].\n",
    "\n",
    "3. The data is then scaled using min-max scaling (MinMaxScaler) with the fit method applied to the training and validation data (data.iloc[:52608,:]). This ensures that the scaling is performed without including the test dataset.\n",
    "\n",
    "4. The maximum and minimum values of the third column (assuming it represents discharge) are calculated to later use for reference (q_max and q_min).\n",
    "\n",
    "5. The series_to_supervised function is called to convert the scaled data into a supervised learning format, considering the specified hours_of_history and hours_to_predict.\n",
    "\n",
    "6. The train_valid_test_split function is called to split the data sequence into training, validation, and test sets based on the provided parameters.\n",
    "\n",
    "7. The training data is then split into separate arrays for different input features: train_x_rainfall, train_x_discharge, and train_x_et. The train_discharge array is reshaped to include the required hours of history and prediction.\n",
    "\n",
    "8. Similarly, the validation and test data are split into separate arrays for input features and output labels.\n",
    "\n",
    "9. The function returns a list of arrays containing the training inputs (train_x_et, train_x_discharge, train_x_rainfall), the training output labels (train_y), the validation inputs and labels, the test inputs and labels, and the maximum and minimum discharge values (q_max and q_min).\n",
    "\n",
    "In summary, this function reads and scales the data, converts it into a supervised learning format, and splits it into training, validation, and test sets. It also separates the data into different arrays based on the input features and output labels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "executionInfo": {
     "elapsed": 1074,
     "status": "ok",
     "timestamp": 1607938388793,
     "user": {
      "displayName": "Zr Xiang",
      "photoUrl": "",
      "userId": "10347344196072452629"
     },
     "user_tz": 360
    },
    "id": "kFW20INaHjoO"
   },
   "outputs": [],
   "source": [
    "def prepare_data(station_id, hours_of_history, hours_to_predict, parameters_included):\n",
    "\n",
    "    data = pd.read_csv('./data/'+str(station_id)+'_data.csv').iloc[:,1:]\n",
    "\n",
    "  # simple min-max scaling. Other pretreatments such as normalization also work.\n",
    "    scaler = MinMaxScaler()\n",
    "    scaler.fit(data.iloc[:52608,:]) # min-max scaling without the test dataset.\n",
    "    q_max = np.max(data.iloc[:52608,2]) # manually check the maximum and minimum discharge\n",
    "    q_min = np.min(data.iloc[:52608,2])\n",
    "    data_scaled = scaler.transform(data)\n",
    "\n",
    "  # data split\n",
    "    data_sequence = series_to_supervised(data_scaled, hours_of_history, hours_to_predict)\n",
    "    data_train, data_valid, data_test = train_valid_test_split(data_sequence, hours_of_history, hours_to_predict, parameters_included)\n",
    "\n",
    "  # Split data into 2 parts for encoder (history) and decoder(future).\n",
    "    train_x_rainfall = data_train[:,0::3].reshape(-1, hours_of_history+hours_to_predict, 1)\n",
    "    train_discharge = data_train[:,2::3].reshape(-1, hours_of_history+hours_to_predict, 1)\n",
    "    train_x_discharge = train_discharge[:,:hours_of_history,:]\n",
    "    train_y = train_discharge[:,hours_of_history:,:]\n",
    "    train_x_et = data_train[:,3*hours_of_history+1].reshape(-1, 1) # the current hour et\n",
    "\n",
    "    valid_x_rainfall = data_valid[:,0::3].reshape(-1, hours_of_history+hours_to_predict, 1)\n",
    "    valid_discharge = data_valid[:,2::3].reshape(-1, hours_of_history+hours_to_predict, 1)\n",
    "    valid_x_discharge = valid_discharge[:,:hours_of_history,:]\n",
    "    valid_y = valid_discharge[:,hours_of_history:,:]\n",
    "    valid_x_et = data_valid[:,3*hours_of_history+1].reshape(-1, 1) # the current hour et\n",
    "\n",
    "    test_x_rainfall = data_test[:,0::3].reshape(-1, hours_of_history+hours_to_predict, 1)\n",
    "    test_discharge = data_test[:,2::3].reshape(-1, hours_of_history+hours_to_predict, 1)\n",
    "    test_x_discharge = test_discharge[:,:hours_of_history,:]\n",
    "    test_y = test_discharge[:,hours_of_history:,:]\n",
    "    test_x_et = data_test[:,3*hours_of_history+1].reshape(-1, 1) # the current hour et\n",
    "\n",
    "    return [train_x_et, train_x_discharge, train_x_rainfall], train_y, [valid_x_et, valid_x_discharge, valid_x_rainfall], valid_y, [test_x_et, test_x_discharge, test_x_rainfall], test_y, q_max, q_min"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The code provided defines a custom loss function called nseloss, which stands for Nash-Sutcliffe Efficiency. This loss function measures the normalized squared difference between the true values (y_true) and the predicted values (y_pred). The formula for the nseloss is as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "executionInfo": {
     "elapsed": 1064,
     "status": "ok",
     "timestamp": 1607938388795,
     "user": {
      "displayName": "Zr Xiang",
      "photoUrl": "",
      "userId": "10347344196072452629"
     },
     "user_tz": 360
    },
    "id": "_w-hAdAcFqNl"
   },
   "outputs": [],
   "source": [
    "# define custome loss function (you can use the simple 'mse' as well)\n",
    "def nseloss(y_true, y_pred):\n",
    "    return K.sum((y_pred-y_true)**2)/K.sum((y_true-K.mean(y_true))**2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The provided code defines a function called EDLSTM that constructs an Encoder-Decoder LSTM (EDLSTM) model for a time series prediction task. Here's an explanation of the code:\n",
    "\n",
    "1. The function EDLSTM takes three arguments: hours_of_history (the number of hours of historical data to include as input), hours_to_predict (the number of hours to predict in the future), and parameters_included (a list of parameters/features to include in the model).\n",
    "\n",
    "2. The code begins by defining the input layers for the three types of data: runoff observation (input_1), rainfall observation and forecast (input_2), and other non-timeseries data (input_phys).\n",
    "\n",
    "3. The LSTM encoder layers are defined for the runoff observation and rainfall observation/forecast inputs. Both LSTM layers have 256 units and return sequences set to False, indicating that they only output the last hidden state of the LSTM.\n",
    "\n",
    "4. The concatenate function is used to combine the input_phys, LSTM1, and LSTM2 layers into a single tensor.\n",
    "\n",
    "5. The RepeatVector layer is used to repeat the combined tensor along the time dimension to match the hours_to_predict.\n",
    "\n",
    "6. The LSTM decoder layer is defined with 512 units and return sequences set to True.\n",
    "\n",
    "7. The fully-connected dense layers are defined using a loop. The dim_dense list specifies the dimensions of each dense layer. For each dimension, a TimeDistributed dense layer with a ReLU activation function and a dropout layer with a dropout rate of 0.2 is applied.\n",
    "\n",
    "8. The final output layer is a TimeDistributed dense layer with a single unit and a ReLU activation function. The output is flattened to a one-dimensional array.\n",
    "\n",
    "9. The model is created using the Model class, with the defined inputs and main_out as the outputs.\n",
    "\n",
    "10. Finally, the constructed model is returned.\n",
    "\n",
    "This code builds an EDLSTM model that takes multiple inputs (runoff observation, rainfall observation/forecast, and other non-timeseries data) and predicts a single output. The model utilizes LSTM layers for encoding and decoding the temporal information and fully-connected dense layers for learning complex relationships in the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "executionInfo": {
     "elapsed": 1058,
     "status": "ok",
     "timestamp": 1607938388798,
     "user": {
      "displayName": "Zr Xiang",
      "photoUrl": "",
      "userId": "10347344196072452629"
     },
     "user_tz": 360
    },
    "id": "Vy6tFQRBRuUD"
   },
   "outputs": [],
   "source": [
    "def EDLSTM(hours_of_history, hours_to_predict, parameters_included):\n",
    "\n",
    "  # design network\n",
    "  # input of runoff observation, LSTM encoder\n",
    "    input_1 = Input(shape=(hours_of_history, 1), name='LSTM1_input') # shape should be 72*1 for runoff observation\n",
    "    LSTM1 = LSTM(256, return_sequences=False)(input_1)\n",
    "\n",
    "  # input of rainfall observation and forecast, LSTM encoder\n",
    "    input_2 = Input(shape=((hours_of_history+hours_to_predict), 1), name='LSTM2_input') # shape should be (72+24)*n=96*n, for rainfall observation (72) and predictions (24) for rainfall and additional stations (if there is no upstream station, n=1)\n",
    "    LSTM2 = LSTM(256, return_sequences=False)(input_2)\n",
    "\n",
    "  # input of other non-timeseries data, such as daily or monthly data.\n",
    "    input_phys = Input(shape=(1,), name='phys_input') # one single value of ET. shape = 1.\n",
    "\n",
    "  # connect all data\n",
    "    x = concatenate([input_phys, LSTM1, LSTM2]) # Get state vector.\n",
    "    x = RepeatVector(hours_to_predict)(x) # 24 is the output time dimension\n",
    "\n",
    "  # LSTM decoder\n",
    "    x = LSTM(512, return_sequences=True)(x)\n",
    "\n",
    "  # define fully-connected dense layers\n",
    "    dim_dense=[512, 256, 256, 128, 64]\n",
    "\n",
    "  # final fully-connected dense layer for final result\n",
    "    for dim in dim_dense:\n",
    "        x = TimeDistributed(Dense(dim, activation='relu'))(x)\n",
    "        x = TimeDistributed(Dropout(0.2))(x) # Some dropout for dense layers. Some paper mentioned that it is not recommend to have dropout between the RNN/LSTM/GRU layers. Thus, I only apply dropout in the dense layer.\n",
    "    \n",
    "    main_out = TimeDistributed(Dense(1, activation='relu'))(x) # here relu provides the final output non-negative, which is corrosponding to my min-max pre-prossing.\n",
    "    main_out = Flatten()(main_out)\n",
    "\n",
    "    model = Model(inputs=[input_phys, input_1, input_2], outputs=main_out)\n",
    "  \n",
    "    return model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "executionInfo": {
     "elapsed": 1047,
     "status": "ok",
     "timestamp": 1607938388799,
     "user": {
      "displayName": "Zr Xiang",
      "photoUrl": "",
      "userId": "10347344196072452629"
     },
     "user_tz": 360
    },
    "id": "PvTU6Ob4cc8A"
   },
   "outputs": [],
   "source": [
    "# identify KGE, NSE for evaluation\n",
    "def nse(y_true, y_pred):\n",
    "    return 1-np.sum((y_pred-y_true)**2)/np.sum((y_true-np.mean(y_true))**2)\n",
    "  \n",
    "def kge(y_true, y_pred):\n",
    "    kge_r = np.corrcoef(y_true,y_pred)[1][0]\n",
    "    kge_a = np.std(y_pred)/np.std(y_true)\n",
    "    kge_b = np.mean(y_pred)/np.mean(y_true)\n",
    "    return 1-np.sqrt((kge_r-1)**2+(kge_a-1)**2+(kge_b-1)**2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "executionInfo": {
     "elapsed": 1039,
     "status": "ok",
     "timestamp": 1607938388801,
     "user": {
      "displayName": "Zr Xiang",
      "photoUrl": "",
      "userId": "10347344196072452629"
     },
     "user_tz": 360
    },
    "id": "MptO3dI7Hucj"
   },
   "outputs": [],
   "source": [
    "def main():\n",
    "  \n",
    "  # parameters\n",
    "    station_id = 521\n",
    "    hours_to_predict = 24\n",
    "    hours_of_history = 72\n",
    "    parameters_included = 3\n",
    "\n",
    "    batch_size = 64\n",
    "    lr = 0.0001\n",
    "    epochs = 300\n",
    "    test_name = './'+str(station_id)+'_model1_'\n",
    "\n",
    "  # load data\n",
    "    x_train, y_train, x_valid, y_valid, x_test, y_test, q_max, q_min = prepare_data(station_id, hours_of_history, hours_to_predict, parameters_included)\n",
    "    model1 = EDLSTM(hours_of_history, hours_to_predict, parameters_included)\n",
    "\n",
    "  # compile settings\n",
    "    reduce_lr = ReduceLROnPlateau(monitor='val_loss', factor=0.3, patience=15, cooldown=30, min_lr=1e-8)\n",
    "    earlystoping = EarlyStopping(monitor='val_loss', min_delta=0, patience=20, verbose=1, mode='auto')\n",
    "    checkpoint = ModelCheckpoint(test_name+'model.h5', monitor='val_loss', verbose=1, save_best_only=True, mode='min')\n",
    "    optimizer = RMSprop(lr=lr)\n",
    "  \n",
    "    model1.compile(optimizer=optimizer, loss=nseloss) # I used the build-in RMSprop, loss function is 1-NSE. You can use 'mse', 'mae' as well.\n",
    "\n",
    "    # train model\n",
    "    history = model1.fit(x_train, y_train, epochs=epochs, batch_size=batch_size,\n",
    "              validation_data=(x_valid, y_valid), callbacks=[reduce_lr, earlystoping, checkpoint], verbose=1)\n",
    "\n",
    "  # save training loss\n",
    "    loss_train = history.history['loss']\n",
    "    loss_valid = history.history['val_loss']\n",
    "    loss_train = pd.DataFrame({'TrainLoss':loss_train})\n",
    "    loss_valid = pd.DataFrame({'TestLoss':loss_valid})\n",
    "    LossEpoches = pd.concat([loss_train, loss_valid], axis=1)\n",
    "    LossEpoches.to_csv(test_name+'loss.csv', index = True)\n",
    "\n",
    "  # Final Test Review\n",
    "    model1.load_weights(test_name+'model.h5')\n",
    "\n",
    "    y_model_scaled = model1.predict(x_test)\n",
    "    y_model = y_model_scaled*(q_max-q_min)+q_min\n",
    "    y_test = y_test*(q_max-q_min)+q_min\n",
    "\n",
    "  # hourly evaluation\n",
    "    NSEs=[]\n",
    "    KGEs=[]\n",
    "    \n",
    "    for x in range(0, 24):\n",
    "        y_pred = y_model[:,x]\n",
    "        y_True = y_test[:,x]\n",
    "        \n",
    "    NSEs.append(nse(y_True[:,0],y_pred))\n",
    "    KGEs.append(kge(y_True[:,0],y_pred))  \n",
    "    \n",
    "    NSEs=pd.DataFrame(NSEs)\n",
    "    NSEs.columns = ['NSE_Test']\n",
    "    KGEs=pd.DataFrame(KGEs)\n",
    "    KGEs.columns = ['KGE_Test']\n",
    "    \n",
    "    eva = pd.concat([NSEs, KGEs], axis=1)\n",
    "    eva.to_csv(test_name+'eva.csv', index = True)\n",
    " "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 444533,
     "status": "ok",
     "timestamp": 1607938836656,
     "user": {
      "displayName": "Zr Xiang",
      "photoUrl": "",
      "userId": "10347344196072452629"
     },
     "user_tz": 360
    },
    "id": "o03f8xK7ao9g",
    "outputId": "8425a0d2-1c7b-433f-839f-26271924be2a"
   },
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "Length mismatch: Expected axis has 288 elements, new values have 219 elements",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "Cell \u001b[1;32mIn[11], line 2\u001b[0m\n\u001b[0;32m      1\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;18m__name__\u001b[39m \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m__main__\u001b[39m\u001b[38;5;124m\"\u001b[39m:\n\u001b[1;32m----> 2\u001b[0m       \u001b[43mmain\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n",
      "Cell \u001b[1;32mIn[10], line 15\u001b[0m, in \u001b[0;36mmain\u001b[1;34m()\u001b[0m\n\u001b[0;32m     12\u001b[0m   test_name \u001b[38;5;241m=\u001b[39m \u001b[38;5;124m'\u001b[39m\u001b[38;5;124m./\u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;241m+\u001b[39m\u001b[38;5;28mstr\u001b[39m(station_id)\u001b[38;5;241m+\u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m_model1_\u001b[39m\u001b[38;5;124m'\u001b[39m\n\u001b[0;32m     14\u001b[0m \u001b[38;5;66;03m# load data\u001b[39;00m\n\u001b[1;32m---> 15\u001b[0m   x_train, y_train, x_valid, y_valid, x_test, y_test, q_max, q_min \u001b[38;5;241m=\u001b[39m \u001b[43mprepare_data\u001b[49m\u001b[43m(\u001b[49m\u001b[43mstation_id\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mhours_of_history\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mhours_to_predict\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mparameters_included\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m     16\u001b[0m   model1 \u001b[38;5;241m=\u001b[39m EDLSTM(hours_of_history, hours_to_predict, parameters_included)\n\u001b[0;32m     18\u001b[0m \u001b[38;5;66;03m# compile settings\u001b[39;00m\n",
      "Cell \u001b[1;32mIn[6], line 13\u001b[0m, in \u001b[0;36mprepare_data\u001b[1;34m(station_id, hours_of_history, hours_to_predict, parameters_included)\u001b[0m\n\u001b[0;32m     10\u001b[0m   data_scaled \u001b[38;5;241m=\u001b[39m scaler\u001b[38;5;241m.\u001b[39mtransform(data)\n\u001b[0;32m     12\u001b[0m \u001b[38;5;66;03m# data split\u001b[39;00m\n\u001b[1;32m---> 13\u001b[0m   data_sequence \u001b[38;5;241m=\u001b[39m \u001b[43mseries_to_supervised\u001b[49m\u001b[43m(\u001b[49m\u001b[43mdata_scaled\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mhours_of_history\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mhours_to_predict\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m     14\u001b[0m   data_train, data_valid, data_test \u001b[38;5;241m=\u001b[39m train_valid_test_split(data_sequence, hours_of_history, hours_to_predict, parameters_included)\n\u001b[0;32m     16\u001b[0m \u001b[38;5;66;03m# Split data into 2 parts for encoder (history) and decoder(future).\u001b[39;00m\n",
      "Cell \u001b[1;32mIn[4], line 18\u001b[0m, in \u001b[0;36mseries_to_supervised\u001b[1;34m(data, n_in, n_out, dropnan)\u001b[0m\n\u001b[0;32m     16\u001b[0m \u001b[38;5;66;03m# put it all together\u001b[39;00m\n\u001b[0;32m     17\u001b[0m   agg \u001b[38;5;241m=\u001b[39m pd\u001b[38;5;241m.\u001b[39mconcat(cols, axis\u001b[38;5;241m=\u001b[39m\u001b[38;5;241m1\u001b[39m)\n\u001b[1;32m---> 18\u001b[0m   agg\u001b[38;5;241m.\u001b[39mcolumns \u001b[38;5;241m=\u001b[39m names\n\u001b[0;32m     19\u001b[0m \u001b[38;5;66;03m# drop rows with NaN values\u001b[39;00m\n\u001b[0;32m     20\u001b[0m   \u001b[38;5;28;01mif\u001b[39;00m dropnan:\n",
      "File \u001b[1;32m~\\anaconda3\\lib\\site-packages\\pandas\\core\\generic.py:5915\u001b[0m, in \u001b[0;36mNDFrame.__setattr__\u001b[1;34m(self, name, value)\u001b[0m\n\u001b[0;32m   5913\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m   5914\u001b[0m     \u001b[38;5;28mobject\u001b[39m\u001b[38;5;241m.\u001b[39m\u001b[38;5;21m__getattribute__\u001b[39m(\u001b[38;5;28mself\u001b[39m, name)\n\u001b[1;32m-> 5915\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mobject\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[38;5;21;43m__setattr__\u001b[39;49m\u001b[43m(\u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mname\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mvalue\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m   5916\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mAttributeError\u001b[39;00m:\n\u001b[0;32m   5917\u001b[0m     \u001b[38;5;28;01mpass\u001b[39;00m\n",
      "File \u001b[1;32m~\\anaconda3\\lib\\site-packages\\pandas\\_libs\\properties.pyx:69\u001b[0m, in \u001b[0;36mpandas._libs.properties.AxisProperty.__set__\u001b[1;34m()\u001b[0m\n",
      "File \u001b[1;32m~\\anaconda3\\lib\\site-packages\\pandas\\core\\generic.py:823\u001b[0m, in \u001b[0;36mNDFrame._set_axis\u001b[1;34m(self, axis, labels)\u001b[0m\n\u001b[0;32m    821\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21m_set_axis\u001b[39m(\u001b[38;5;28mself\u001b[39m, axis: \u001b[38;5;28mint\u001b[39m, labels: AnyArrayLike \u001b[38;5;241m|\u001b[39m \u001b[38;5;28mlist\u001b[39m) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[0;32m    822\u001b[0m     labels \u001b[38;5;241m=\u001b[39m ensure_index(labels)\n\u001b[1;32m--> 823\u001b[0m     \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_mgr\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mset_axis\u001b[49m\u001b[43m(\u001b[49m\u001b[43maxis\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mlabels\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m    824\u001b[0m     \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_clear_item_cache()\n",
      "File \u001b[1;32m~\\anaconda3\\lib\\site-packages\\pandas\\core\\internals\\managers.py:230\u001b[0m, in \u001b[0;36mBaseBlockManager.set_axis\u001b[1;34m(self, axis, new_labels)\u001b[0m\n\u001b[0;32m    228\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mset_axis\u001b[39m(\u001b[38;5;28mself\u001b[39m, axis: \u001b[38;5;28mint\u001b[39m, new_labels: Index) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[0;32m    229\u001b[0m     \u001b[38;5;66;03m# Caller is responsible for ensuring we have an Index object.\u001b[39;00m\n\u001b[1;32m--> 230\u001b[0m     \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_validate_set_axis\u001b[49m\u001b[43m(\u001b[49m\u001b[43maxis\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mnew_labels\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m    231\u001b[0m     \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39maxes[axis] \u001b[38;5;241m=\u001b[39m new_labels\n",
      "File \u001b[1;32m~\\anaconda3\\lib\\site-packages\\pandas\\core\\internals\\base.py:70\u001b[0m, in \u001b[0;36mDataManager._validate_set_axis\u001b[1;34m(self, axis, new_labels)\u001b[0m\n\u001b[0;32m     67\u001b[0m     \u001b[38;5;28;01mpass\u001b[39;00m\n\u001b[0;32m     69\u001b[0m \u001b[38;5;28;01melif\u001b[39;00m new_len \u001b[38;5;241m!=\u001b[39m old_len:\n\u001b[1;32m---> 70\u001b[0m     \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mValueError\u001b[39;00m(\n\u001b[0;32m     71\u001b[0m         \u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mLength mismatch: Expected axis has \u001b[39m\u001b[38;5;132;01m{\u001b[39;00mold_len\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m elements, new \u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[0;32m     72\u001b[0m         \u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mvalues have \u001b[39m\u001b[38;5;132;01m{\u001b[39;00mnew_len\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m elements\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[0;32m     73\u001b[0m     )\n",
      "\u001b[1;31mValueError\u001b[0m: Length mismatch: Expected axis has 288 elements, new values have 219 elements"
     ]
    }
   ],
   "source": [
    "if __name__ == \"__main__\":\n",
    "      main()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 42,
     "status": "ok",
     "timestamp": 1607938836664,
     "user": {
      "displayName": "Zr Xiang",
      "photoUrl": "",
      "userId": "10347344196072452629"
     },
     "user_tz": 360
    },
    "id": "s2fcQj_ET3oG"
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "authorship_tag": "ABX9TyMldi9Jo6R9GQCQodu38jKC",
   "collapsed_sections": [],
   "machine_shape": "hm",
   "name": "Model1_Encoder_Decoder_LSTM.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}