Modifying Hidden Layer in Neural Network Models to Improve Prediction Accuracy: A Combined Model for Estimating Stock Price

Investment experts, who deal with stock price estimation, commonly look for the most accurate and appropriate statistical techniques to make decisions on investment. The aim of this study is to improve the accuracy of stock price prediction models through modifying the structure of a combined neural network model with time-series data, in which the main contribution is to insert the time-series analysis prediction into the hidden layer of the neural network. The proposed structure is made up of neural networks and time-series analysis, with variable reduction used to remove attributes with inter-correlations. Data has been collected over six years (72 months) from the Iranian stock market, including the number of trades, new-coin price, gold-18 price, US Dollar and Euro equivalent currencies, oil-index price, Brent-oil price, industry index, and balanced stock index, followed by developing the prediction models. Comparing the performance criteria of the proposed structure to the traditional ones in terms of the mean square and mean absolute errors revealed that inserting time-series estimated variables into hidden layers would improve the performance of neural network models to estimate stock prices for making investment decisions.


Stock Market Prediction
Prediction of stock prices and market situations are essential issues in financing where they are getting more attention from investment experts who are willing to make proper decisions and to ensure that the positive direction of the market is successfully predicted [1]. On the other hand, the existence of the nonlinearity and volatility of the financial market has been identified by many researchers and financial analysts [2]. Therefore, many models have been proposed utilizing a variety of fundamental, technical, and time-series forecasting techniques to gain accurate predictions in this field. One of the main concerns in stock market investment is to gain an overall view of the future and predict the trend of stock prices as well as illustrative graphs to make the right decisions and affordable plans for the future. Although precise forecasting of markets may be impossible, the researchers intend to tackle this problem by proposing methods that are more accurate and comparing the accuracy of the prediction models to select the best method. Some characteristics of financial time-series data, such as non-stationarity, nonlinearity, and high volatility, prompt investment professionals to create more accurate and fitted models [3]. While some of the forecasting models' and applications' results do not satisfy investors, they can learn the stock market's operation rules to gain a clear context of what happens in various situations. In this regard, forecasting studies are almost always regarded as difficult due to the existing uncertainty in the stock price system and the complexity of external economic and environmental factors.

Neural Network Models for Prediction
Because of the rapid development of artificial intelligence and computer technology, the stock price forecasting models are constantly updated, integrated, and improved. Among the above techniques, neural network modeling is getting more attention because, compared to other traditional methods, it has achieved amazing results in the field of prediction [4]. The basic concept behind the neural network structure is to develop a model to interpret the relationships between variables according to its internal relations, so it is widely used in pattern recognition, intelligent control, signal processing, and other fields in which there are relations between attributes. The artificial neural network models are also used for predicting the stock price. In the modeling of stock prices, mainly in non-linear format, the rationality and applicability of the model construction have their own advantages that can provide the nonlinear prediction model with a wider space for sensitivity analysis [5]. In the case of financial time-series, data is extremely nonlinear and fluctuates. Time-series approaches are usually attributed to dynamic systems, so we need algorithms to interpret the hidden patterns and underlying dynamics behavior of data, so called machine learning techniques. Although machine-learning techniques have been recently improved and applied in various research fields, the estimating methods still need to be improved to be suitable for time-series analysis [6]. To deal with the above challenge, neural network models have been developed. Despite the advantages of the neural networks that have been extensively considered for the prediction of the stock market, sometimes they fail to predict the financial markets accurately [7].
Putrier [8] believed that investment and fund management could be defined as optimal dynamic problems, and that they should predict portfolio dynamical behavior in order to optimize investors' capital structures. A comprehensive study has also been conducted, focusing on the common parameters for designing a back propagation neural network and providing a systematic methodology for forecasting economic time-series data [9]. Komo et al. [10] developed and compared two neural network models, Radial Basis Function (RBF) and back propagation (also known as multilayer perceptron (MLP)), for predicting stock market prices by employing data from the Wall Street Journal's Dow Jones as a benchmark. A notable success of the proposed models was achieving prediction accuracies of over 80% based on the Dow Jones monthly industrial index predictions, and the results demonstrated that RBF neural networks are preferred to MLP networks.

Stock Market Prediction Methods
Let us enumerate some of the available forecasting methods used in the prediction of stock prices. They are not only used for stock market prediction but also in fields that generate numeric measures that can be computed based on time-series data.
Fundamental Analysis: Fundamental analysis is a kind of investment analysis adopted by investors for taking decisions [9]. By studying a company's sales, management proficiency, earnings, dividends, profits, and a host of other economic factors, they basically estimate the intrinsic worth of a company's share where the above factors have a bearing on the company's profitability and business prospects. The process leads experts to estimate the price of a particular company's share and consider the estimated value as the intrinsic or true value of the share, which reflects the inherent worth and value. The estimated intrinsic price would help investors judge whether the shares are currently over-priced or under-priced. A fundamentalist who uses fundamental analysis makes money by purchasing underpriced stocks and selling them when they become overpriced. Fundamental analysis is more useful for long-term investments.
Technical Analysis: In technical analysis, a large number of rules and indicators are committed to identifying and explaining the regularity of dynamic historical prices. Technical analysis makes use of patterns in a financial instrument's price history to forecast price behavior in the future [11]. Technical analysts argue that prices gradually adjust to new information, so the moving average method is one of the most common techniques utilized by technical analysis. Although the MA method is easy to use and apply in investment decision-making or empirical tests [12], the research conducted by Dzikevicius [13] showed that moving average methods may generate errors and deviations in forecasting, so they would not be successful in estimating the trend of prices in long-term decisions. Technical analysts, who are also called "chartists" as they use charts and graphs to keep a record of share price movements, believe that an accurate study of share price charts and graphs would reveal regular and recurrent patterns of price behavior that are likely to be repeated in the future [14]. In brief, technical analysis attempts to predict the future price of a particular share based on a study of its price movements in the past, so it is more commonly used for taking "buying" and "selling" decisions in the stock market than the other techniques.

Time-series Forecasting:
Time-series forecasting intends to predict a dependent variable or an attribute for the future based on its past behavior. This is a significant concern in the field of stock market investment, where investors are willing to make the right decisions at the right time to maximize their financial profit. Time-series forecasting usually comes across a specific trend in the past data to predict the future data, so conventional research usually uses time-series analysis techniques like mixed auto regression, moving average, and multiple regression models [15]. Overall, if data is available for a long period but at the same intervals, the easier way is to find a pattern to predict the future, but if the history of a stock is not enough or segregated data is available, the accuracy of the analysis and forecast is getting a little difficult. There are a lot of methods in this field, but the most well-known ones are the moving average, autoregressive moving average, and autoregressive integrated moving average, followed by seasonal prediction methods. Among them, the moving average works such that the average of a fixed number of items in the time-series is attributed to the next time interval. Fluctuating data that moves through the series is smoothly uniformed by dropping the top items and adding the below ones with each successive average [16]. For application and in the modeling of linear and stationary time-series data, researchers usually employ combination models for many purposes due to their superiority, ease of implementation, and robustness [17].

Artificial Neural Networks Structure in Stock Market Prediction
Investing in the stock market usually involves higher risk due to its uncertainty and volatility [18], so forecasting the stock price behavior is crucial in terms of accuracy. The difficulty arises when nonlinear and complex behavior of stock prices is observed. Due to the dynamic environment together with existing incomplete or noisy data that may be observed, for example in traffic conditions [19], the artificial neural network models are suitable to be utilized because of their proper adaptation to this kind of data [20]. Therefore, in the last two decades, research has constantly attempted to develop neural network models for forecasting stock prices [21]. There are many types of neural network models where Multi-Layer Perceptron (MLP) are feed-forward neural networks with one or more layers between the input and output layers. Feed-forward means that data flows in one direction from an input layer to an output layer (forward) through a transitional layer, usually called a hidden layer. An MLP consists of multiple layers of nodes in a directed graph in which each layer is connected to the next one. Except for the input nodes, each node is a neuron, known as a processing element, with a nonlinear activation function to connect the previous nodes and make another node in its own layer. The above characteristics would improve the prediction model to be able to resolve problems that are not linearly-based structured with one or more hidden layers [22]. In a neural network model, the output Yi of each neuron of the n th layer is mathematically defined by a derivable nonlinear function based on Equation 1, where F is the non-linear activation function, Wji is the weight of the connection between the neuron Nj and Ni, and finally Yi is the output of the neuron of the (n−1)th layer [23].

Performances of Neural Network Models
In general, neural network modeling is among data processing techniques called data mining. Setty et al. [24] reviewed the applications of data mining techniques to evaluate the performance of stock markets and concluded that there is a rising gap between storage and retrieval systems. Since storage is more powerful, a technological leap needs to be made to prioritize information about end-user problems. When Dase et al. [25] reviewed the literature on the application of artificial neural networks to stock market predictions, they demonstrated that data mining tools are useful in this field. They revealed that predicting stock indexes through performing traditional time-series analysis is too difficult, but the artificial neural network may be more suitable, so they revealed that the artificial neural network is a useful technique for predicting stock markets. In the field of combining prediction techniques, Kumar et al. [26] introduced some basic ideas of time-series data, the need for ANN, the importance of stock indices, and a survey of the previous works, and investigated neural network models' applications for time-series in forecasting, and the result was that existing functional relationships are the main scientific characteristics of the above models. In the preceding study, ANN performance measures such as mean square errors, root mean square errors, mean absolute errors, and others were defined for model comparison, and it was concluded that the ANN model achieved the lowest prediction errors when fitting them to a large amount of stock market data.
Attempts and observations in the literature for improving ANN performance include changing the number of nodes in the hidden layer [27], changing the number of network modules and channels [28], and improving accuracy through knowledge distillation [29], where filtering is used as a compression technique. As more studies for more approaches, Li [30] improved the accuracy of ANN models through combing the above networks with intelligent diagnosis in medical treatments where the combined model improved diagnosis efficiency and saved doctors' time. Instead of outputs, the residuals from a two-step combined model are estimated, and the accuracy of the prediction is determined by the set of errors and indices [31]. The prediction accuracy of neural networks can also be improved by composing them with time-series data as well as developing hybrid methods [32].

Vision Statement
Following the above mentioned, it is concluded that the combination of prediction techniques to achieve more accurate predictions of stock exchanges by neural network models may be a useful approach, but not in the common way where all attributes are simultaneously considered as input variables. Therefore, the novelty of the present research methodology is to develop another approach different from the previous ones. It is developed to insert a part of a data series into the hidden layer of the neural network instead of the input layer. Therefore, the concept behind this research work is to estimate the desired variable during a specific period in the first stage and predict the stock exchange price in the second stage. The time-series analysis results are inserted as hidden nodes in the hidden layer of the neural network model.

Data Gathering
This research work focuses on the stock price forecasting, so Tehran Stock Exchange has been selected as case study because of data availability and Shiraz Petrochemical Company  Table 1.

Inter-correlation Test
Some of so-called independent variables may have inter-correlation with each other or the researchers may intend to reduce the number of variables. Although, many techniques of data reduction are studied and implemented over the financial studies [33], in particular in prediction models [34], but the inter-correlation test is commonly utilized to ensure that the variables those are considered as input in the modeling procedure would not have inter-correlation. Statistically, the correlation between to variables is obtained by Equation 2 followed by utilizing the hypothesis test of t-test where the Equation 3 calculates t-stat and the corresponding P-Value is obtained based on t-Stat. The P-Value is checked to decide if the variables are significantly independent or correlated. The procedure will continue until the variables do not have inter-correlation [35]. The whole procedure is named variable reduction which is needed prior to modeling stage.

Neural Network Modeling
Developing model is following the variable reduction stage where input, hidden, and output layers are defined based on the purified variables and time-series node is inserted as a hidden node in hidden layer. The number of hidden nodes excluding time-series node which are all depicted in Figure 1 where the time-series node is shown as TS-Node. As shown it is a combined model of neural network and time-series analysis where the results of time-series analysis is a part of neural network model.

Running Model
The stock price of Shiraz Petrochemical Company (SPC) is now predicted by inserting time-series analysis into the neural network model and the results are discussed to estimate the output for the future. In addition, the structure of neural network model will be developed in various types to reach the best performances of the model.

Validation
The final stage of the research methodology is validation stage in which the performance criteria of the combined time-series and neural network models will be evaluated. Two well-known criteria of mean square errors (MSE) and mean absolute errors (MAE), respectively formulated by Equation 4 and 5, are compared where is the observed value for Stock Market Price Shiraz Petroleum Company at time t and ̂ is the estimated one at the same time. A schematic view of the research stages is depicted in Figure 2.

Descriptive Statistics
The first stage of the research methodology is to investigate the data downloaded from the sites mentioned in the previous section. Data has been gathered duration six years (72 months) from May 2012 to March 2018. Regardless to the variables selected for developing the neutral network model, Table 2

Inter-correlation Test
The second stage is to check the inter-correlation for all candidate variables as inputs. The t-test has been utilized to check whether there is inter-correlation between the independent variables or not. The first step is to calculate pairwise correlations between all variables by Equation 2 which are eventually tabulated in Table 3. The second step is to calculate t-Stat through Equation 3. The results have been tabulated in Table 4. Finally, the third step is to obtain P-Value for the first type of errors in hypothesis testing followed by tabulating them in Table 5. As shown, the P-Values that are more than 0.05 and highlighted in gray that should be removed from the modeling process because they have inter-correlations with the other variables.  According to what has been concluded from the inter-correlation test, among all variables which corresponding data are available, the independent variables are Trade Number (TN), New-Coin price(NC), Gold 18 (G18), US-Dollar (USD), Oil-Index (OIN), Industry-Index (IND), and eventually Balanced-Index (BLI). From now on, the model structuring uses the independent variables to develop the neural network model as well as the stock price for Shiraz petroleum company (SPC) that will be considered as dependent variable.

Network Modeling
Following the purification process of candidate variables and extracting independent ones, it is time to develop the neural network models. The stock price for Shiraz Petroleum Company (SPC) is estimated based on the dependent variables. According to the approach followed in this study, the neural network model is constructed in two stages. The first stage is to obtain coefficients for constructing hidden nodes' values (HNs) based on independent variables, and the second stage is to obtain output values where one of the hidden nodes is the moving average value of stock price for k last periods. Equations 6 and 7 represent the general functions developed in the above mentioned stages. The hidden layer can be also received another node by an exponential smoothing time-series value in which the Equation 8 indicates the exponential time-series where α is the smoothing factor, F denotes the forecasted, and R is the observed value of stock for dependent variable.
To illustrate how the model is developed, one of the models, as a sample, is constructed in neural network format and represented by Equations 9 to 12. As shown, the time-series node is used in the second series of equations where a two period moving average for SPC is inserted for formulation. It should be mentioned that the coefficients have been obtained where the target is to minimize mean square errors. The other possible models have been also developed and the results are discussed in the next subsection.

Validation
The last stage is to validate the model performances in different models. Two criteria of mean square errors (MSE) and mean absolute errors (MAE) have been calculated and tabulated in Table 6. The first column identifies the timeseries method combined with neural network. Three different approaches of no time-series, adding moving average, and exponential smoothing are studied where the structure of input layers are the same for all models. The third column represents the hidden layer components in which moving average approaches compose of one, two, and three moving average durations and smoothing factor is different model to model for exponential smoothing approaches. The fourth and fifth columns respectively represent the mean square errors (MSE) and mean absolute errors (MAE) all depicted in Figures 3 and 4 as well where they respectively show the performance criteria for moving average and exponential smoothing combined models. In both figures, mean square errors (MSE) is depicted at the left side and mean absolute errors (MAE) is depicted at the right side. Looking more carefully at what has been derived from Table 6 and Figures 3 and 4, it is observed that the worst performance criteria belong to the model developed in the traditional method in which time-series variables are absent from the model. Inserting time-series values into the hidden layer significantly affects the performance criteria where both MSE and MAE have the same behavior in that they are decreased if time-series values are inserted into the hidden layer. Therefore, it is concluded that modifying the structure of the hidden layer would improve the efficiency of neural network prediction performances by not only adding a moving average node in the hidden layer but also other outputs of time-series analysis, such as exponential smoothing values.

Conclusion
In the present research work, the structure of the neural network model has been modified to improve the performance of prediction ability for estimating dependent variables based on predictors. Two methods of time-series analysis, including moving average and exponential smoothing, have been separately inserted as hidden nodes in the structure of the hidden layer. The modified neural network models have been utilized to predict stock exchanges in the Iranian stock market, where Shiraz Petroleum Company (Iran) was under study. Neural network models have been developed by employing experimental data collected over six years, followed by testing inter-correlation coefficients to ensure that dependent variables would not have inter-correlation. All models in which the time-series results are inserted as hidden nodes have been evaluated based on two criteria: mean and absolute square errors. Looking more carefully at the obtained results, it is revealed that the modification of the hidden layer in the neural network models would improve the accuracy of prediction models, at least in stock market predictions.

Data Availability Statement
The data presented in this study are available on request from the corresponding author.

Funding
The authors received no financial support for the research, authorship, and/or publication of this article.

Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.