Machine learning based artificial neural networking (ANN) has been acknowledged to be an authentic way to accomplish intricate pattern identification and regression analysis without any obvious necessity to paradigm and resolve the primary physical models. ANN has been introduced and adopted in various fields of life based-on their key advantages including learning and adapting ability, parallel distributed computation, robustness, and many more. This review articles discussesthe working principle, classes and structure of ANN. The potential applicable areas of ANN, the challenges and their solutions that have been identified through literature survey, are discussed and summarized.
Machine learning, Artificial neural networking (ANN), Potential applications of ANN, Challenges and solutions in ANN
During industrialization, remains a timeless manufacturing goal: To produce high quality products at minimum cost. In this context, manufacturing processes have intensely changed due to implementation of advanced manufacturing technologies specifically automation that can be done using computer programming and algorithms employing artificial intelligence (AI). In industrial AI, the process known as "training", enables the machine learning algorithms to detect anomalies correlations while performing test searching for patterns across the various data feeds, so facilitating the manufacturing process and making it easier for people to work [1].
Although, there are tremendous applications of AI in various aspects of manufacturing systems but expert systems are the most commonly used application of AI. Expert systems are commonly used in designing, scheduling, process planning, quality control, and so on; However, there are various pros and cons as well such as Ali, et al. (1990) explained that the number of variables that must be engaged into consideration while neural model formation has become excessively abundant [2]. Due to vagueness and sheer extent of items to be considered in calculation approaches cannot rationally resolve the problem; Therefore, ample development needs to be done in both the neural network complexity and computing capacity prior to real-intelligent program formation [3]. These expert systems are found to be least efficient and effective in modern manufacturing systems due to complex, varying, and highly productive manufacturing environment.
In this modern era, there is a strong need of AI category that must capable to respond quick alternations within automated productive environment; furthermore, it must be able to learn and absorb quick changes, identify the trends, and forecast human being thoughts with minor human interferences. Moreover, it should identify the patterns and correlations among dominant factors which are not deceptive to human beings.
In the recent few years, several studies have been stated that artificial neural network (ANN) have proved their above mentioned-capabilities effectively in many practical areas such as planning, financial administration, investment and assurance amenities, engineering, and manufacturing systems [4-7]. ANN is one of the most interesting topics of research, and an incredible quantity of literature has been produced, in the last 15 to 20 years, specifically on ANN, their abilities and implementation in various fields of life. However, this literature is extensively dispersed in various disciplines and publications.
The intention of this study is to present an overview of ANNs and their applications that make it appropriate for various aspects of life. This review article is divided into major three sections. The 1st section discusses the working principle, classes and structure of ANN. The potential applicable areas of ANN have been identified in 2nd section. The 3rd section discusses the challenges and their solutions that have been identified through literature survey. The purpose of this review is to present the readers an overall picture on concepts, applications and perspectives of ANN in various fields of life, thus providing some useful guidelines and references for the research and implementation.
In this section, introduction, working principle, various ANN classes, and structure of ANN.
Machine learning (ML) can be classified into two major learnings: (a) Supervised ML and (b): Unsupervised ML. One of the easiest ways to identify between the two is to check whether the given dataset has labels on it or not. Therefore, ANN is identified as a type of supervised ML because in an ANN, the model has been told the response to the inputs. This makes suitable the ANN for various applications, since clear targets and requirement methods are already identified. An ANN has robust assessing techniques for complex and extremely nonlinear associations between the given input and output. It has been identified that a network having single hidden layer, but an adequate number of neurons can build an arbitrary function [8].
An ANN usually consist of three layers: (a) An input layer, (b) A hidden layer, and (c) An output layer [9], as shown in Figure 1 and Figure 2. Each layer contains several neurons or nodes, which is defined as the point where computation starts. The ANN calculates the connection magnitude of each input and assigns weightage of the given inputs. The weightage values are usually determined by training the ANN model iteratively, to minimize the loss function between the estimations and actual outputs. To train an ANN model, two methods are commonly used (a): Feedforward data input (FFDI) and (b): Backward Error Propagation (BEP) [10,11]. A comparison between the FID and BEP is given in the following Table 1. The ANN has the capacity to predict the outputs for the unseen inputs, once the ANN model is trained.
The three classes of ANN, given in Table 2, have proved their worthiness and gained popularity.
The structure of an ANN usually consists of four major sub-sections: (a) No. of hidden layers, (b) No. of neurons in each layer, (c) Activation function, (d) and loss estimation function. The details are summarized in Table 3.
Figure 3 illustrates the possible various applications of ANN. It can be observed tremendous efforts have been carried out by a lot of researchers to implement ANN in their respective fields, which identifies the potential of ANN algorithms to reach for a feasible solution with the designed inputs.
Hopfield and Tank (1982 and 1985) carried out numerous studies on the application of ANN in resource management and allocation, transport phenomena, and manufacturing systems [30,31]. They used ANN to find out the rapid solution of salesman problem (SP) and validate that ANN can be used to find out the quick results of a situation which is hard to analyze using traditional methods. An analog circuit along with the SP plotting was used to validate the computing power and speed of an analogous and immensely connected neurons networks. A map and graph K-coloring problem using Hopfield network was solved by Dahl (1987). Initially, a 100-region map problem (NP-Complete) was proved, and then transformed into SP before implementing an ANN to achieve an optimum solution [32]. Moreover, Hopfield network was used by Page and Tagliarini (1988) in order to resolve the problem related to resource allocation [33]. Tangliarini and Page (1987) implemented ANN technique to resolve queens' problems; the results show that ANN resulted in ninety (90) achievable solutions out of ninety-two problem constraints [34].
Marcus (1987) designed an approach, with a set of algorithms, alternative to the Hopfield's networking strategy; the results show that Marcus' algorithm is proficient in producing solutions analogous to Hopfield' solutions [35]. Levy and Adams (1987) presented an optimization strategy which combines computational annealing procedure with Hopfield' network, found to be much more efficient than Hopfield' strategy [36]. It has been that ANN reduces the computation time as compared to the conventional problem-solving methods. Saylor and Stork (1986) modelled a decision-making procedure using ANN, found to be effective and consumes less time in comparison to a conventional technique [37]. Moreover, Udo (1991) solved warehousing problems using ANN, and the results indicated that the ANN algorithms performed efficiently with the reduction in 66% computation time and average memory storage [38].
Various researches have been carried to implement ANN in maintenance and repairing planning and solutions. In this scenario, Chaudhri and De (2010) developed Rough Fuzzy Multi-Layer Perception ANN for job scheduling problems. ANN algorithms produce optimal schedules for the given inputs, which enhances the organization strength, and apprehends the predictive knowledge linked to the assignment of task to operators in sequence [39]. Foo (1998) used an ANN algorithm to resolve 4-men with 3-machines (men-machine) scheduling problem that resulted in an optimum solution with the given inputs [40]. In addition, Foo and Takefuji (1987) developed an ANN with linear programming (ANNLP) and used to solve man-machine job problem; the results exhibited a near to optimal solution can be obtained using ANNLP [41]. Rabelo and Alptekin (1989) proposed an algorithm, flexible manufacturing systems (FMS), in which ANN in combination with expert manufacturing systems were used to produce optimum solutions. The designed algorithms resulted in quality decision making that leads to quality enhancement in manufacturing processes [42]. Noaker (1988) proposed that the technology must be employed for six functions mainly: (1) Locating, (2) Condition analysis, (3) Absence recognition, (4) Examination, (5) Identification of condition, and (6) With the emphasis on detection when the conditions are no more favorable and fixing them using proactive approach [43].
Subbaraj and Kannapiran (2010) presented the design and development of an ANN model for the fault detection of Pneumatic valve in cooling system with the capability to identify total nineteen faults. The model was trained using real time data, and the predicted results were found close to the real-time results [44]. Ray (1991) applied ANN in engine fault analysis; the results obtained from the case study explained that ANN provided the analogues judgements as human experts with the efficiency of 75% time [45]. Yam, et al. (2001) reported a study on implementation of ANN on condition-based maintenance. In their study, an intelligent predictive decision system model, using the recurrent artificial neural network method, was established and verified for equipment in a power plant; the results presented that the ANN model provided the fault diagnosis with an accuracy, and predicted the equipment life while considering the deterioration [46]. Oladokun, et al. (2006) implemented the ANN for maintenance management; the results show that the model can predict the expected downtime with 70% accuracy [47].
Various researches have been carried on the implementation of ANN for quality control and process planning. In this regard, Chang and Jiang (2002) developed an ANN model to review the dependency between the quality of finished product and sensor measurements to examine the tool failure during manufacturing [48]. Port (1990) reported on the implementation of ANN in "Anderson Company" w.r.t quality inspection with the intention to improve the performance from 350 per million-100 per million [47].
Thomas (1995) explained that ANNs can be used to control a process statistically, classify the kind of variations had occurred within the system [49]. In order to identify the defects in loudspeakers, an ANN strategy was implemented by "CTS electronics" that resulted in precise, faster, and accurate inspection. Moreover, Ford motor implemented the ANN models to inspect the paint quality of cars and to detect the engine problems [50]. English and Sastri (1990) explained a new technique regarding quality control in a steady-state manufacturing process using ANN technique and a close correlation was found between ANN model prediction results and real life results [51]. Patrick (1991) presented an application of ANN on washing operations. The results indicated a 25% improvement washing efficiency with good quality, and improved profits [52].
ANN can be used to control, manage, and supervise database. Bandara, et al. (2020) presented an estimation model that can be used with various recurrent neural network models. The model was tested using long short-term memory networks. A closed correlation was found between competitive outcomes and benchmarked datasets [53]. Amorim, et al. (2019) implemented semi-supervised techniques to categorize unlabeled trainings set with the training on convolution ANN. The technique was experimentally tested to identify soybean leaf, and herbivorous pests [54]. Janowski and Horzyk (2018) presented an ANN model for maze examination and knowledge-based adaptation. Their unique approach is beneficial for the situations when there is no initially well-defined set of data for training, but can be developed gradually within training progression based on the factors that can be introduced from the working environment [55].
Shahid, et al. (2019) reported that health care organizations are using ANN technique to identify health care management decisions from a given database [56]. Hillman (1990) presented a technique in which ANN and expert systems are collectively used to develop a database; the results exhibited that an increment in productivity was found without a declination in the final product quality [57]. An application of ANN in database management field was reported by Lelu (1991); Test or multimedia-based browser can be placed and queried using this database management technique [58].
Wang, et al. (2019) presented a unique technique using robot learning demonstration. In this technique, an ANN-based controller was designed for the robot to track and identify the trajectories created using motion-model, ANN radial function was used to accumulate the consequences generated using dynamic situations. The experimental results performed using a robot (Baxter), were validated using ANN model [59]. Chen, et al. (2019), proposed a technique with constrained optimization recurrent neural network so that mobile robots can be handled easily; a comparison was made between numerical simulations and existing literature on recurrent neural network which resulted in good understanding between the two [60]. Zheng, et al. (2019) presented a robust design for controller to control the movement of a robot. An ANN model was used to estimate the relation between the actuators and the position of the robot (soft); The reliable results can be achieved using Zheng' robust controller [61].
ANN are commonly used in simulations as well. Sahiner and Liu (2020) proposed an improved method for the analyses of gamma spectra produced by neutron, using an ANN model in combination with Monte Carlo simulations. The model can predict the concentrations of elements within a range of absolute errors (1.8~2.8%) [62].
Efficiency, quality, and production in a manufacturing system can be enhanced using computer assisted processing and planning by reducing lead times and expenses by implementing improved manufacturing practices thus improving competitiveness in the market. CAPP attempts to capture the thoughts and methods of the experienced process planner. Variant systems are understandable, generative systems can plan new parts. Expert systems increase flexibility, fuzzy logic captures vague knowledge while neural networks learn. The combination of fuzzy, neural and expert system technologies is necessary to capture and utilize the process planning logic. A system that maintains the dependability and clarity of variant systems, is capable of planning new parts, and improves itself through learning is needed by industry.
Various researches have been carried out in injection molding (IM) processes. Bensingh, et al. (2019) implemented hybrid ANN model and particle swarm optimization (PSO) technique in IM process of lens (bi-aspheric) to achieve least variation in volume shrinkage, which is more challenging and crucial for the optical quality. The developed hybrid network was trained and tested using experimental for the optimization of the IM process parameters. It was observed that the developed hybrid ANN model presented the optimized IM process parameters of the bi-aspheric lens having an optical power equal to 27.73 Diopter and seventh order spherical abnormalities [63]. Everett and Dubay (2017) designed and developed an ANN model to regulate the process parameters to govern the mold cavity temperature in IM process. The results demonstrated that the developed ANN model accurately predicted the dynamic performance of the mold cavity temperature. This is one of the keys in many industrial applications, where the critical factors in a system are not directly observable [64]. Yin, et al. (2011) developed a Back-Propagation ANN model for warpage estimation and optimization of IM parts based on the operating process parameters. The finite Element (FE) simulations were carried out on the basis of input and output, and the results of FE simulations were used to train the developed the ANN model. It was found that the ANN model has the ability to estimate the warpage in IM parts within a range of 2% error. Moreover, ANN has the capability to optimize process parameters including energy consumption and production cycle. The optimized warpage value equal to 1.58 mm was achieved with ANN model, which is 32.99% shorter than the initial warpage (2.358 mm), and the cooling time was reduced from 20 to 10s [65].
The ANN models are widely used by various researchers in the forming processes. Monajati, et al. (2010) used a feed forward back-propagation neural network model to illustrate the influence of operating parameters on the evolution of mechanical properties and formability of steel sheets deep drawing. This model contains nineteen operating parameters. After the training process of ANN model, the results showed that the model can be used in a quantitative way to guide and control the final properties of formed low carbon steel items [66]. Downes and Hartley (2006) developed an ANN model to assist the design of roll in cold-roll forming processes, which can deliver guidance to the inexpert designers to design a better roll on the basis of input parameters [67]. Brahme, et al. (2009) presented an ANN model-based model for the prediction of textures generated in cold rolling of steels sheets. A feedforward interconnected ANN with standard backpropagation error algorithm was used. The texture data in the form of texture intensities including amount of carbon, carbide magnitude and the amount of reduction ratio was used as input to train the model while the output of the model was in the form of fiber texture. It was found that the predictions of the ANN can provide an excellent match between experimental data and estimated analyses [68].
In addition, ANN model has been implemented in the electric discharge machining (EDM) process. Surya, et al. (2017) developed an ANN model to predict the machining parameters in case of Wire-EDM of Al7075+TiB2 composites to attain maximum material removal rate, minimum dimensional errors and better surface roughness. The controlling factors indulged during the study were pulse-on and-off time, and current and bed speeds. The predicted parameters were found in the good agreement with the experimental ones within a range of 0.003~3.87% for material removal rate, 1.3~12.51% for surface roughness and 0.03~1.31% for dimensional accuracy [69]. Ugrasen, et al. (2014) presented a study on optimization and influence of machining parameters on precision, roughness and volumetric material removal rate of manufactured parts in Wire-EDM process. The important machining parameters influencing the precision, roughness and volumetric material removal rate were identified using analyses of variance method. The controlling factors identified were Pulse-on and-off time, Current and Bed speed. An ANN model with back propagation feed forward neural network and Levenberg-Marquardt algorithm was used to establish and train the model. It was found that the 70% training data set correlated well with the experimental parameters [70]. Vishnu, et al. (2018) established an analytical model in combination with an ANN model (back propagation algorithm) for machining Inconel-718 via EDM using various cutting conditions of polarity, pulse on- and -off time, and peak most current. The material removal rate, surface roughness and cutting tool wear rate were selected as output. Sufficient level of fitness was observed for the trained model. A comparison was made between experimental and the estimated analyses, which shows that ANN can estimate the values with an accuracy of 98.82%, 88.02%, and 93.97% for material removal rate, cutting tool wear rate, and surface roughness, respectively [71].
Various applications can be found for the implementation of ANN in casting process. Soundararajan, et al. (2017) implemented an ANN model (feed forward back) and full factorial design to measure the influence of operating variables on mechanical properties of A413+B4C composites produced by squeeze casting. The designed ANN was found in the good agreement with the experimental data having mean-correlation and percentage error equal to 0.95 and 0.87, respectively [72]. Tirian, et al. (2014) presented an ANN-based technique to predict crack formation designed to improve the casting process of steel. The output of the ANN was used as an input in fuzzy decision system considered as decision block. The proposed system was found capable enough to estimate crack formation for the given operating conditions, moreover, it can be implemented in any system with minimum adaption [73]. Yu, et al. (2009) developed an ANN (three-layer back propagation) to predict the performance of porous Si3N4 ceramics. In ANN, the effects of compositions on the performance of porous Si3N4 ceramics were taken into account. By using the ANN model, it was found that the approach presented the encouraging predictions for the performances of porous Si3N4 ceramics, which shows that the ANN can be used as a very useful tool to analyze the performance of Si3N4 [74].
The ANN tool has been extended to additive manufacturing applications. Yadav, et al. (2020) implemented an ANN model for the multi-material (50% Acrylonitrile Butadiene Styrene + 50% Polyethylene Terephthalate Glycol) 3D printing by fused deposition modeling (FDM). The density and extrusion temperature were optimized to improve the tensile strength of FDM-ed parts. The results show that ANN has the capability to maximized tensile strength up to 4.54% [75]. Hu, et al. (2019) employed ANN in stereolithography (SLA) technique. Three main operating parameters in SLA including orientation (0~90°), post curing time (20~60 min), and annealing time (0~4 hrs.) were used as input to build and train the ANN model with the emphasis on hardness as output. This study proves that the hardness of the SLA-ed objects can be optimized up to 1% error using ANN method [76].
Following potential challenges have been identified on the basis of literature survey.
The performance of an ANN model depends on the quantity and type of the data provided while training. Therefore, an ANN is a data-driven model. On the other hand, it is expensive and time consuming to collect and organize the data for the training of an ANN model. There are certain methods available that can enlarge a dataset in an artificial and one of the ways to do it is to use artificial generative models [77]. Kingma and Welling (2014) explained that the autoencoder has a capability to generate randomly new data sets, by keeping view the training data, the extension of this autoencoder is known as variational auto-encoder [78]. Other generative models, such as generative adversarial nets and adversarial autoencoders, can also provide ways to perform data augmentation [79,80].
The training of an ANN model majorly depends on the input parameters, while the operating parameters play a significant role in any process. A selection of excess number of input parameters can easily overfit the ANN model. Therefore, it is of great importance to confirm that the ANN model is trained and operating at optimum number of input parameters. There are two techniques available, given in Table 4, for pre-processing of given to find out the significant input parameters.
The main objective of an ANN model is a good estimation of output on the basis of previously given input data. But this performance can be disturbed due to the over- or under-fitting of an ANN model on the basis of given input data. An overfit ANN algorithm means a model tries to fit itself on every data point within the given dataset, the model becomes vulnerable to noises within the dataset. In contrary, an underfit model means that the model has failed to dig-out the mandatory relationships among the data points in the training dataset. These problems can be avoided by selecting the optimum number of neurons within each layer [82,83].
In order to avoid lots of experimentations and producing a pool of datasets for the optimization of ANN model, is to develop analytical or numerical simulations. The results of analytical simulations can be used to train the ANN models. Koeppe, et al. (2018) proposed a methodology that combined the experiments, Finite Element (FE) simulations and ANN. Initially, the experimental analyses the FE-simulations were conducted to validate FE-simulations. Followed on, 85 simulational analyses were carried out using various parametric combinations such as load, displacement, strut radius and cell scale (input), while the maximum Von-Mises and principal stresses were categorized as outputs. After training conducted for ANN model, a close correlation for the loading history was found between an ANN estimations and FEM simulations [84].
This paper provides a comprehensive overview of the applications of ANN in various fields of life. Various researches have been carried, which proved the robustness and trustworthiness of ANN as potential candidate to estimate output on the basis of given inputs. The scope of this work covers many variants of ANNs in different scenarios including: Resource management and allocation, scheduling for maintenance and repairing, quality control and process planning, database control and supervision, robotics, simulations and machine vision, and manufacturing operations. However, the behavior of output strongly depends on the data provided to ANN for its training (input) and the number of neurons required in each layer (input-hidden-output). On the basis of the above fact, the problems encountered in ANN and their solutions are provided in current study. However, there are still more studies and investigations to be done to improve the application of ANN and to achieve the numerous benefits.