ERP success prediction: An artificial neural network approach

Abstract

The Enterprise Resource Planning system (ERP) has been pointed out as a new information systems paradigm. However, achieving a proper level of ERP success relies on a variety of factors that are related to an organization or project environment. In this paper, the idea of predicting ERP post-implementation success based on organizational profiles has been discussed. As with the need to create the expectations of organizations of ERP, an expert system was developed by exploiting the Artificial Neural Network (ANN) method to articulate the relationships between some organizational factors and ERP success. The expert system role is in preparation to obtain data from the new enterprises that wish to implement ERP, and to predict the probable system success level. To this end, factors of organizational profiles are recognized and an ANN model is developed. Then, they are validated with 171 surveyed data obtained from Middle East-located enterprises that experienced ERP. The trained expert system predicts, with an average correlation coefficient of 0.744, which is respectively high, and supports the idea of dependency of ERP success on organizational profiles. Besides, a total correct classification rate of 0.685 indicates good prediction power, which can help firms predict ERP success before system implementation.

Keywords

Enterprise resource planning (ERP) ; ERP success ; Organizational profiles/ factors ; Artificial neural network (ANN) ; Expert system

1. Introduction

Enterprise Resource Planning (ERP) systems are defined as single software systems allowing the complete integration of information flow from all functional areas in companies by means of a single database, such a system is accessible through a unified interface of communication  [1] . These systems have been increasingly adopted by organizations across various industries in both developed and developing countries. Organizations implement ERPs to enhance both operational efficiency and business efficacy  [2] , [3]  and [4] . They improve operational efficiency by integrating business processes and by providing better access to integrated data across the entire enterprise, while a company that wishes to enhance its efficacy may redesign its business practices by using the best practices embedded in the ERP  [1]  and [5] . Although ERP systems can bring about benefits such as competitive advantage, decision support capabilities and business intelligence competences (e.g.  [6] , [7]  and [8] ), the high failure rate is a major concern  [1] . The failure rate of ERP implementation has been estimated at between 60% and 90%  [9] . These projects are, on average, 178% over budget, take 2.5 times longer than intended and deliver only 30% of the promised benefit  [10] . Other research findings indicate that organizations do not always achieve their desired level from their ERP investments  [6]  and [11] .

These statistics imply that ERP projects are one of the most difficult system development projects. They are quite complex projects and often require fundamental organizational changes. To avoid such costly failures and help organizations take more advantage of their system implementation, a great deal of effort has been made by researchers. Some researchers have provided valuable insight into the process of ERP implementation [12] , [13] , [14] , [15]  and [16] and others have identified a variety of critical factors influencing the success or failure of the system  [10] , [17]  and [18] . Understanding the nature of Enterprise Systems (ESs) and ERPs success has also been the focus of scholarly research interest in recent years  [10] , [19] , [20] , [21] , [22] , [23] , [24]  and [25] .

Despite these studies, there is little research on measuring the ERP success/failure possibility except for the work conducted by Chang et al.  [26] in which they proposed a framework based on incomplete linguistic preference relations under a multi-criteria decision making environment to measure the success/failure possibility of initiating an ERP system. They presented a model in which three technological, environmental and organizational aspects are associated with the post-implementation success of ERP. Although their work extended the current studies on ERP from the implementation stage to the post-implementation stage, we believe that their work has limitations, in terms of the number of both factors and cases investigated. In another related work, Tsai et al.  [27] developed a framework for investigating how ERP selection criteria are linked to system quality and the service provided by suppliers and consultants, and, thus, how these influenced ERP implementation success and system net benefits. Their results implied that the system, supplier and consultant selection criteria had positive influences on net benefits through the service quality process and user perspective. However, they have not taken organizational circumstances into account in their analysis.

In this paper, therefore, with adopting a more holistic and comprehensive view of some of the organizational profiles (such as industry type, size, structure, management style, IT systems and so on), it has been attempted to point out the relationships between some organizational profiles and ERP system success. Meanwhile, the paper has tried to answer the following research question:

• How can organizational profiles be related to ERP success?

The response to the above research question would create a dynamic model to help firms determine the desired success level in ERP implementation and supply the needed organizational characteristics. This means that an ERP success prediction instrument and model would serve as a facilitator to cast light on ERP preparation plans. Since this model illuminates the relationships between the desired success level and the needed organizational characteristics, it can provide proper justification for preparation plans.

Generally, when ERPs are being taken into consideration, there are numerous parameters and factors affecting its success. So, it sounds difficult to answer this question by developing a precise mathematical model for eliciting the relationships. Meanwhile, another approach is required to identify such hidden patterns. Due to such complexity, the proposed model should have classification and forecasting capabilities, which makes Artificial Neural Networks (ANNs) an appropriate technique to apply to expert systems, as is now commonly used in the literature  [28] , [29]  and [30] .

The objective of this research, consequently, is to classify a set of organizational parameters and factors, on the one hand, and a set of ERP success factors, on the other hand, and then develop and design the ANN-based expert system in order to predict and recommend the probable ERP success level. Validation of the expert system is also done by the authors.

The paper is organized as follows: Section  2 presents literature reviews on ERP success, organizational profiles and ANN-based expert systems. Section  3 describes the proposed research method and ANN architecture. Section  4 discusses the result of ANN training, validation and testing, as well as expert system implementation. Finally, Section  5 , the conclusion, offers the research findings.

2. Literature review

The three basic elements of this research are: the ERP post-implementation success, the organizational profiles, and the ANN-based expert system. In the following, these elements will be described with respect to the most relevant literature.

2.1. ERP post-implementation success

ERP has been stated as the new Information System (ISs) paradigm  [31] and ERP success models draw upon information systems success evaluation patterns. The most famous model that emerges from IS successful research projects is the Delone and McLean  [19] model (D&M model). Delone and McLean  [19] introduced their framework as a review of 180 empirical studies in 1992. This model consists of six major surrogates, including “system quality”, “information quality”, “use”, “user satisfaction”, “individual impact” and “organizational impact”. Following the Delone and Mclean  [19] model, Seddon  [32] published an extended version of the D&M model by separating two sub-models (use and success) and eliminating the process model interpretation. Later on, Delone and McLean  [20] revised their model and replaced individual impact and organizational impact with net benefit. Meanwhile ERP success assessment models were presented by some researchers.

One of the most accredited models to have been presented in recent years is the Markus and Tanis  [33] model. Markus and Tanis argued that the definition and measurement of ERP success depends on the viewpoint from which you measure it. They pointed out that the ERP implementation cycle could be divided into three distinct phases: project phase, shakedown phase, and onward and upward phase. They showed that success in ERP implementation was derived from an incremental process and could not be acquired overnight. Next, Tan and Pan  [34] presented their model. They believed that in the model proposed by Markus and Tanis, little attention had been paid to the softer side of ERP systems. So, they offered a new model that defined ERP success as an “infrastructure success”, “info structure success” and “knowledge success”. This model combines both sides of technical and strategic values in ERP post-implementation success assessment.

Another prominent research in this area is the Gable et al. [22] model that has been validated and widely considered in the literature  [23] , and which grew out of the Delone and McLean  [19] model. However, Gable et al.  [22] asserted that several measures in the D&M model were redundant and inappropriate to measure the success of ERP systems. Furthermore, they concluded that user satisfaction was not a distinct surrogate and also removed user satisfaction from their final model. They also pointed out that D&M measures of organizational impacts focus on financial ones and do not cover business process improvement and organizational change. To overcome the shortcomings in the previous models, Gable et al.  [22] proposed their model in four surrogates, including “system quality”, “information quality”, “individual impact”, and “organizational impact”, which was also considered a base for succeeding models.

A year after Gable et al.  [22] , Zhang et al.  [10] presented a model based on Ives et al.’s  [35] information systems research and the Delone and McLean  [19] model. They developed a conceptual research framework to guide the relationship between the organizational environment, the user environment, the ERP system environment, the ERP vendor environment and ERP post-implementation success. Nevertheless, the Zhang et al.  [10] model has been recognized more as a guidance framework to implement the ERP system successfully rather than as a practical model to measure ERP post-implementation success. So, it attracted less attention than the Gable et al.  [22] model from succeeding researchers.

Drawing upon the Gable et al.  [22] model, Ifinedo and Nahar  [23] tried to develop their own model. They continued their work by asking this question: “Is the Gable et al.  [22] ERP success measurement model comprehensive? If not, can the model be extended to incorporate other relevant surrogates of success?” Hence, Ifinedo and Nahar  [23] added workgroup impact and vendor and consultant quality to the model surrogates. Later on, Ifinedo et al.  [24] reorganized his model by replacing vendor and consultant quality surrogates with service quality. Service quality refers to the support that the organization receives from the ERP provider, often operationalized by reliability, dependability, quality of expertise, and so forth. Finally, Moalagh and Zare Ravasan  [36] proposed their model, which strives to highlight the importance of considering the nonlinear relationships among factors and sub-factors, as well as unique organizational goals, in ERP success assessment. They used a fuzzy ANP method for weighting the Ifinedo and Nahar  [23] model surrogates and sub-factors.

In this paper, the model of Ifinedo and Nahar  [23] has been employed. This model is composed of six main surrogates as “system quality”, “information quality”, “vendor quality”, “individual impact”, “workgroup impact”, and finally “organizational impact” with an overall 30 sub-factors for the assessment, as displayed in Table 1 .

Table 1. Six main surrogates of ERP success assessment model  [26] .

Systems Quality (SQ)	Data accuracy
	Easy to learn
	Good features
	Data integration
	Efficiency
Information Quality (IQ)	Timely information
	Important information
	Relevant information
	Usable information
	Available information
Vendor Quality (VQ)	Adequate technical support
	Credibility and trustworthiness
	Good relationships
	Experience
	Good communication
Individual Impact (II)	Organizational learning
	Improving individual productivity
	Benefits for individual’s tasks
	Higher-quality decision making
	Saving time
Workgroup Impact (WI)	Improved workers’ participation
	Improved organizational-wide communication
	Creating a sense of responsibility
	Improved efficiency of sub-units
	Solution effectiveness
Organizational Impact (OI)	Competitive advantage
	Customer service/satisfaction
	Facilitating business process change
	Supporting decision making
	Better use of organizational data resource

2.2. Organizational profiles

Razmi et al.  [37] and Hanafizadeh and Zare Ravasan  [38] proposed that the success of ERP projects can be enhanced by running ERP readiness assessment at the initial stages of an ERP project. They also offered some factors for such assessments. As there is a need to gather organizational profile data before system implementation, we believe that such ERP readiness assessment factors can be employed for our purpose. Hence, here we exploited the factors and assessment questionnaire proposed by Hanafizadeh and Zare Ravasan  [38] , as their framework seems to be the most comprehensive and applicable ERP readiness assessment framework currently available in the literature. Their framework draws upon the McKinsey 7S model comprised of 7 dimensions, namely, structure, strategy, systems, skills, style, staff, and shared values, and also 3 assessment factors in each main dimension, as summarized briefly in Table 2 .

Table 2. ERP readiness assessment dimensions and factors  [38] .

Dimensions/factor	Description
Strategy
Vision and mission	Clear vision statement and adequate business plan; justification for the ERP investment; feasibility-evaluation of ERP project; well understood vision and mission across the organization.
Goals/objectives	Conceptualization and justification of the goals and possible ways to fulfill them; clearly defined critical business needs and the business values of the system; clear and measurable and well-understood goals across the organization; clearly defined scope of the ERP project.
Strategic IT plans	Effective strategic thinking; alignment of IS planning and business planning; continuous and up to date strategic IT plans; top management involvement in strategic IT plans.
Structure
Formalization	Standardization of work processes and documentation; clearly documented rules and procedures and made known to all employees.
Size	Size of IT departments; availability of resources; employee workforce; and/or annual turnover/sales.
CIO position	Empowered CIO in the organization; CIO reporting directly to the CEO; strategic rather than supportive role of the CIO in the organization.
Systems
IT Infrastructure	Adequate IT infrastructure; legacy systems; suitability of hardware and software; technology or infrastructure in place; integration and communication between legacy system and ERP.
Business processes	BPR; business process reengineering; business process change; business process improvement; optimization; and reengineering; alignment of the business with the new system; minimum customization.
Data	Availability and timeliness of accurate data; finding the proper data to load into the system; data analysis and conversion; data quality controls; educating users on the importance of data accuracy and correct data entry procedures.
Style
Organizational culture	Learning and development culture; participative decision making culture; power sharing culture; support and collaboration culture; and tolerance of conflicts and risk culture
Top management support	Top management/executive involvement; top management/executive commitment; top management/executive awareness; top management/executive participation; dedicated resources; funds support.
Communication	Clear and effective communication plan; open and honest communication among the stakeholders; free flow of information; communication plan for all stages of the project including project goals and tasks.
Staff
Human resource management	The ability of an organization to recruit; select; place; appraise and develop appropriate employees; proper mechanisms to recruit and preserve qualified employees; and nurture and maintain a high level of employees’ morale and motivation among them; having younger and more educated staff.
Training and education	Project team training; user training; training of business practices and processes; as well as and ERP skills; allocated money and time resources to training; ERP training plan; well-documented education and training strategy; effective training.
Project team	Balanced; cooperative; cross functional; and the full time basis project team member; project team empowerment; project team competence; project team’s prior experience in large IT projects; the domain knowledge of the ERP project team; teamwork participation; attitude of the ERP project team.
Skills
Management’s skills	Communication skills; controlling skills; leadership skills; planning skills; IT management skills.
IT staff’s skills	Communication skills; IT management skills; planning skills; technical skill; ERP experience.
Users’ skills	Communication skills; interpersonal skills; planning skills; technical skills; ERP experience.
Shared values
Project champion	Project leader expertise; strong and committed leadership; high level official in the organization; continually manage resistance and change during the implementation; strong leadership skills and business; technical; personal; as well as managerial competencies.
Company-wide commitment	Company-wide support; companywide commitment; support from all functional segments of an organization; commitment and cooperation of personnel from all segments of the business; personnel involvement.
Shared beliefs	The shared belief with employees and managers regarding the benefits of the ERP system; a shared understanding of why a technology is being implemented.

As can be seen in Table 2 , the aforementioned factors are mainly associated with ERP project factors. Besides, other factors might also contribute to achieving ERP success, such as industry type. O’Leary  [39] suggested that any analysis of ERP benefits which does not allow differentiation by industry could be seen as inherently limited and benefits cannot be seen as universally equivalent across industries. Similarly, ERP system life and the implementation strategy as suggested by Mabert et al.  [40] , and Olhager and Selldin  [41] have been adapted here.

It should be noted that the above factors should be evaluated at the beginning of the ERP project during the adoption process. Therefore, they reflect the potential of organizations for success in the following ERP mission.

2.3. ANN-based expert system

Expert systems are software-intensive systems that combine the expertise of one or more experts in a specific decision area, in order to provide specific advice to a problem, which supports the user in making a better decision. An expert system is a combination of system and processes designed to imitate the judgment of experts  [42] . An expert system is also defined as the computer program that emulates the behavior of human experts in a well-specified manner, and narrowly defines the domain of knowledge  [28] . It captures the knowledge and heuristics that an expert employs in a specific task  [43] . The prevalent type of these systems would be developed based on the Artificial Neural Network (ANN), which gets help in its inference computation from the capabilities of ANN classifications.

Artificial neural networks are computational modeling tools that have been widely utilized in many fields to model, analyze and solve complex real-world problems. ANNs may be classified as structures embracing densely interconnected adaptive simple processing elements (called artificial neurons) that are capable of carrying out massive parallel computations for data processing and knowledge representation  [44] . Although Biological Neural Nets are the naturally occurring equivalent of the ANNs, the idea of ANNs is not to replicate the process of the biological systems, but to exploit what is known about the functionality of the biological neurons to solve compound and complex problems. Neural networks discover patterns and relationships in huge amounts of data by using machinery computations that simulate the processing patterns of the human brain. Contrary to the usual and traditional computing techniques, ANNs are ‘trained’ to generate the desired input–output relationships. Throughout the training phase, examples of data are presented to the artificial network and, by using a learning algorithm, the parameters are tuned to adjust the network behavior  [45] .

Given the available knowledge of the problem and the objective of the operator, the learning procedure employed can be either ‘supervised’, ‘unsupervised’ or both. The supervised learning method is performed with pairs of known input–output training data, whereas in unsupervised learning, training examples are presented to the network input while the network organizes itself progressively to reach maximal separation between the naturally occurring classes of cases  [45] .

A significant advantage of neural networks (referred to as generalization) is that when they are properly trained, they can appropriately process data that has not been used for training or validation. The trendiest neural networks are multi-layer perceptions, which are generally supervised–trained with the error back-propagation algorithm  [46] . Besides, the radial basis function (RBF) network is a particular class of multi-layer network  [47] , where learning occurs usually in two steps: learning in the hidden layer (usually by an unsupervised bottom–up self-organizing method such as K-means clustering) followed by the output layer (a top–down supervised method such as the least squares estimation). Another popular class of network is the self-organizing map, or Kohonen network  [48] . A Kohonen network consists of two fully connected-unit layers and the output layer is normally ordered in a low-dimensional structure of units. The objective of this network is to build a map, where units of an area are activated when inputs with similar characteristics are presented. Among other popular networks are Adaptive Resonance Theory (ART) networks, and their derivates (ART1, ART2, fuzzy ART etc.)  [49] , as well as Hopfield models  [50] .

The main specific feature of neural networks is the fact that they can be considered nonlinear statistical methods. Nevertheless, a large amount of data is required to overcome the existing nonlinearity in the data structure. Therefore, the attractiveness of ANNs comes from the remarkable information processing characteristics of a biological system, such as nonlinearity, high parallelism, robustness, fault and failure tolerance, learning, ability to handle imprecise and fuzzy information, and their capability to generalize  [45] . Once the ANN model has been trained on given data, they are capable of carrying out accurate predictions of future events. ANNs have been successfully used in predicting corporate failures and organizational improvement strategies, and have been applied to a number of settings, including engineering and medicine  [51]  and [52] .

3. Research method

The research steps, including identifying variables, survey instrument designing, data collection, ANN architecture, network training and system implementation are shown in Figure 1 . These steps are described as below:

Figure 1. The steps of research method.

3.1. Identifying variables

In the first step of the research procedure, input and output variables should be identified for conducting research, surveying and training ANN. In this research, the primary variables were defined according to the literature, based on the research purpose and our research assumptions. Therefore, the 24 (21 variables from Table 2 and three extra variables from other references) were identified as input variables. On the other hand, the success of ERP implementation could be measured by the Ifinedo and Nahar  [23] model. Consequently, 30 variables (Table 1 ) were identified as output variables. In other words, the research independent variables are organizational profiles (24 variables), and research dependent variables are ERP post-implementation success (30 variables). These variables were utilized in the research questionnaire and ANN architecture.

3.2. Survey instrument designing

As a research instrument, a questionnaire was developed and structured in three sections (see Appendix ). Information concerning the factors and profiles of the surveyed organizations was requested at the beginning of the questionnaire in the form of twenty four questions. In the second part, the 30 questions attempted to ask for the surveyed organizations attitudes, based on the ERP post-implementation success criteria listed in Table 1 . In other words, the second part of the questionnaire requested opinions about the perceived success of the ERP system after implementation. Except for industry type, other responses were evaluated on a 5-point “Likert scale”. Following these 54 questions, the third part of the questionnaire was designed to ask for organization name, contact information and the website address for more information.

The content validity of the questionnaire is guaranteed by utilizing the research questionnaires developed by Hanafizadeh and Zare Ravasan  [38] in organizational profiles, and also Ifinedo and Nahar  [23] in ERP success assessment. Also, the test–retest method was used to evaluate the reliability of the questionnaire. To conduct the test–retest method, authors asked eight Chief Information Officers (CIOs) during a 10-day interval to participate in the study. The resulted Cronbach’s alpha, estimated to be 0.86 (greater than 0.7), implies the high reliability of the instrument  [53] .

3.3. Data collection

One of the most important aspects in designing a supervised ANN is the data collection and preparation, thus, the cases or examples for training have to be representative of all the possibilities concerning the application. The research targets were CIOs (Chief Information Officers), IT managers and ERP project managers of Middle East firms that have ERP at least one year existing. This is due to the fact that the results of some studies suggest that the full effects of ERP adoption for firms do not surface until after a considerable time-lag [54]  and [55] . The data for this study was collected by a survey sent out via mail and e-mail from April to December 2011. The survey questionnaire, along with a cover letter, was sent to the CIO or IT manager of each firm. The letter served as a guide to fill out the questionnaire, as well as to highlight the research rationale.

About 340 surveys were sent to the firms. The returned questionnaires were 175, which showed a response rate of 51.47%. Four of the returned questionnaires were discarded because they were not completed, so, the number of valid questionnaires were reduced to 171, that is, the response rate reached 50.02%.

3.4. ANN architecture

The ANN models that are a huge success in classification problems are feed-forward multi-layer networks  [56] . Consequently, the neural network applied here is a multi-layer, feed-forward and fully connected neural network. This network is also comprised of three layers: input, hidden and output layers; therefore, it is a multi-layer network. Since each node is connected to all nodes of the other layers, it is called a fully connected network.

Considering the research variables, the architecture of the proposed ANN should be made up of two main layers, an input layer for organizational profiles and an output layer for ERP post-implementation success. Organizational profile variables are categorical 5-point Likert, except for the industry type of organization which has 10 categories. For each categorical value or state, we should have one binary node, and all binary nodes for one variable form a 0 and 1 vector. ERP post-implementation success variables are also categorical 5- point Likert to which five binary nodes were assigned for these variables too.

Thus, the input layer should have 125 ${\textstyle (23{_{\ast }}5+1{_{\ast }}10)}$ nodes (23 organizational profile variables and a 5 Likert state for each variable, and 10 categories for an industry type variable). The output layer should also have 150 ${\textstyle (30{_{\ast }}5)}$ nodes (30 success variables and a 5 Likert state for each variable).

Based on technical research on ANN, one hidden layer is sufficient for computing judgment boundaries for outputs  [57] . If the hidden layer has a sufficient number of neurons, the ANN can estimate any continuous function. There is no exact base for determining the number of neurons in the hidden layer. Parten et al.  [58] demonstrated that the bound of neurons in first hidden layer should be between ${\textstyle 2N+1}$ and OP (${\textstyle N+1}$ ), where ${\textstyle N}$ is the number of input variables and OP is the number of output variables. In the present research, the bound of neurons in the hidden layer is [251, 18 900]. Through some trials in the range, the final number of neurons in the hidden layer was set at 350. In Figure 2 , the ANN architecture is depicted.

Figure 2. The ANN architecture.

3.5. Network training (validation and test)

ANNs are fundamentally nonlinear models that distinguish patterns and classify variables. To do so, researchers have developed some supervised and unsupervised training methods. Dasgupta et al.  [59] found that the Back Propagation (BP) model and Levenberg–Marquardt algorithm of ANNs training outperform other models in classification. Therefore, a feed-forward BP model and Levenberg–Marquardt (trainlm) were employed in the current research. Besides, 171 records of data were gathered for this research. Out of these 171 records, 121 records were considered for the training set, 25 for the validation set and the remaining 25 for the test set. By utilizing the validation and test sets, it can be ensured that the network is generalized and will not be over-fitted. The selected activation functions are Tansig for the hidden layer and Purelin for the output layer; further, the training algorithm is Levenberg–Marquardt (trainlm). To check out the accuracy of the model, the Mean Squared Error (MSE) criterion was computed. MSE was evaluated using Eq. (1) . It is worth mentioning that the ANNs model achieves a better performance when MSE is small.

$MSE={\frac {\sum _{j=0}^{P}\sum _{i=0}^{N}(t_{ij}-y_{ij})^{2}}{NP}}{\mbox{,}}$

( 1)

where ${\textstyle p}$ is the number of output possessing elements and ${\textstyle N}$ is the number of exemplars in the dataset. ${\textstyle t_{ij}}$ and ${\textstyle y_{ij}}$ are desired output and network outputs, respectively.

3.6. System implementation

In order to make the current research applicable, an expert system was designed and developed in a VB.NET environment. This system has three functionalities: to predict the success level, to edit/view organizational profiles and to compare the predicted success of different firms. The engine of this expert system is the trained ANN, which can predict the ERP post-implementation success by asking the organizational profiles of a firm. That is, the inference engine of the developed expert system utilizes the knowledge of the trained ANN, which is based on the aggregation of survey data. An interface helps the users to declare their organizational profiles and create managerial reports for a possible success level if they wish to implement ERP. The system also has a database to store predictions for comparing success prediction and profiles of the firms. Figure 3 illustrates the expert system model that utilizes the trained ANN.

Figure 3. The developed expert system model.

4. Results

The ANN with the proposed architecture was trained by 171 data records; 121 records are considered for the training set, 25 for the validation set and 25 for the test set, to be exact. In the training phase, the training and validation sets are used together. The training method of ANN was Levenberg–Marquardt; in this algorithm, the connection weight of neurons is justified by the decreasing measure of error between network outputs and real or observed outputs. As mentioned before, to examine the accuracy of different models, MSE is computed in each epoch. The optimum trained network will be the point at which the MSE of the validation set is below the MSE of the training set, and by using the validation and test sets, it can be ensured that the network is generalized and will not be over-fitted.

As shown in Figure 4 , the MSE value drops when the iterations of training rise. It confirms the fact that the neural network is appropriately trained. MSE training starts at 0.6350 in the first epoch and converges at 0.0109 in the final epoch.

Figure 4. Comparison of training and validation MSE in convergence.

After training the network, a test data set was used to evaluate the accuracy of the ANN. The test aims at assessing the accuracy of the results predicted by the system and the real success levels that were gained in the cases. The system performance evaluation for the test set is illustrated in Table 3 .

Table 3. The expert system performance evaluation for test set.

Cases	C1	C2	C3	C4	C5	C6	C7	C8	C9	C10	C11	C12	C13
Number of CP	22	18	26	28	24	21	27	12	19	17	19	21	17
CCR	0.733	0.600	0.867	0.933	0.800	0.700	0.900	0.400	0.633	0.567	0.633	0.700	0.567
MSE	0.01	0.02	0.00	0.00	0.01	0.01	0.00	0.03	0.01	0.02	0.01	0.00	0.01
r	0.91	0.86	0.97	0.98	0.98	0.92	0.98	−0.08	0.97	0.84	0.92	0.98	0.35
Cases	C14	C15	C16	C17	C18	C19	C20	C21	C22	C23	C24	C25	Ave
Number of CP	12	18	15	19	26	25	17	19	23	29	19	21	20.56
CCR	0.400	0.600	0.500	0.633	0.867	0.833	0.567	0.633	0.767	0.967	0.633	0.700	0.6853
MSE	0.03	0.01	0.02	0.02	0.00	0.01	0.00	0.01	0.00	0.00	0.02	0.01	0.0104
r	−0.04	0.74	0.63	0.69	0.97	0.87	0.89	0.72	0.92	0.92	−0.01	0.72	0.7440

For example, in case 1 of the test set, the number of correct predictions (CP) was 22 out of 30 success variables, i.e. a correct classification rate (CCR) of 0.733. In this case, the MSE between predicted neurons and real benefits neurons was 0.01. In addition, the correlation coefficients between the actual outputs and the network outputs for the test data sets were calculated. If the network performance is high, the correlation coefficients between the actual outputs and the network outputs should take values that are very close to one. For case 1, the correlation coefficient (${\textstyle r}$ ) between the actual output and the network output was 0.91.

Analyzing all 25 test data sets, the expert system predicted 20.56 benefits correctly, on average. The prediction power of the expert system or total correct classification rate was 0.685 in total. Besides, the average correlation coefficient for the test data was 0.744, which is relatively high and supports the idea of dependency of ERP post-implementation benefits on organizational profiles.

5. Conclusion

Given their many established benefits, the ERP systems have been adopted by many firms, even in less developed countries. But, one of the crucial issues in achieving ERP system success and, thus, deriving benefits, is the nature and profile of organizational factors in addition to implementation issues. Although researchers have proposed a variety of approaches to analyze the relationships between some organizational factors and ERP system success, no one has ever used the capability of ANNs in articulating such relationships. To bridge the gap, this research attempted to link the organizational profiles to the ERP success level. To this end, using the available models, organizational factors/profiles and ERP success were identified and classified in 24 and 30 variables, respectively. Then, using questionnaires, the variables data were gathered through Middle-East located firms which have implemented ERPs.

After data collection, these records were used to train ANNs with the aim of predicting the ERP success level based on organizational profiles. The multi-layer feed-forward architecture, with 125 input neurons (organizational profiles) and 150 output neurons (ERP success), were manipulated to design the network. The Levenberg–Marquardt training algorithm, with the MSE measure, was also selected for training and validation. To insure the accuracy of the network, a 25 test set record was used to test the network. With the analysis of all 25 test data sets, the expert system predicted 20.56 out of 30 success variables correctly, on average. The prediction power of the expert system or total correct classification rate was 0.685 in total. Besides, the average correlation coefficient for the test data was 0.744, which is relatively high and supports the idea of dependency of ERP post-implementation success on the organizational profiles.

The major contributions of this research are as follows: First, this paper illustrates the idea of predicting the ERP success level based on the organizational profiles before system implementation. Second, an ANN has been utilized to train a system to predict the ERP success level for a firm which would like to implement the ERP based on their organizational factor. Third, the trained ANN was embedded in the VB.net developed expert system to make it applicable for firms to predict their achievable ERP success level, based on their organizational factors and profiles, and make simpler decisions. Fourth, as an inherent limitation of the Ifinedo and Nahar  [23] success assessment, their framework categories do not take the time frame into account for benefits, while, as noted by Davenport  [60] , there are different types of benefits of the ERP system, and some are likely to arise earlier than others, which are addressed in this research by adding ERP system life in interpreting system benefits.

This research and its findings (expert system) come in handy and can apply to three major audience groups; the first group is comprised of IT managers and CIOs who wish to examine whether or not their firms, with their profiles, achieve the desired and expected values and successes via ERP implementation. The second group embodies consultants who would like to convince and encourage top managers to adopt the ERP project with a precise view in mind and assert them with realistic expectations. The third group constitutes the ERP project managers who wish to know the ERP readiness situation and prepare readiness plans based on their organizational profiles.

ERP implementation takes time and its benefits are not derived immediately. So, the predicted success values are calculated to enable organizations to decide whether to initiate the ERP project, inhibit adoption, or take corrective actions to increase the possibility of a successful ERP. It is obvious that if there were any improvement plans, the status of the organizational profile would be improved and the levels should be re-evaluated using the proposed instrument after a few months. Moreover, the authors believe that the proposed model and the developed expert system can assist firms in predicting the likely ERP success level each year after implementation, and create their expectations from the system, which can be significantly useful in project expectation management. In conclusion, the results of the paper can help organizations to learn how to boost their ERP success level by improving their organizational factors and to observe the effect of their organizational factors on the ERP success level after implementation.

One of the main limitations of the work is the low degree of ERP penetration in organizations, to date, in Middle-East countries. This limitation not only made data collection a tremendous effort for the authors, but also might have lessened model prediction accuracy due to the limited number of cases surveyed. Another limitation of the study was that the model presented here does not consider all possible factors and parameters that might be associated with ERP success level acquisition as utilized in the Hanafizadeh and Zare Ravasan  [38] framework. However, this limitation exists on the other side of the model, in which the Ifinedo and Nahar  [23] model has been utilized, as it does not link the success variables to the reasons for ERP implementation, which might have a negative effect on the application of the proposed model in practice. As the model suffers from these limitations, further research may address these limitations by extending the main factors of this model by adding new ones. Likewise, it could be interesting for researchers to compare the performance of the ANN approach with other meta-heuristics (genetic algorithm, fuzzy neural network, ant colony, etc.) or other methods (linear/nonlinear regression, system dynamics) to particularly examine whether each approach has any superiority over others in solving such problems.

Acknowledgments

The authors would like to thank the Editor and anonymous referees of the Scientia Iranica journal for their useful comments to enhance the quality of this paper.

Appendix. Survey instrument

• What type is your industry?
• Automobile
• Electronics
• Food
• Information technology
• Logistics/Courier services
• Manufacturing
• Pharmaceutical
• Telecommunications
• Utility and Oil/Gas
• Wholesale/Retail.
• What was your implementation strategy?
• Single ERP package
• Single ERP package with other systems
• Multiple ERP packages with other systems
• Best-of-breed from several ERP packages
• Totally in-house developed.
• How old is your ERP system?
• 1–3 years
• 3–5 years
• 5–7 years
• 7–10 years
• More than 10 years.
• What was the overall state of organizational profiles/factors in your organization before system implementation in terms of details given in Table A.1 .

Table A.1. Organizational profile questionnaire.

Organizational profiles/factors		Overall statea
1	Vision and mission (Clear vision statement and adequate business plan; justification for the ERP investment; feasibility-evaluation of ERP project; well understood vision and mission across the organization.)	VL	L	M	H	VH
2	Goals/objectives (Conceptualization and justification of the goals and possible ways to fulfill them; clearly defined critical business needs and the business values of the system; clear and measurable and well-understood goals across the organization; clearly defined scope of the ERP project.)	VL	L	M	H	VH
3	Strategic IT plans (Effective strategic thinking; alignment of IS planning and business planning; continuous and up to date strategic IT plans; top management involvement in strategic IT plans.)	VL	L	M	H	VH
4	Formalization (Standardization of work processes and documentation; clearly documented rules and procedures and made known to all employees.)	VL	L	M	H	VH
5	Size (Size of IT departments; availability of resources; employee workforce; and/or annual turnover/sales.)	VL	L	M	H	VH
6	CIO position (Empowered CIO in the organization; CIO reporting directly to the CEO; strategic rather than supportive role of the CIO in the organization.)	VL	L	M	H	VH
7	IT Infrastructure (Adequate IT infrastructure; legacy systems; suitability of hardware and software; technology or infrastructure in place; integration and communication between legacy system and ERP.)	VL	L	M	H	VH
8	Business processes (BPR; business process reengineering; business process change; business process improvement; optimization; and reengineering; alignment of the business with the new system; minimum customization.)	VL	L	M	H	VH
9	Data (Availability and timeliness of accurate data; finding the proper data to load into the system; data analysis and conversion; data quality controls; educating users on the importance of data accuracy and correct data entry procedures.)	VL	L	M	H	VH
10	Organizational culture (Learning and development culture; participative decision making culture; power sharing culture; support and collaboration culture; and tolerance of conflicts and risk culture.)	VL	L	M	H	VH
11	Top management support (Top management/executive involvement; top management/executive commitment; top management/executive awareness; top management/executive participation; dedicated resources; funds support.)	VL	L	M	H	VH
12	Communication (Clear and effective communication plan; open and honest communication among the stakeholders; free flow of information; communication plan for all stages of the project including project goals and tasks.)	VL	L	M	H	VH
13	Human resource management (The ability of an organization to recruit; select; place; appraise and develop appropriate employees; proper mechanisms to recruit and preserve qualified employees; and nurture and maintain a high level of employees’ morale and motivation among them; having younger and more educated staff.)	VL	L	M	H	VH
14	Training and education (Project team training; user training; training of business practices and processes; as well as and ERP skills; allocated money and time resources to training; ERP training plan; well-documented education and training strategy; effective training.)	VL	L	M	H	VH
15	Project team (Balanced; cooperative; cross functional; and the full time basis project team member; project team empowerment; project team competence; project team’s prior experience in large IT projects; the domain knowledge of the ERP project team; teamwork participation; attitude of the ERP project team.)	VL	L	M	H	VH
16	Management’s skills (Communication skills; controlling skills; leadership skills; planning skills; IT management skills.)	VL	L	M	H	VH
17	IT staff’s skills (Communication skills; IT management skills; planning skills; technical skill; ERP experience.)	VL	L	M	H	VH
18	Users’ skills (Communication skills; interpersonal skills; planning skills; technical skills; ERP experience.)	VL	L	M	H	VH
19	Project champion (Project leader expertise; strong and committed leadership; high level official in the organization; continually manage resistance and change during the implementation; strong leadership skills and business; technical; personal; as well as managerial competencies.)	VL	L	M	H	VH
20	Company-wide commitment (Company-wide support; companywide commitment; support from all functional segments of an organization; commitment and cooperation of personnel from all segments of the business; personnel involvement.)	VL	L	M	H	VH
21	Shared beliefs (The shared belief with employees and managers regarding the benefits of the ERP system; a shared understanding of why a technology is being implemented.)	VL	L	M	H	VH

a. VL = Very Low; L = Low; M = Moderate; H = High; VH = Very High.

• What is the level of ERP success achieved in your organization in terms of:

ERP success	Achieved level
Data accuracy	VL	L	M	H	VH
Easy to learn	VL	L	M	H	VH
Good features	VL	L	M	H	VH
Data integration	VL	L	M	H	VH
Efficiency	VL	L	M	H	VH
Timely information	VL	L	M	H	VH
Important information	VL	L	M	H	VH
Relevant information	VL	L	M	H	VH
Usable information	VL	L	M	H	VH
Available information	VL	L	M	H	VH
Adequate technical support	VL	L	M	H	VH
Credibility and trustworthiness	VL	L	M	H	VH
Good relationships	VL	L	M	H	VH
Experience	VL	L	M	H	VH
Good communication	VL	L	M	H	VH
Organizational learning	VL	L	M	H	VH
Improving individual productivity	VL	L	M	H	VH
Benefits for individual’s tasks	VL	L	M	H	VH
Higher-quality decision making	VL	L	M	H	VH
Saving time	VL	L	M	H	VH
Improved workers’ participation	VL	L	M	H	VH
Improved organizational-wide communication	VL	L	M	H	VH
Creating a sense of responsibility	VL	L	M	H	VH
Improved efficiency of sub-units	VL	L	M	H	VH
Solution effectiveness	VL	L	M	H	VH
Competitive advantage	VL	L	M	H	VH
Customer service/satisfaction	VL	L	M	H	VH
Facilitating business process change	VL	L	M	H	VH
Supporting decision making	VL	L	M	H	VH
Better use of organizational data resource	VL	L	M	H	VH

References

1. [1] T. Davenport; Putting the enterprise into the enterprise system; Harvard Business Review, 76 (4) (1998), pp. 1–12
2. [2] E. Wang, J. Chen; Effects of internal support and consultant quality on the consulting process and ERP system quality; Decision Support Systems, 42 (2) (2006), pp. 1029–1041
3. [3] T.F. Gattiker, D.L. Goodhue; What happens after ERP implementation: understanding the impact of interdependence and differentiation on plant-level outcomes; MIS Quarterly, 29 (3) (2005), pp. 559–585
4. [4] W. Ke, K.K. Wei; Organizational culture and leadership in ERP implementation; Decision Support Systems, 45 (2) (2008), pp. 208–218
5. [5] S.W. Chou, Y.C. Chang; The implementation factors that influence the ERP (enterprise resource planning) benefits; Decision Support Systems, 46 (1) (2008), pp. 149–157
6. [6] McDonald, M.P. “Executive summary: meeting the challenge: the 2009 CIO agenda”, Gartner Executive Program Report, 2009.
7. [7] M. Ghazanfari, M. Jafari, S. Rouhani; A tool to evaluate the business intelligence of enterprise systems; Scientia Iranica, 18 (6) (2011), pp. 1579–1590
8. [8] S. Rouhani, M. Ghazanfari, M. Jafari; Evaluation model of business intelligence for enterprise systems using fuzzy TOPSIS; Expert Systems with Applications, 39 (3) (2011), pp. 3764–3771
9. [9] K.Y. Kwahk, J.N. Lee; The role of readiness for change in ERP implementation: theoretical bases and empirical validation; Information and Management, 45 (7) (2008), pp. 474–481
10. [10] Z. Zhang, et al.; A framework of ERP systems implementation success in China: an empirical study; International Journal of Production Economics, 98 (1) (2005), pp. 56–80
11. [11] CSC; Tenth Annual Technology Issues for Financial Executives Survey; Financial Executives Research Foundation Inc., Florham Park, NJ (2008)
12. [12] P. Mandal, A. Gunasekaran; Application of SAP R/3 in on-line inventory control; International Journal of Production Economics, 75 (1–2) (2002), pp. 47–55
13. [13] P. Soja; Examining the conditions of ERP implementations: lessons learnt from adopters; Business Process Management Journal, 14 (1) (2008), pp. 105–121
14. [14] G. Subramanianh, C. Hoffers; An exploratory case study of enterprise resource planning implementation; International Journal of Enterprise Information Systems, 1 (1) (2005), pp. 23–38
15. [15] E. Wang, et al.; The consistency among facilitating factors and ERP implementation success: a holistic view of fit; Journal of Systems and Software, 81 (9) (2008), pp. 1609–1621
16. [16] Y. Yusuf, A. Gunasekaran, M. Abthorpe; Enterprise information systems project implementation: a case study of ERP in Rolls-Royce; International Journal of Production Economics, 87 (3) (2004), pp. 251–266
17. [17] A. Amid, M. Moalagh, A. Zare Ravasan; Identification and classification of ERP critical failure factors in Iranian industries; Information Systems, 37 (3) (2012), pp. 227–237
18. [18] F. Nah, S. Delgado; Critical success factors for enterprise resource planning implementation and upgrade; Journal of Computer Information Systems, 46 (5) (2006), pp. 99–113
19. [19] W. Delone, E. McLean; Information systems success: the quest for the dependent variable; Information Systems Research, 3 (1) (1992), pp. 60–95
20. [20] W. Delone, E. McLean; The DeLone and McLean model of information systems success: a ten-year update; Journal of Management Information Systems, 19 (4) (2003), pp. 9–30
21. [21] J. Floropoulos, et al.; Measuring the success of the Greek taxation information system; International Journal of Information Management, 30 (1) (2010), pp. 47–56
22. [22] Gable, G., Sedera, D. and Chan, T. “Enterprise systems success: a measurement model”, Proceedings of the 24th International Conference on Information Systems , Seattle, Washington, USA (2003).
23. [23] P. Ifinedo, N. Nahar; ERP systems success: an empirical analysis of how two organizational stakeholder groups prioritize and evaluate relevant measures; Enterprise Information Systems, 1 (1) (2007), pp. 25–48
24. [24] P. Ifinedo, et al.; Relationships among ERP post-implementation success constructs: an analysis at the organizational level; Computers in Human Behavior, 26 (5) (2010), pp. 1136–1148
25. [25] J. Wu, Y. Wang; Measuring ERP success: the key-users’ viewpoint of the ERP to produce a viable IS in the organization; Computers in Human Behavior, 23 (3) (2007), pp. 1582–1596
26. [26] T.H. Chang, et al.; Measuring the success possibility of implementing ERP by utilizing the incomplete linguistic preference relations; Applied Soft Computing, 12 (5) (2012), pp. 1582–1591
27. [27] W.-H. Tsai, et al.; A comprehensive study of the relationship between enterprise resource planning selection criteria and enterprise resource planning system success; Information and Management, 49 (1) (2012), pp. 36–46
28. [28] P. Hanafizadeh, A. Zare Ravasan, H. Ramazanpour Khaki; An expert system for perfume selection using artificial neural network; Expert Systems with Applications, 37 (12) (2010), pp. 8879–8887
29. [29] K. Veera Babu, R. Ganesh Narayanan, G. Saravana Kumar; An expert system based on artificial neural network for predicting the tensile behavior of tailor welded blanks; Expert Systems with Applications, 36 (7) (2009), pp. 10683–10695
30. [30] K. Omoteso; The application of artificial intelligence in auditing: looking back to the future; Expert Systems with Applications, 39 (9) (2012), pp. 8490–8495
31. [31] H. Klaus, M. Rosemann, G. Gable; What is ERP?; Information Systems Frontiers, 2 (2) (2000), pp. 141–162
32. [32] P. Seddon; A respecification and extension of the Delone and McLean model of IS success; Information Systems Research, 8 (3) (1997), pp. 240–253
33. [33] M. Markus, C. Tanis; The enterprise systems experience-from adoption to success; Framing the Domains of IT Research: Glimpsing the Future Through the Past, Vol. 173 (2000), pp. 173–207
34. [34] C. Tan, S. Pan; ERP success: the search for a comprehensive framework; R.D. Banker, H. Chang, Y.-C. Kao (Eds.), Proceedings of the 8th. Americas Conference on Information Systems, AMCIS Press, Dallas, TX (2002), pp. 925–933 AIS e-Library
35. [35] B. Ives, S. Hamilton, G. Davis; A framework for research in computer-based management information systems; Management Science, 26 (9) (1980), pp. 910–934
36. [36] M. Moalagh, A. Zare Ravasan; Developing a practical framework for assessing ERP post-implementation success using fuzzy analytic network process; International Journal of Production Research (2012) http://dx.doi.org/10.1080/00207543.2012.698318
37. [37] J. Razmi, M. Sangari, R. Ghodsi; Developing a practical framework for ERP readiness assessment using fuzzy analytic network process; Advances in Engineering Software, 40 (11) (2009), pp. 1168–1178
38. [38] P. Hanafizadeh, A. Zare Ravasan; A McKinsey 7S model-based framework for ERP readiness assessment; International Journal of Enterprise Information Systems (IJEIS), 7 (4) (2011), pp. 23–63
39. [39] D.E. O’ Leary; Enterprise resource planning (ERP) systems: an empirical analysis of benefits; Journal of Emerging Technologies in Accounting, 1 (2004), pp. 63–72
40. [40] V.A. Mabert, A. Soni, M.A. Venkataramanan; Enterprise resource planning survey of US manufacturing firms; Production and Inventory Management Journal, 41 (2) (2000), pp. 52–58
41. [41] J. Olhager, E. Selldin; Enterprise resource planning survey of Swedish manufacturing firms; European Journal of Operational Research, 146 (2) (2003), pp. 365–373
42. [42] V. Arnold, et al.; Impact of intelligent decision aids on expert and novice decision-makers judgments; Accounting and Finance, 44 (1) (2004), pp. 1–26
43. [43] L. Medsker, J. Liebowitz; Design and Development of Expert Systems and Neural Networks; McMillan College Publishing Company, NY, USA (1994)
44. [44] R.J. Schalkoff; Artificial Neural Networks; McGraw-Hill, New York (1997)
45. [45] S.H. Liao, C.H. Wen; Artificial neural networks classification and clustering of methodologies and applications–literature analysis from 1995 to 2005; Expert Systems with Applications, 32 (1) (2007), pp. 1–11
46. [46] D.E. Rumelhart, G.E. Hintont, R.J. Williams; Learning representations by back-propagating errors; Nature, 323 (6088) (1986), pp. 533–536
47. [47] T. Poggio, F. Girosi; Networks for approximation and learning; Proceedings of the IEEE, 78 (9) (1990), pp. 1481–1497
48. [48] T. Kohonen; Self-organization and associative memory; Self-Organization and Associative Memory, Springer Series in Information Sciences, vol. 8(1), , Springer-Verlag, Berlin, Heidelberg, New York (1988), p. 312 100 figs, XV
49. [49] G.A. Carpenter, S. Grossberg; A self-organizing neural network for supervised learning, recognition and prediction; IEEE Communications Magazine, 30 (9) (1992), pp. 38–49
50. [50] J.J. Hopfield; Neural networks and physical systems with emergent collective computational abilities; Proceedings of the National Academy of Sciences of the United States of America, 79 (8) (1982), p. 2554
51. [51] J.N.D. Gupta, R.S. Sexton; Comparing backpropagation with a genetic algorithm for neural network training; Omega, 27 (6) (1999), pp. 679–684
52. [52] R.S. Sexton, R.E. Dorsey, J.D. Johnson; Toward global optimization of neural networks: a comparison of the genetic algorithm and backpropagation; Decision Support Systems, 22 (2) (1998), pp. 171–185
53. [53] J. Nunnally; Psychometric Theory; McGraw-Hill, New York, NY (1978)
54. [54] J.E. Hunton, R.A. McEwen, B. Wier; The reaction of financial analysts to enterprise resource planning (ERP) implementation plans; Journal of Information Systems, 16 (1) (2002), pp. 31–40
55. [55] R. Poston, S. Grabski; Financial impacts of enterprise resource planning implementations; International Journal of Accounting Information Systems, 2 (4) (2001), pp. 271–294
56. [56] S. Haykin; Neural Networks: A Comprehensive Foundation; (2nd Edn)Prentice-Hall, Upper Saddle River, NJ (1999)
57. [57] J.F.C. Khaw, B. Lim, L.E.N. Lim; Optimal design of neural networks using the Taguchi method; Neurocomputing, 7 (3) (1995), pp. 225–245
58. [58] C. Parten, et al.; Handbook of Neural Computing Applications; (1990)
59. [59] C.G. Dasgupta, G.S. Dispensa, S. Ghose; Comparing the predictive performance of a neural network model with some traditional market response models; International Journal of Forecasting, 10 (2) (1994), pp. 235–244
60. [60] T. Davenport; Mission Critical: Realizing the Promise of Enterprise Systems; Harvard Business Press (2000)