Validation of project management information systems for industrial construction projects

ABSTRACT Project Management Information Systems (PMIS) have been widely used for effective construction project management especially for EPC (Engineering, Procurement, and Construction) projects. It saves a significant amount of time and effort, making project management tasks more effective and efficient. However, most Korean EPC firms develop and use PMIS without validating the system’s compliance with global project management standards. This research presented a methodology to validate the composition and functions of PMIS to ensure it conforms to the global project management standards of project management practice. The researchers defined the core and sub-management processes as well as the required capability level. Proposed global practice, which serves as evaluation criteria, was developed in the form of a work process. Sample PMIS evaluation results confirmed that the core management processes and their functions are compliant with overall global practice. The proposed work processes and evaluation criteria can serve as a global project management requirement for developing PMIS in the future.


Research objective
Project Management Information Systems (PMIS) have been widely used for effective and efficient construction project management, especially for EPC (Engineering, Procurement, and Construction) projects. With the help of information technology, PMIS serves as an essential tool to save, share, and provide key information required for decision making. As the scale and complexity of EPC projects have increased in the global market, effective application of PMIS throughout the whole project execution process becomes more important. The advantage of PMIS is to generate project information during the project's execution process and provide the information to various project participants who require different information to support their management decisions. An effective PMIS application saves significant amounts of time and effort, making project management tasks more sustainable. Thus, development and utilization of PMIS is perceived as an essential tool to project success. However, most EPC firms develop and use PMIS without verifying and validating the system's compliance with global project management standards. As PMIS application increases, it is very important to evaluate validity of PMIS functions developed for the global EPC projects. The PMIS developed and utilized on a project basis needs to be compliant with the global project management standards.
The objective of this study is to define the essential functions that PMIS must perform based on global project management standards and also to develop evaluation criteria for existing PMIS. The researchers presented a methodology to verify and validate the composition and functions of PMIS to ensure it conforms to the standards of global project management practice. By using the evaluation criteria, PMIS development tasks for most EPC projects become more sustainable.

Research methodology
In order to evaluate the validity of PMIS, the researchers developed evaluation criteria by identifying the core management processes (CMP) and submanagement processes (SMP) from the project management process group. The CMP include schedule, cost, and risk management process even though project management process groups include other management processes such as integration, communication, quality, etc. The global project management standards include the standard of practice from Construction Management Association of America (CMAA), Project Management Body of Knowledge (PMBOK) from Project Management Institute, International Standard Organization (ISO), and International Federation of Consulting Engineers (FIDIC).
From these standards, the researchers developed a proposed global practice (PGP) for each SMP. Each PGP represents specific tasks that the PMIS functions carry out to ensure the PMIS used for the EPC projects conform to the global project management standards. This study was conducted using the following procedures.
The purpose and function of PMIS were defined through the existing literature review.
The CMP and SMP by project execution phase was defined from global project management standards.
Required capability levels per SMP were defined. PGP was developed by using process mapping diagrams.
Evaluation Criteria were developed to evaluate whether PMIS is performing in accordance with the PGP workflow.
A sample PMIS was selected to evaluate validity according to the evaluation criteria.

PMIS
The PMIS provides access to management tools, such as cost, schedule, documents, work authorization, configuration, information collection and distribution, or interfaces to other systems (Lee, Lee, and Yu 2010). Automated gathering and reporting on key performance indicators can be part of PMIS. PMIS refers to new means of project and construction management for efficient management in order to optimize construction duration and cost expenditure by supporting prompt and accurate decision-making (PMI 2013). It collects and shares various information related to construction management practice for the purpose of successful project execution. Research on PMIS has been conducted in various ways. Taherkhani (2018) identified key success factors of PMIS by performing hypothetical tests and concluded PMIS influenced the performance of projects. Required functions for PMIS were pointed out by analyzing contract and cost payment method of construction projects (Kang et al. 2015). PMIS for construction claims was studied to manage project information dedicated to claim issues (Wang, Park, and Choi 2018). Tulupov (2020) pointed out the problems of PMIS from the aspects of business and IT and developed a methodology to integrate PMIS with enterprise resource planning solutions. These research commonly analyze construction project management tasks and develop work processes to derive functional requirements. In order to develop an effective PMIS, the functions built into PMIS needs to be structured and standardized (Yu and Lee 2004).

Global project management standards
A standard is a formal document that describes established norms, methods, processes, and practices. As with other professions, the knowledge contained in this standard has evolved from the recognized good practices of project management practitioners who have contributed to the development of the standard. Global project management standards incorporated into this research include PMBOK, Standard of Practice from CMAA, ISO 21500, and FIDIC book even though PRINCE 2 (PRojects IN Controlled Environments) and the Standard by IPMA (International Project Management Association) are also generally accepted.

PMBOK
A guide to the PMBOK provides guidelines for managing individual projects and defines project management related concepts (Lee, Lee, and Yu 2010). It defines 10 important project management knowledge areas, including integration, scope, schedule, cost, quality, resource, communications, risk, procurement, and stakeholder management. It also describes the project management life cycle and related processes. The PMBOK Guide contains the globally recognized standard and guide for the project management profession.

CMAA standard of practice
CMAA's standard contract (CMAA document A-2), the general terms (CMAA document A-3), and the standard contract form between the owner and the contractor are presented as a standard contract form (CMAA 2015). The content consists of relationships with the parties, project definitions, basic services, additional services, duration of construction manager (CM) service, basic services and compensation of construction manager (CM), responsibility of owner (operator), compensation for CM services and payments, insurance and mutual compensation issues, termination and suspension of contract, dispute settlement, and additional provisions.

FIDIC Silver Book
Established in 1913 by the Association of Consulting Engineers, FIDIC is an organization that operates in three European countries and regularly announces the terms of standard contracts applicable to international construction contracts (Choi 2002). Currently, FIDIC contract terms are the most widely used international standard worldwide. Most of the public development projects, such as the West Bank for Reconstruction and Development (IBRD) and the Asian Development Bank, are referenced by many international construction contracts, even if they are not directly used. The official name of the FIDIC Silver Book is "Condition of Contract for EPC/Turnkey Projects" (Yoo et al. 2011). It is used as an international standard contract that applies to EPC projects that are both responsible for engineering, procurement, and construction (FIDIC 2017).

ISO 21500
ISO 21500 is an international standard that provides project management guidance (ISO 21500 2013). This international standard describes the concepts and processes considered best practices in project management. This international standard provides guidance on project management and is applicable to any type of organization, including government, public institutions, enterprises, non-profit institutions. It has the characteristics to be applicable to all projects regardless of project complexity, scale and duration. ISO 21500 describes 11 important project management areas, including project management, organizational strategy, project environment, project operation, and life cycle of project. It also consists of five process groups for project management, including initiation, planning, implementation, control, and termination, as well as ten subject groups: stakeholders, scope, resources, cost, time, risk, quality, procurement, communication, and information.

Software validation method
Software testing includes a verification and validation process (Eom et al. 2004). Verification is the process of determining whether the input of each process conforms to the output of the previous process during the software development phase (DeLone and McLean 2003). It enables users to be confident that the software is designed and implemented as intended. More specifically, such a process verifies that the software is developed as desired by the user and that the system operates according to user requirements described in the requirements specification. Verification is performed at the design, implementation, and testing stages, and the system is examined as to whether or not the results of each previous step of system development are correctly reflected and consistent with the specifications set forth in the previous step.
Validation involves whether or not the results of the requirement specification reflects the needs of users and the developed software product meets specifications (DeLone and McLean 2003). This process ensures the system meets the goals and requirements to validate that the product desired by the user has been developed. Previous research indicated successful application of PMIS's heavy reliance on the factors of user intentions and user satisfaction (Sung et al. 2012). Thus, most PMIS developers significantly improved system architecture and user interface. This approach is limited to the users' perspective regardless of addressing global project management standards.

Defining a project phase
General phases to execute EPC projects are "Bidding", "Planning", "Engineering", "Procurement", "Construction" and "Start-up". In PMBOK, the phases of the project are comprised of "Initiating", "Planning", "Execution", "Control" and "Closing". Each includes tasks from the planning of the EPC project to the startup phase. In CMAA SOP, the project's phases consist of "pre-design", 'design ', "procurement", "construction" and "post-construction". In this study, the phases of the project were organized into "Bidding", "Execution", and "Closing" phases from the perspective of EPC project contractors, and the "Execution" phase was further divided into "Planning" and "Progress Control" phases. The definitions for each phase are shown in Table 1.

Defining management processes and required capability levels
From the three CMP (time, cost, and risk management), SMPs and required capability levels are defined in accordance with global project management standards. SMP means the mandatory tasks required for managing EPC projects. For example, time management as a CMP includes SMP's "Contract schedule preparation", "Evaluating project duration", "Scheduling management planning", "Progress tracking", and "Performance trending", and "Schedule record management". Capability levels are defined based on the CMAA guidelines and examples of EPC projects, meaning the capability required to perform each SMP.

Project time management
Project Time Management includes all the tasks to complete the project scope, including planning, Contract terminated and storage information generated during the execution phase progress tracking, and reporting of progress from the bidding to the closing phases. The Sub-management processes for each stage of the project and required capability levels are shown in Table 2.

Project cost management
Project cost management requires capabilities, from home office and site-level cost management to projects executed by integrated process management. It includes the capability to analyze the feasibility of the cost plan, budgeting, and cash flow from the bidding phase to the project execution, and manage project cost in conjunction with the enterprise resource planning (ERP) as the project progresses. Project cost management includes the function of monitoring cost progress based on contact agreement with the owner and sub-contractors and reporting of the progress. The SMPs for each phase of the project and required capability level are shown in Table 3.

Project risk management
Project risk management includes identifying risks in the bidding phase and maintaining risks registered throughout the project execution and closing phases. A response plan for identified risks means avoiding, transferring, accepting, and reducing risks and requires the capability of management to make the optimal decision based on evaluation. The SMPs for each phase of the project and required capability level are shown in Table 4.

Proposed Global Practices (PGP) and work process development
Once SMPs and required capability levels were defined, Proposed Global Practices (PGP) were developed for the CMP's (Time, Cost, and Risk management) as seen in Tables 5-7. Global project management standards and the corresponding contents were also identified for each PGP. For example, "Contract schedule preparation", one of the SMPs for Time management, and PGP were defined as "PGP-SH-01, Proposal schedule" based on CMAA, PMBOK, and ISO 21500. Furthermore, the work process for each PGP was developed using process diagrams. Among the work process diagrams that the researchers developed, three samples were displayed as seen in Figures 1-3.

PGP's and work process for time management
PGPs for time management consist of nine work processes from "PGP-TIME-01, Proposal schedule" to "PGP-TIME-09, Extension of Time analysis", as seen in Table 5. Once a project schedule was proposed (TIME-01) and agreed upon by the project owner in the form of a baseline schedule (TIME-04), the progress in every phase of execution is monitored (TIME-06) to prepare a comprehensive progress report. When variation increases between Progress tracking Capability to monitor performance against plan during execution, monitor progress status, and analyze project delays Performance trending Capability to determine necessity of schedule revision for issues such as schedule delay, and provide information for alternatives when necessary Closing Schedule record management Capability to systematically store and reuse productivity, resource utilization information used to execute projects Capability to systematically store and reuse unit cost, resource utilization information used to execute projects planned and actual progresses, and if it is deemed necessary to establish recovery plans from the progress delay, the recovery plans are established.

PGP's and work process for cost management
PGPs for cost management consist of seven work processes from "PGP-COST-01, Contract Cost Analysis" to "PGP-COST-07, Final Cost report/Project closeout", as seen in Table 6. The purpose of establishing a cost management plan is to complete the project within the target budget. Project budgets are determined based on the contract information and the budgetary criteria. It is important to review contract terms in the contract documents so that the entire project scope is evaluated and captured when developing project Capability to manage risks systematically and continuously by establishing an overall management plan for the risks that may arise in the course of the project Risk identification Capability to identify existing and potential risks Risk analysis Capability to analyze the impact of identified risks by assessment and to analyze the probability of occurrence Risk response plan Capability to compare the probabilities of establishing a response plan and provide information to determine whether to win a bid Planning Implementing risk response plans Capability to minimize risk before initiating a project by executing a simulated response plan

Progress control
Risk control Capability to monitor risks in progress and execute risk management procedures when new risks are identified.

Risk record management
Capability to save and assess risk information by type, cause, impact, and mitigation strategy  (2017) Variations and adjustments budgets. Table 6 shows PGPs developed for SMPs of cost management.

PGP's and work process for risk management
The purpose of the project is to establish procedures and standards for the types of tasks so that appropriate risk management tasks can be carried out during the project.
PGPs for risk management consist of five work processes from "PGP-RISK-01, Plan risk Management" to "PGP-RISK-05, Risk Monitoring", as seen in Table 7. Prior to risk identification, the project is investigated through each aspect of strengths and weaknesses to expand the scope of risk. The project risk team gathers ideas and information to identify risk items. Once the items are identified, they are grouped by Risk Breakdown Structure (RBS). The items are reviewed and analyzed by comparing them with previous similar projects and their historical data. Development of Evaluation Criteria Through the defined Sub-management process and Proposal Global Practice, PMIS evaluation criteria were developed as shown in Table 8. The evaluation criteria ensure a PMIS complies with management functional requirements by the global project management standards.
The time management clearly defines the organizational structure. It assesses whether the project progress management is managed by a regular reporting system and whether a response plan, such as additional resource inputs and resource reassignment, is established to respond if the progress of the project differs from the plan. Furthermore, it determines if the task is planned to facilitate revision or track when changes occur.  Cost management evaluates whether a cost execution plan has been established and whether a management plan has been established to regularly understand the current status of the execution costs by predicting the total cost of the project.
Risk management establishes risk management guidelines during project execution and evaluates whether they are planned to be identified, analyzed, and managed periodically. Therefore, the assessment of whether PMIS includes PGP's work or conforms to the standards of international project work is    • Was a procedure for the construction duration analysis established?
• Was the adequacy of the bid schedule reviewed according to the procedure?
Planning and scheduling for management PGP-TIME-03 Milestone schedule • Does the milestone schedule include the entire project scope?
• Was the milestone schedule provided to management presented in the simplest and most concentrated form? PGP-TIME-04 Baseline schedule • Does the baseline schedule meet a management standard?
• Was the baseline schedule consistent with the contractual plan?
PGP-TIME-05 Details progress schedule • Do you perform detailed and timely progress status and resource utilization?
Progress tracking PGP-TIME-06 Progress control • Do you measure the planned versus actual progress based on progress measurement criteria? Performance trending PGP-TIME-07 Extension of time • Do you develop a cost management plan to achieve the target budget by establishing cost management procedures and standards for project execution? PGP-COST-03 Estimate costs • Do you analyze the resources required for execution of the project scope?
• Do you estimate the optimal cost to perform the project?
• Do you systematically build resource, work crew, and work item information used for project cost management and cost estimation? PGP-COST-04 Determine budget • Do you determine the optimal project budget based on the effective project implementation procedure? Cost review PGP-COST-05 Control costs • Do you manage and analyze the cost expenditure against the planned budget during the entire project execution period? ERP integration PGP-COST-06 ERP integration • Do you utilize the ERP system by linking with the project CBS?
Cost record management PGP-COST-07 Final cost report project closeout • Do you store and reuse information on project execution costs? (Continued) evaluated through the 5-point scale evaluation rating system in Table 9.

Validation of evaluation criteria
The researchers selected a PMIS for implementing and validating the evaluation criteria. The PMIS consists of seven modules, including contract, time, cost, risk, document, quality, and project administration. Each module provides the user with the required management functions. In order to validate developed evaluation criteria, three modules (time, cost, and risk management) were selected. Figures 4-6 are the screenshots of each management module.

Time management
Time management module (level 1) consists of functions for Baseline Schedule, Plan Schedule, Control Schedule, Master Schedule, S-Curve, Primavera I/F (level 2). Level 2 functions include detailed functions (level 3), including activity details, crew and resource lists, as well as a milestone schedule, progress track, S-curve, and data log. Planning and progress tracking in addition to a performance trending function were integrated with cost management modules.

Cost management
The cost management module is designed to monitor cost expenditure progress execution from the project • Do you revise the plan for potential risks related to project execution? The Global Standard requirements and system functions are executed in full accord, the functions required by the Proposed Global Practice are implemented in the system, and the contents are systematically in place.

(Good)
Although most of the assessment items are satisfied, the integrated level between the global standard and PGP is insufficient. 3 (Average) Most system functions are integrated with the PGP, but lack the individual decision making process.

(Poor)
System functions are not integrated with the PGP and considerably lack most PGPs. 1 (Very poor) System functions and Proposed Global Practice requirements are not met.
budget. It includes functions such as a bill of quantity, cost breakdown (CBS) list, unit cost list, overall cost progress, actual cost, EVMS, and cash flow projection.

Risk management
The risk management module is designed to import the risk information from the resource breakdown system (RBS) to the risk management system. It includes functions of the RBS list, risk identification, risk evaluation, monitoring, and tracking.

System evaluation results
The sample PMIS was evaluated in conjunction with the PGP's of the evaluation criteria developed. The result is seen in Figure 7. PGP rating scores for time, cost, and risk management appeared over average. For time management, PGP-02_Construction duration adequacy analysis, PGP-06_Progress control, and PGP-08_Schedule recovery plan scored the highest, while PGP-05_Details progress schedule and PGP-09_Claims review scored the lowest. Construction duration is one of the most critical contractual  obligations. In order to complete the construction duration, progress tracking and recovery planning become core tasks.
For cost management, PGP-03_Estimate costs and PGP-06_ERP integration scored the highest while PGP-01_Bill of quantity and PGP-04_Determine budget scored the lowest. Cost estimation is the critical task of coming up with the budget amount. Because the actual cost was tracked from the ERP system, the ERP integration was an important function in the sample PMIS.
For risk management, PGP-02_Identify risk and PGP-03_Risk analysis scored the highest, while PGP-06_Risk monitoring scored the lowest. The function of PGP-02 discovers specific risk factors that may affect the scope, quality, schedule, and  cost of the project and PGP-03 estimates the effects of them before they occur.
Overall evaluation results showed that the sample PMIS captures the most critical functions for project management. The degree of functional capability built into the PMIS was highly dependent on the actual capability of project management. Therefore, the functions found in the global project management standards were not necessarily systemized as in the sample PMIS.

Conclusion
This study showed the PMIS evaluation method applied to the EPC market in order to validate compatibility with global project management standards. Sample PMIS evaluation results confirmed that core management processes and their functions are compliant with global project management practice overall. Securing compatibility with the global practice is essential for the Korean EPC firms to increase their market share in the global market. This study has defined the core and sub-management processes as well as the required capability level. Proposed global practice, which serves as evaluation criteria, was developed in the form of work process and descriptions. The evaluation criteria developed are applicable to other PMIS's by following the same process presented in this research.
Through this practice, it is expected that Korean domestic EPC firms will strengthen their competitiveness in the global market. The acquisition of one PMIS with system architecture and detailed functions was one limitation of this research that resulted in confidentiality issues with the EPC firms. Even though the sample PMIS used for the evaluation purposed was obtained from a Korean construction firm, validating a PMIS functions can benefit international EPC firms and the evaluation criteria may serve as a global project management requirements for developing PMIS in the future.