Cost assessment model for sustainable operation and maintenance of large public research infrastructures

ABSTRACT For efficient operation and maintenance of Large Public Research Infrastructures (LPRIs) built with expensive equipment and operating engineers, reasonable budget needs to be allocated. However, it is difficult to fulfill sustainable operation and maintenance (O&M) because there is no standard on budgeting for efficient LPRI operation. There have been a lot of cost assessment studies regarding O&M of high demand facilities such as hospitals, hotels and residential buildings, but a very few on sustainable O&M of LPRI. The objective of this paper is to propose a cost assessment model for sustainable operation and maintenance of large public research infrastructures. Actual O&M data of six LPRIs are collected, and cash flow model and regression analysis model are used for development and evaluation a combined cost assessment model (CAM). When O&M cost estimated by CAM is compared with actual O&M cost of LPRI, the actual O&M cost is 21% less than the estimated cost. The result implies that O&M cost of existing LPRI may have been over or under appropriated than the reasonable standard for efficient operation, and it reveals that it is necessary to establish a budgeting standard for sustainable O&M. Graphical Abstract


For efficient operation and maintenance of Large Public
Research Infrastructures (LPRIs) built with expensive equipment and operating engineers, reasonable budget needs to be allocated. 2. However, it is difficult to fulfill sustainable operation and maintenance (O&M) because there is no standard on budgeting for efficient LPRI operation.
The objective of this paper is to propose a cost assessment model for sustainable operation and maintenance of large public research infrastructures.

Background Objective
1. Review is conducted in relation to the importance of O&M from a facility management (FM) perspective. 2. 10-year O&M cost data of 6 LPRIs at Phase 1 in operation are collected and analyzed to identify key influence factors. 3. With the identified factors, CAM integrated with CFM and RAM is developed and evaluated.

Methodology
1. The study developed a realistic cost assessment model that makes it possible for related parties to set construction plans and budget plans from the initial planning stage for sustainable and settled O&M of LPRI.

Introduction
The Korean government has invested a tremendous amount of money in the last decade to build large public research infrastructures (LPRI). Accumulative investment for LPRI with more than 4.5 million dollars owing to introduction of national R&D is approximately 5.5 trillion dollars (NFEC (National Research Facility & Equipment Center) 2017). We are seeing a convergence of and expansion in scientific technologies in the Fourth Industrial Revolution (Li, Hou, and Wu 2017;Petrillo et al. 2018;Lee, Park, and Kang 2018;Kovalchuk 2011), and the importance of LPRI for testing scientific technologies is more evident. It is regarded as one of the most significant requirements to maximize R&D outputs as LPRI dependence increased due to the convergence of and expansion in emerging technologies (Bronzini and Piselli 2009;Lee 1988;Nadiri and Mamuneas 1991). However, it is difficult to fulfill sustainable O&M because there is no standard on budgeting for efficient LPRI operation, including expensive equipment and manpower allocation (Hong 2012).
There have been various studies on high-demand facilities such as hospitals, hotels and residential buildings for O&M cost assessment. Lai and Yik (Lai 2010) analyzed energy cost which accounts for the largest portion in O&M cost of hotels and proposed O&M benchmarking solutions that may be applicable to similar projects (Lai and Yik 2008). Hassanain, Assaf, S., Al-Ofi and Al-Abdullah (Hassanain et al. 2013) identified factors that have influence on O&M cost in the view of facilities managers and suggested solutions for cost reduction in hotels (Hassanain et al. 2013). Ali et al (Ali et al. 2010) presented solutions to minimize O&M cost of residential buildings by conducting a questionnaire survey for facilities managers and supervisors (Ali et al. 2010). As mentioned above, studies in a wide range of fields have been conducted for O&M cost assessment.
However, there are a very few studies on O&M cost assessment of LPRI, which is the subject of this study, although reasonable budget allocation is needed for efficient O&M of expensive test equipment, professional engineers and facilities. Therefore, a budgeting standard for efficient O&M of LPRI in the initial planning stage is required and a cost assessment model to support this should be developed. The objective of this paper is to propose a cost assessment model for sustainable operation and maintenance of large public research infrastructures. To do so, O&M cost data of six LPRIs in operation for 10 years after their completion are collected and analyzed to identify key influence factors. Based on the deducted influence factors, a cost assessment model is developed and the developed model is applied to cases for reliability evaluation. The study result will support sustainable operation of LPRI from a business perspective and be used as basic data for continuous development of cost assessment models to establish budgets for LPRI operation from an academic perspective.

Definition and current status of LPRI (Large public research infrastructure)
LPRI in Korea is defined as research facilities and equipment built with national R&D investment that exceeds 4.5 million dollars. With their outstanding performance in the fields of scientific technologies, they are applied to high-tech researches and these infrastructures have a major influence on related fields (NFEC (National Research Facility & Equipment Center) 2014a). LPRI offers a space for researchers to perform tests and experiments to develop technologies and research capacity, and such infrastructure is a must to put research result to practical use for people's lives and safety (Hailey et al. 2005;Broekel and Schlump 2009;Stankiewicz 1995;Freeman 2004;Adams, Marcu, and Wang 2008).
Additionally, research facilities for public interest that are difficult to be built with private investment are put together for concentrated investment to prevent overlapping investments of LPRI and a quick, convenient one-stop service from certification to commercialization of test results is offered for researchers.
Accumulative investment is about 54.2 billion dollars in 98 projects and the current status of investment by research body is as follows: government research institute 3.1 billion dollars (57.4%), university 0.88 billion dollars (16.3%) and the others 1.39 billion dollars (26.3%). There are 84 LPRI in operation, excluding those in the construction stage and as shown in Figure 1, there are 55 facilities (65%) with basic operation plans for sustainable O&M and the remaining 29 facilities (34.5%) rely on income from facilities use, etc. (NFEC (National Research Facility & Equipment Center) 2014b). 29 LPRI demonstrate unstable operation and have problems such as increase in idle facilities/equipment and difficulty in securing professional manpower. There are even facilities that have stopped operation because they did not have sufficient income to cover operating expenses.
In case of large research facilities of the National Science Foundation (NSF) and Department of Energy (DOE) in America, they are owned by the state and an operator is assigned with a 5-year contract authorizing operation as detailed in Table 1. DOE research facilities are owned by the state and operators are consigned for operation with the government supporting most operating expenses. Public officials from DOE manage and supervise overall LPRI operation including accounting and safety. In Germany, the German Research Foundation (DFG) owns the LPRI and oversees the management and support of national research facilities and equipment. Similarly, in Japan, JST owns and manages research facilities and equipment built with government research and development funds. It also manages and supervises over the overall operation of research facilities.
Therefore, when planning LPRI, it is a must to have both operation plan on operating party and source of income and sustainable O&M cost plan. With this in consideration, LPRI constructed with a huge government budget will continue to operate and a sustainable O&M cost plan can be set (Sánchez-Silva et al. 2016;Lenferink, Tillema, and Arts 2013;Park, Park, and Lee 2016).

Influencing factors of operation and maintenance cost
According to a study on O&M cost estimation factors conducted by Barco (Barco 1994), it is rocket science to allocate O&M budget for public infrastructures and it is very important to benchmark similar facilities to establish an accurate budget plan. The study indicates that the roles of engineers who have full understanding of the entire procedure from construction to O&M as well as labor cost are important to benchmark O&M cost of similar facilities and allocate budget by major factor.
Sliteen and Catarina (Sliteen, Boussabaine, and Catarina 2011) monitored O&M cost data and operating performance of several hospitals in operation to establish a practical yet efficient O&M cost plan for hospitals. They analyzed resources per unit area and it was found that labor cost and facility and equipment maintenance cost are important factors that account for a great portion in budgeting.
Lai (Lai 2010) classified commercial buildings by type to identify key influence factors on O&M cost of commercial buildings and conducted interviews with managers in charge of hotels, offices and retail stores. As a result, labor and energy cost are key influence factors of O&M cost and with an applicable O&M cost plan, labor and energy expenses can be minimized. Sharma and Qasim (Sharma, Najafi, and Qasim 2013) developed an O&M cost assessment model for water treatment plants and stated that the model can be established with the past O&M cost data. They developed a preliminary cost estimation model on elements from O&M cost data and proposed a method of using single cost index for O&M cost updates. They mentioned that energy cost and labor cost act as key influence factors and a precise design and equipment specification are needed for accurate cost assessment.
According to the preceding studies, major factors that have influence on cost assessment include labor, facility and equipment maintenance, and energy cost. The study herein refers to the above-mentioned studies to analyze O&M cost data and identify key influence factors.

Cost assessment techniques
Looking into the studies on cost assessment techniques, Marchionni, Cabral, Amado Covas (Marchionni et al. 2016) used regression analysis for assessment of water supply infrastructure O&M cost. Regression analysis can analyze trend reflected on numerous data and may sufficiently contribute to sustainable infrastructure management in mid/long term. However, it is said that it is difficult to collect O&M data of similar infrastructures, although they are in need.
Lowe, Emsley Harding (Lowe, Emsley, and Harding 2006) developed a linear regression model based on data of 286 construction sites to provide accurate construction cost for clients. They stated that a regression equation deducted from data of numerous construction sites can be used for correlation analysis of variables and relatively accurate cost prediction is possible. However, they also pointed out that unless data of 286 construction sites are classified according characteristics such as site/building size, underground conditions, etc. for regression analysis, result that fails to reflect the site characteristics may be deducted. Thus, it has a limit in predicting cost only with regression analysis.
Chen, O'Brien and Herbsman (Chen, O'Brien, and Herbsman 2005) used cost-schedule integration (CSI) techniques to perform a study on evaluating accuracy of a case flow model depending on variables such as delayed project cost payment, component and frequency. They reported that with accurate data input, accurate cost prediction according to variables may be realized, but it is relatively difficult to predict cost that reflects trend.
According to the preceding studies related to costassessment techniques, mainly regression analysis or cash flow technique was used. In summary, regression analysis is capable of conducting correlation analysis of variables with prediction reflecting trend based on past data analysis, yet it is difficult to collect similar data and it has limitations that error may occur during data analysis and it is impossible to figure out the details. On the other hand, cash flow model is capable of performing cost prediction in details, yet it is relatively difficult to predict cost that reflects trend. Therefore, the study herein refers to the abovementioned studies and uses a combined cost assessment model (CAM) that integrates the cash flow model (CFM) and regression analysis model (RAM).

Methodology
The study procedure is as illustrated in Figure 2. Firstly, the definition and current status of LPRI in Korea as well as six LPRIs at Phase 1 of civil and transportation engineering are reviewed. Additionally, literature review is conducted in relation to the importance of O&M from a facility management (FM) perspective, key influence factors on O&M cost and cost assessment techniques. Secondly, 10year O&M cost data of six LPRIs at Phase 1 in operation are collected and analyzed to identify key influence factors. Thirdly, with the identified factors, CAM integrated with CFM and RAM is developed. Here, an arithmetical mean of O&M cost estimated from CFM and RAM respectively is calculated for final O&M cost (CCAM) calculation. Fourthly, the developed model is applied to LPRI at Phase 2 in operation (completed in 2019) for reliability evaluation. Here, if O&M cost calculated by CFM and RAM fails to meet the allowable error (εa), CFM and RAM are reexamined for O&M cost assessment. Finally, as a way of improving reliability of the study and as a part of follow-up study, appropriate rate of operation per equipment reflecting the characteristics of test field is estimated to propose a study that may calculate proper O&M cost in relation to income.

Development of cost assessment model
This section explains the concept of CAM. CAM is a combination of CFM and RAM at the same time. CFM is applied to estimate details of O&M cost based on actual data and for relatively accurate cost assessment (Casey and Bartczak 1985;Krishnan and Largay 2000;Hwee and Tiong 2002), whereas RAM is applied to develop a multiple regression model that reflects trend based on data from the past and for verification using regression equations and cost prediction based on the verification (Alshamrani 2017;Trost and Oberlender 2003). Making use of the advantages of both methods, CFM and RAM are applied to develop a combined cost-assessment model (CAM) for O&M cost prediction as shown in Figure 3.
First, O&M cost (C CFM ) is assessed by CFM. Here, labor cost, maintenance cost of facility and equipment, and management cost calculation formulas of LPRI are used to figure out each cost and the sum of each cost is calculated to estimate total O&M cost (C TTL_CFM ). For reference, the management cost here is the overhead costs for LPRI operation such as insurance cost, tax, promotion cost and so on.
Second, O&M cost (C RAM ) is assessed by RAM. Here, multiple regression analysis is applied for cost calculation (C TTL_RAM ) based on 60 actual O&M data of 6 LPRIs that have been in operation for the last 10 years.
Third, O&M cost by CFM is compared with the cost by RAM. Here, interviews with hands-on workers of six LPRIs at Phase 1 are conducted to set an allowable error (ε a ) as 10%.
Fourth, if the difference between 2 values meets the allowable error (ε a ), an arithmetical mean of each O&M cost is calculated to obtain final O&M cost (C CAM). If it fails to meet the allowable error, review (routine A, B) is performed and O&M cost is recalculated to satisfy the set allowable error (ε a ).

Data collection and analysis
There are 12 types of LPRI in the civil and transportation engineering sectors and are divided in to 2 phases according to the construction period. Six types at Phase 1 were planned in 2004 and constructed for 5 years. In 2009, they were built separately in universities and are in operation for about 10 years. Construction cost is estimated to be around 64.1 million dollars. Additionally, construction cost of six types at Phase 2 is approximately 108.5 million dollars, they were completed in 2019 and are at the initial operation stage. For this study, 60 ten-year O&M cost data of 6 facility types at Phase 1 are collected and analyzed. We collected O&M data of 6 out of the 84 LPRIs and performed regression analysis with normalization. Data collected include the annual income, which include usage and other fees, and expenditure of each facility, which include labor cost, facility and equipment management cost, and operational cost. Developing CAM by collecting O&M data of 84 LPRIs will be reliable. However, collecting O&M data on 84 LPRIs is difficult in practice; therefore, it is reasonable to continue collecting O&M data in future to increase the reliability of CAM.
The testing field of six LPRIs includes (1) structural engineering, (2) geotechnical engineering, (3) material engineering, (4) earthquake engineering, (5) windresistant engineering and (6) coastal harbor engineering, and description of each facility as well as construction cost are detailed in Table 2.
As shown in Figure 4, actual amount for each O&M cost item of 6 LPRIs is analyzed and items with cumulative ratio of 80% are labor cost (46.6%), facility and equipment maintenance cost (19.6%) and management cost (16.5%). In other words, three cost items account for 80.0% or more of the total operation and maintenance cost. This also implies that these items represent the total O&M cost.
The three major factors described in Figure 4 and the data distribution characteristics of the O&M cost are shown in Figure 5. First, as shown in Figure 5(a), the O&M cost is maximum at 2,392 (1,000 USD), average at 528 (1,000 USD), and minimum at 81 (1,000 USD). As shown in Figure 5(b), the labor cost is maximum at 1,454 (1,000 USD), average at 227 (1,000 USD), and minimum at 28 (1,000 USD). As shown in Figure 5(c), the facility and equipment maintenance cost is maximum at 367 (1,000 USD), average at 85 (1,000 USD) and minimum at 3 (1,000 USD). As shown in Figure 5 (d), the management cost is maximum at 409 (1,000 USD), average at 77 (1,000 USD) and minimum at 0.45 (1,000 USD). Furthermore, as shown in Figure 5, the collected data have normality. At this time, we applied the exchange rate of KRW 1,105.7 per USD announced by the Bank of Korea on 17 November 2020. Table 3 shows the result of analysis on correlations between O&M cost and key influence factors. As shown in Table 3, Pearson's correlation coefficient between O&M cost and labor cost, facility and equipment maintenance cost, and management cost is 0.961, 0.872 and 0.773 respectively. Significance probability is less than 0.05, which indicates that it is statistically significant. The result demonstrates that O&M cost and each factor are very closely related. Furthermore, Figure 6 and Table 4 represents the result of regression analysis on O&M cost and key factors. There is a strong positive correlation between O&M cost and labor cost, and R 2 value is 0.924, which indicates that it has high explanation power (fitness) of 92.4%.
There is a strong positive correlation between O&M cost and facility and equipment maintenance cost, and R 2 value is 0.861, indicating that it has high explanation power (fitness) of 86.1%. Finally, there is a positive correlation between O&M cost and management cost and R 2 value is 0.817, which indicates that it has high explanation power (fitness) of 81.7%. The result reveals that increase in labor cost, facility and equipment maintenance cost and management cost means increase in O&M cost. Hence, these factors should be considered for O&M cost plans of LPRI and systematic, reasonable budget plans should be established.

Cash flow model
Cash flow model (CFM) of the study is applied to calculate O&M cost with the sum of three factors (labor cost (C Labo r), facility and equipment maintenance cost (C F&E_m ) and management cost (C M ) as shown in Equation (1).
here, x 1 = Labor cost, x 2 = Facility Equipment Maintenance cost, x 3 = Management cost As shown in Equation (2), C Labor can be calculated by applying unit wages per class (professional engineer (PE), senior engineer (SE, engineer with more than 7 years of experience), intermediate engineer (IE, engineer with 4-7 years of experience) and junior engineer (JE, engineer with 2-4 years of experience)) as defined by the Korea Engineering & Consulting Association based on 20 working days per month.
(2)   (3), maintenance cost of facility and equipment (C F&E_m ) is calculated by first multiplying purchase cost of facility and equipment (C F&E_p ) by maintenance rates of facility and equipment (R F&E_m ) and then dividing the multiplied value by designated annual usage of facility and equipment (T F&E_year ). Then, it is multiplied by actual usage of facility and equipment (T F&E_use ) and facility and equipment usage rates (R F&E_use ). Here, a standard for each country or facility is applied to each variable excluding the purchase cost. For this study, the reference variable per equipment in Korea is applied. R F&E_m is 0.04, T F&E_year is 1,000, T F&E_use is 8 and R F&E_use is 100. This is the maintenance cost of one equipment and the sum of maintenance cost of each main equipment is calculated to obtain C F&E_m .
C F&E_m : Maintenance cost of facility and equipment, C F&E_p : Purchase cost of facility and equipment, R F&E_m : Maintenance rates of facility and equipment, T F&E_year : Designated annual usage of facility and equipment, T F&E_use : Actual usage of facility and equipment, R F&E_use : Facility and equipment usage rates.
As shown in Equation (4), management cost (C M ) is calculated by adding promotion cost (C P ), insurance cost (C I ) and tax.
C M : Management cost, C P : Promotion cost, C I : Insurance cost Figure 7. Shows case flow information of 1 type among the investigated six LPRIs that has been in operation for 10 years.  It presents cash flow of C Labor , C F&E_m and C M , which are factors that have the most impact on the total O&M cost. Since the actual amount executed changes over time, NPV (net present value) as of the reference point can be used to compare it with the planned budget for review. Additionally, the amount executed by year and by item can be used to find out whether it exceeds the planned budget and cumulative amount over time can be used to check the remaining balance.

Regression analysis model
Regression analysis model (RAM) is applied to identify a regression equation and such equation is used to calculate O&M cost (C TTL_RAM ). Based on factors analyzed in the previous section, multiple regression analysis is performed as shown in Table 5.
As shown in Table 5, when three independent variables are applied, the coefficient of determination (R 2 ) is 0.982 having explanation power (fitness) of 98.2% with the dependent variable (O&M cost) and the adjusted R 2 reflecting degree of freedom is 0.981. Additionally, as a result of analyzing influencing relations between O&M cost and three factors, the t-value and significance probability of labor cost (C Labor ) are 17.528 and 0.000 respectively; those of facility and equipment maintenance cost (C F&E_m ) are 4.359 and 0.000; those of management cost (C M ) are 10.604 and 0.000. It indicates that they are significant in relations with O&M cost. Furthermore, as shown in Table 5, tolerance and variance inflation factors of collinear statistics are indicators for determining collinearity between independent variables. In the O&M cost impact analysis, the tolerance limit is greater than 0.10, and the VIF is less than 10; therefore, no problem exists with collinearity.
Standardized coefficients are compared as shown in Table. As a result, it is in order of C Labor (0.661), C M (0.271) and C F&E_m (0.159). Thus, it can be said that C Labor is the most influential factor among 3 independent variables to O&M cost. A multiple regression model demonstrating the result is expressed in Equation (5).
here, Y: C TTL_RAM, X1: C Labor , X2: C F&E_m , X3: C M As mentioned above, this section is on multiple regression analysis for O&M cost and key factors. With the identified regression model, O&M cost in consideration of C Labor , C F&E_m and C M can be estimated.

Cost-assessment model
A combined CAM that applies both CFM and RAM is used to estimate final O&M cost. Here, interviews are conducted with hands-on workers of six LPRIs at Phase 1 to set an allowable error (ε a ), which is the difference between CFM and RAM. Methods for determining allowable error include engineers' experience, the  worst on worst (WOW) allowable error analysis method, and root sum square (RSS) allowable error analysis method (Lin et al. 1997). In the present study, the average value was used for conducting interviews with 12 hands-on workers of six types at Phase 1. The allowable error for this study is 10% and it may be applied differently depending on the facility characteristics and professional engineers' perspectives. When it meets the defined allowable error (ε a ), an arithmetical mean of each O&M cost is calculated as shown in Equation (6) to estimate final O&M cost (C CAM ).
If it exceeds the allowable error (ε a ), routine A and B reviews are performed to have it satisfied. O&M cost of CFM and RAM is recalculated and final O&M cost is estimated. CFM and RAM are calculated as the sum of labor, facility and equipment maintenance, and operating costs. The cost for each item is calculated from a number of variables, such as in equations 2-6, whose variables can be adjusted. For example, in the facility and equipment maintenance cost, the maintenance and usage rates of the facility and equipment can be adjusted, and it can be recalculated.

Evaluation of the model
A case study is conducted by comparing CAM proposed in the study with the actual O&M cost of OO testing center, LPRI at Phase 2 in operation. OO testing center is located in Chungcheongbuk-do, the Republic of Korea and was completed in 2019 after about 4 years of construction with construction amount of 11,000,000 USD. This LPRI is a comprehensive laboratory facility that is capable of replicating actual climate environment such as temperature, humidity, solar radiation, rainfall and snowfall for overall performance assessment of the actual facilities as well as specific performance assessment of mock-up test specimen and materials/facilities/equipment. It is a three-story building with a basement floor and the total ground area is 6,280㎡. LPRI workers include 1 PE, 1 SE, 2 IEs and 2 JEs, and main equipment in operation are climate chambers (L, M, S). The summary of OO testing center is as shown in Table 6. C Labor , C F&E_m and C M of the LPRI case that has been operation for a year since 2020 is 283,552, 105,815 and 59,342 USD respectively and its total O&M cost is 448,709 USD. Finally, O&M cost (CCAM) calculated by CAM is compared with the actual O&M cost of this building for evaluation.
First, CFM equation is applied to calculate C Labor , C F&E_m and C M and they are added up to calculate total O&M cost (C TTL_CFM ). For labor cost (C Labor )    (2) is applied to calculate labor cost by engineer class and they are added up to calculate total labor cost. For instance, labor cost of professional engineers is calculated as shown in Equation (7).

estimation, Equation
With the same equation above, labor cost of SE, IE and JE who worked at LPRI is calculated as follows: 57,870, 94,755 and 83,105 USD. Thus, the total annual labor cost of this LPRI is the sum of labor costs calculated which is equivalent to 305,376 USD.
For maintenance cost of facility and equipment (C F&E_m ) estimation, Equation (3) is applied to calculate maintenance cost by facility and equipment and the calculated maintenance costs are added to obtain total maintenance cost. For instance, when the abovementioned Equation (3) is applied for one of the main equipment, Chamber (L), it is as shown in Equation (8).
With the same equation above, maintenance cost of Chamber (M) and Chamber (S), the main equipment of LPRI, is 57,882 and 28,941 USD. Thus, the total annual maintenance cost of facility and equipment (C F&E_m ) of this LPRI is the sum of maintenance costs calculated, which is equivalent to 173,646 USD. C M is calculated as The total annual C M cost of this LPRI is 70,347 USD. Accordingly, the sum of C Labor , C F&E_m and C M calculated using Equation (7) ~ (9) which is 549,369 USD is final C TTL_CFM of LPRI calculated by CFM. Second, C RAM calculated with Equation (5) is as shown in Equation (10).
Finally, O&M cost by CFM and that by RAM are compared to estimate O&M cost by CAM. As a result, the calculated O&M cost is 549,369 USD and 592,717 USD respectively with error value of 7%. It is within the defined allowable error (10%) and an arithmetical mean of these values is calculated to obtain final C CAM, which is equivalent to 571,043 USD. When it is compared with the actual cost (448,709 USD), it is approximately 21% less than the cost calculated by CAM (571,043 USD In this way, we compared the actual O&M costs of the remaining five types at Phase 2 LPRIs and the O&M costs derived from each equation, as shown in Table 7. The difference between the actual O&M costs and the O&M costs calculated from the CAM model ranged from the minimum of 6% to the maximum of 21%.
This implies that O&M cost of the case LPRI is overor under-appropriated than the appropriate standard for effective operation, leading to a necessity to set a proper budget plan for sustainable O&M. Table 7 shows the comparison of the actual O&M cost with O&M cost estimated by CFM and RAM. Continuous feedback and data accumulation/evaluation will increase reliability of CAM.

Discussion and conclusion
The study performed O&M cost assessment by CAM which uses both CFM and RAM. CFM may realize cost prediction on details, yet it is relatively difficult to predict cost that reflects trend. RAM may analyze correlations among variables and perform predictions reflecting trend based on accumulated data, but it is difficult to collect similar data and has some limits that errors may occur during data analysis and details are unknown. CAM proposed in the study makes the best use of both CFM and RAM for sustainable O&M of LPRI and as a method of assessing O&M cost, it compensates weaknesses by increasing accuracy of cost prediction and reflecting trend with similar data in the past. The subject of cost assessment was LPRI in the civil and transportation engineering field. This is only one of 12 categories in scientific technologies in Korea and there are various testing fields within the civil and transportation engineering field. In addition, testing equipment has different characteristics. Therefore, model evaluation should be performed with the accumulated O&M data to increase CAM accuracy.
Labor costs, facility and equipment maintenance costs, and operating expenses, which are major factors of the O&M cost, can be applied equally to all the LPRIs. Moreover, if any major factor is different, we change only the specific factor and simulate using the CAM model.
Moreover, for further studies, proper available rate of main facility and equipment considering the characteristics of equipment and experiment field should be estimated, and follow-up studies on predicting O&M cost exceeding facility/equipment rental cost, which is one of the LPRI incomes should be performed. When further studies are conducted, it may be possible to handle exceeding O&M cost with O&M cost prediction per scenario.
The study developed a realistic cost assessment model that makes it possible for related parties to set construction plans and budget plans from the initial planning stage for sustainable and settled O&M of LPRI. Sixty cases of actual O&M cost data accumulated from six LPRIs that have been in operation for 10 years after completion were analyzed and CAM that uses both CFM and RAM is developed based on the analysis. The study result is as described below.
First, C TTL_CFM was calculated by CFM. Here, each cost was estimated using an equation per key influence factor (C Labor , C F&E_m , C M ) and total O&M cost was calculated with the sum of the estimated costs. This indicates that relatively accurate cost estimation per influence factor is possible based on O&M data of LPRI in operation. The calculated C TTL_CFM of the case was 549,369 USD.
Second, C TTL_RAM was calculated by RAM. Here, it was validated by multiple regression analysis. As a result, the coefficient of determination was 0.982, which showed that three independent variables (C Labor , CF&E_m, C M ) had high explanation power (fitness) of 98.2% to the dependent variable (O&M cost). This means that more realistic O&M cost can be calculated using O&M data of LPRI in operation. The calculated C TTL_RAM of the case was 592,717 USD. Third, O&M cost by CFM was compared with O&M cost by RAM and the error value between them was 7%, which meets the defined allowable error (ε a ) of 10%. Thus, C CAM was calculated by CAM, which was equivalent to 571,043 USD. When compared with the actual O&M cost (448,709USD) of LPRI at Phase 2 in operation for a year, the actual O&M cost was around 215 less than the O&M cost by CAM. This indicates that O&M cost of the case LPRI may have been over-or under-appropriated than the proper standard for effective operation, leading to a necessity to set a budget plan for sustainable O&M. Model evaluation with the actual O&M data accumulated for LPRI should be performed for continuous CAM validation. Here, actual O&M data should be verified by auditing so that they are reliable for use.
Accordingly, if CAM is applied for C CAM calculation from the initial planning stage, budget planning with sustainable and settled O&M will be possible. Unstable LPRI operation and possibility of idle facilities/equipment may be reduced and stop to operation owing to financial difficulties may be avoided. The study was conducted with six LPRIs due to a difficulty of securing data. However, if O&M data accumulated for LPRI in various fields are continuously collected and analyzed for model evaluation, reliability improvement as well as O&M cost prediction in consideration of the characteristics of each field will be realized. Seunghyun Son studied construction engineering and management at the department of architectural engineering, Kyung Hee University. He has being served at Mokpo National University as a lecturer since 2021. He is actively conducting research on simulation, optimization, and risk management of building projects and qualitative analysis using machine learning technique in the area of building construction. In addition, he has participated in the development of reinforcement method against seismic impact and free-form concrete panel production technology for about five years.

Sunkuk Kim studied Construction Engineering and
Management at the department of architectural engineering, Seoul National University in Korea. He had joined three Korean construction firms for 12 years. He conducted research at the department of civil engineering, Stanford University from 1994 to 1995 as a visiting scholar. Since September 1995, he has served at Kyung Hee University as a professor. He has concentrated on the research such as health performance evaluation of buildings, development of sustainable construction technology and management, simulation, optimization and risk management, and construction information technology. Especially, for about a decade, he has participated in the development of SMART frame, a sustainable structural system, and production technology of free-form concrete panels.