Measuring management practices in primary health care facilities – development and validation of management practices scorecard in Nigeria

ABSTRACT Background In low- and middle-income countries, there is scarcity of validated and reliable measurement tools for health facility management, and many interventions to improve primary health care (PHC) facilities are designed without adequate evidence base on what management practices are critical. Objective This article developed and validated a scorecard to measure management practices at primary health care facilities under the performance-based financing (PBF) scheme in Nigeria. Methods Relevant management practice domains and indicators for PHC facilities were determined based on literature review and a prior qualitative study conducted in Nigeria. The domains and indicators were tested for face validity via experts review and organized into an interviewer-administered scorecard. A stratified random sampling of PHC facilities in three States in Nigeria was conducted to assess the reliability and construct validity of the scorecard. Inter-rater reliability using inter-class correlation (ICC) (1, k) was assessed with one-way ANOVA. Exploratory factor analysis (EFA) was conducted to assess the construct validity, and an updated factor structure were developed. Results 32 indicators and 6 management practice domains were initially described. Ordinal responses were derived for each indicator. Data on the scorecard were obtained from 111 PHC facilities. The ICC of mean ratings for each team of judges was 0.94. The EFA identified 6 domains (Stakeholder engagement and communication; Community-level activities; Update of plan and target; Performance management; Staff attention to planning, target, and performance; and Drugs and financial management) and reduced the number of indicators to 17. The average communality of selected items was 0.45, and item per factor ratio was 17:6. Conclusions Despite a few areas for further refinement, this paper presents a reliable and valid scorecard for measuring management practices in PHC facilities. The scorecard can be applied for routine supervisory visits to PHC facilities, and can help accumulate knowledge on facility management, how it affects performance, and how it may be strengthened.


Background
In low-and middle-income countries (LMICs), numerous initiatives have been introduced with the aim of improving the quality and performance of primary health services. However, determinants of primary health facility performance in resourcelimited settings have not been well understood [1]. Among many factors that potentially determine the performance of primary health facilities, one of the most under-researched areas is health facility management. Several studies [1][2][3][4][5] have identified specific health facility management practices that are associated with the improvement of health facility performance including: (i) engaging and problem-solving with local stakeholders [3], particularly with community leaders [5]; (ii) building a system of accountability [4], through performance management activities [5]; (iii) motivating health workers for change [2,3,5]; (iv) building work around teams and creating a sense of belonging, trust and respect [1,3,5]; (v) providing management support [1,3,5]; and (vi) improving health facility managers leadership competency to build a supportive environment for staff [2,4,5].
Much of the research on primary health care (PHC) management practices in low and middleincome countries (LMICs) is qualitative in nature, employing case studies and realist evaluations to understand which types of management practices work in which settings [1][2][3][4][5]. Quantitative empirical studies aimed at measuring these critical health facility management practices and exploring the relationship between such practices and health facility performance are limited primarily to inpatient settings in developed countries [6][7][8][9][10][11][12][13][14][15][16][17]. Without a validated and reliable measurement tool, many interventions to improve the management and performance of primary health care facilities in LMICs are designed without an adequate evidence base on what management practices are critical for improving health facility performance. The lack of measurement tool for management practices further limits the assessment of health system strengthening (HSS) interventions in LMICs.
Performance-based financing (PBF) is an example of such HSS interventions and has increasingly been used as a major approach for improving health facility performance in LMICs. It is defined as "fee-for-service conditional on quality of care' [18]. In Nigeria, PBF was rolled out in 3 out of 36 States (Adamawa, Nasarawa, and Ondo States) to improve the quality and quantity of primary health care centers (PHCCs) services in 2014. The major design features of the PBF scheme in Nigeria include: (i) the provision of finance to health facilities on top of existing budgets based on the quantity and quality of services provided; and (ii) increased autonomy for PHCCs so that they may use the received funds to improve health services. Part of the funding received can also be allocated to motivate health workers based on their performance and responsibilities [19]. Given that PBF provides resources and autonomy for PHCCs to manage resources, management practices at the PHCCs covered by such PBF schemes will be even more important than facilities without such a platform. How PBF works at the health facility level and how health facility management influences performance under PBF have been regarded as a 'Black Box' [20]. A reliable measurement tool is required to unveil the dynamics between PBF, health center management, and performance.
The objective of our study was to develop and validate a scorecard to measure management practices at primary health care facilities under the PBF scheme in Nigeria. While the scorecard was designed to include a few measures of PBFspecific activities (such as the development of business plan and use of PBF funds), broader measures of management practices applicable to diverse PHCC programs were also included. We hope that the scorecard will be useful for the assessment of management practices at PHCCs, and improve our understanding of how PBF and similar HSS interventions work to influence health facility management in LMICs.

Methods
This study was conducted in 2014-2016. The management practices scorecard was developed through a two-step process. In the first step, we conducted a literature review to: (i) examine existing tools; and (ii) develop management practices areas and sub-areas that could be built on to develop a management practices scorecard for PBF in Nigeria. The authors reviewed literature collected through a PubMed and Google scholar search for the years 1996-2016. The search combined terms related to: (i) measurement tools; (ii) facility management practice and management competencies; and (iii) primary health care. We applied this search to literature from both developed and developing countries. We selected a tool to form the foundation for the scorecard based on the following criteria: (i) the tool had gone through a validation process that linked management measures to health facility performance; and (ii) the indicators and measurement approach were consistent with findings from a qualitative case study at PHCCs in Nigeria [5] that identified community engagement, performance management and staff management as key management practices. Further, based on over 40 relevant publications in peer-reviewed journals on management competencies, we developed a list of important management practice areas for PHCCs. We also reviewed the PBF literature to identify additional management practice areas for managing PBF schemes in health facilities. We extracted all the elements of management practices mentioned in the relevant publications, and organized them into a shortlist of management practice areas and sub-areas under each area. We also selected relevant indicators under each sub-area based on the review of existing tools.
As the second step, we developed additional indicators, scoring criteria using ordinal responses, and a scoring grid based on the key findings of the qualitative study [5]. For example, the case study research found that high-performing PHCCs carry out frequent outreach activities, visit targeted households in each outreach, change services based on the feedback from patients, and carry out many strategic activities to recruit and retain patients such as creating incentives for women to deliver at PHCCs, individualized follow-up of pregnant women, year-end celebrations. These practices were converted to indicators to measure community/client engagement, and ordinal responses for each indicator were developed by comparing relevant practices between high versus low-performing PHCCs observed in the qualitative study.
We tested the face validity of the scorecard through a review of health system personnel in Nigeria including staff of the National Primary Health Care Development Agency (NPHCDA), and State Primary Health Care Development Agencies (SPHCDAs) in Adamawa, Nasarawa, and Ondo states of Nigeria; PBF technical assistance consultants; the World Bank's Nigeria health team; and health systems experts at the Johns Hopkins University. We then divided the scorecard into two separate questionnaires to be administered (i) to officers in charge (OICs) and (ii) non-OIC health workers. We pilot-tested the questionnaires in six PHCCs under PBF in Nasarawa state that were not included in the main phase of data collection and revised the questionnaires for clarity and interpretation based on feedback from the pilot-test. The revised questionnaires, evaluation criteria and scoring grid were scripted into electronic templates on handheld devices to facilitate data collection and reduce errors in scoring and aggregation of the scores.

Sampling and data collection
We proposed a sample size (N) of 111 PHCCs to explore a scorecard with 32 indicators or questions (p) for a N:p ratio of 3 as described by Arrindell and van der Ende [21] to be adequate for demonstrating the validity and reliability of questionnaires designed for identifying latent constructs with alpha level of 0.05. We selected the 111 PHCCs from 457 PHCCs that have been implementing PBF in the three Nigerian States (Adamawa, Nasarawa, and Ondo). We used a stratified random sampling technique to allocate the 111 PHCCs (Adamawa: 54, Nasarawa: 21 and Ondo: 36) based on the number of PHCCs under the PBF scheme in each State.
In the EFA presented below, average communality of selected items was 0.45. The items per factor ratio (p:r ratio) in the model presented in Table 4 is 17:6. For EFA with low communality case (less than 0.4), an empirical study suggested that N = 100 and N = 200 are needed to have over 95% convergences for 20:3 and 10:3 p:r ratios respectively [22]. They also found that for p:r ratio 10:3 or 20:3, a sample size (N = 60) could still result in over 99% convergence if the level of commonality is wide or high. As the commonality of this EFA result is higher than 0.4, 17:6 p:r ratio may require a little over 100, which is consistent with the sample size of our study, N = 111.
Twelve interviewers were recruited to form 6 survey teams of 2 interviewers. The interviewers were trained for five days, including two days of fieldtesting of the questionnaires. Data collection was conducted during April and May 2016. The questionnaires were administered to the OIC of a selected PHCC, and a randomly selected non-OIC health worker who had worked for the PHCC for more than one year. The non-OIC health worker interviews were carried out in an isolated room or outside the PHCC without the presence of other staff. After having administered each questionnaire, the two interviewers scored the PHCC using the scoring grid and criteria separately without consultation with each otherindividual scores were used for assessing inter-rater agreement. Thereafter, the two interviewers discussed the scores for each question and agreed on the final consolidated scores. Interviewers took short notes of responses to all questions, and the final scores were reviewed and validated by the lead author and a contracted data collection firm by referring to the short notes.
The original management practices measurement tool (from which indicators were derived for this scorecard) assumed interviews would be conducted by individuals with education and work experience in healthcare, such as masters or doctoral degrees in public health, medicine, or business administration to increase the reliability of the score [8]. Given that such resources are not easily available in developing countries, revisions were made to the scorecard so that: (i) scoring criteria for open-ended questions were defined clearly to reduce potential inter-rater reliability issues; and (ii) in addition to questions for OICs, we developed questions for non-OIC health workers to assess practices by OICs and reduce social desirability bias.

Data analysis
We assessed descriptive statistics (arithmetic mean and standard deviation) for all indicators included in the scorecard for the 111 PHCCs. Thereafter, we assessed the inter-rater reliability of the scorecard using inter-class correlation (ICC) (1, k) for two judges and not the same judges for all PHCCs, with one-way ANOVA. The range, mean, standard deviation and histograms of the scores for each indicator were also examined to understand the distribution of the scores.
We then carried out exploratory factor analyses (EFA) to assess the construct validity of the scorecard. We estimated principal components for the indicators in the scorecard and selected the number of factors to retain based on eigenvalues (greater than 1) and a scree plot. A factor analysis with Iterated Principal Factors (IPF) method and PROMAX rotation was used to redefine the factors to improve their interpretability. We reviewed factor loadings to look into groups of indicators loaded on the same latent factors and to drop items with low loading (loading < 0.40) [23]. Also, cross-loading items with values ≥ 0.32 on at least two factors were deleted, especially if there were other items with factor loadings of 0.50 or greater [24]. The result of the EFA was used to develop an updated factor structure with a smaller number of items. We compared the updated factor structure with the original management practices scorecard we developed, and literature review results, to examine whether the updated structure can measure important management practices comprehensively. All analyses were conducted using STATA version 14 (Stata Corp).

Scorecard development
Step 1: Literature review for tool selection and identification of management practices areas The number of tools to measure management practices of health facilities was very limited. We found three instruments for health facilities in developed countries that had been validated, whose results were published in peer-reviewed journals -(i) the Management Practices Measurement tool [6][7][8]; (ii) Baldrige healthcare criteria for performance excellence [9]; and (iii) a questionnaire to measure organizational attributes of primary care practices [25]. We did not find any evidence that the instruments designed for use in LMICs [26][27][28][29] had been validated. had been validated. Of the tools reviewed, we found the Management Practices Measurement tool most relevant to build on, to measure management practices of health facilities under PBF in Nigeria. It has similar components to the management areas identified by Mabuchi et al. [5] including, performance management, staff management and motivation, and community engagement. Also, it uses an external assessment by trained personnel based on a specific scoring grid and criteria, which addresses capacity constraints and self-reporting bias at PHCCs.
Of the about 40 publications and reports reviewed, 13 intended to define health facility managers' practices. Table 1 synthesizes the findings from these 13 papers on management practices in health facilities. It identifies seven different management practice areas from practices requiring hard skills (such as financial management) to practices drawing upon much softer skills (such as communication and team building). In addition, the review of elements of health facility autonomy under PBF developed by Fritche et al. [19] and NPHCDA [30] highlighted one further areapharmaceutical management. Based on these findings, we described 8 key management practice areas for the scorecard, including problem solving, communication, staff and team management, planning, performance management, relationship building and resource mobilization (which was redefined as community/client engagement), financial management and pharmaceutical management. We operationalized the 8 key management areas using relevant indicators from the original Management Practices Measurement tool identified through the literature review. We did not consider indicators that focused on hospital management because these were not relevant for PHCCs in LMICs. Based on findings from the qualitative case studies previously conducted [5], we subsumed communication under the staff management practice area and decided to drop the problem-solving practice as it overlaps with other areas such as planning and target setting and performance management.
Step 2: Development of scorecard Table 2 provides the areas and indicators of the developed management scorecard (Full scorecard available in Appendix A). The scorecard included 32 indicators grouped into 6 broad management practice areas: (i) community/client engagement; (ii) stakeholder engagement; (iii) staff management; (iv) planning and target setting; (v) performance management; and (vi) use of funds and financial management. Each area was broken down into sub-areas (e.g. 'performance tracking' and 'performance review' for the performance management area) and indicators (e.g. 'visualization of performance data' and 'staff attention to performance' for the performance tracking sub-area). Ordinal responses derived for each indicator were assigned value of 1, 2 and 3 respectively, resulting of aggregate score range of 32-96 from 32 indicators for each PHCC.

Mean and distribution of scores for each scorecard indicator
Complete responses for all indicators on the scorecard were obtained from 111 PHCCs, including 222 respondents (111 OICs and 111 non-OIC workers).
Of the 111 PHCCs initially selected, two PHCCs in Ondo state refused the interview because they had just started PBF, and six PHCCs in Adamawa were not accessible due to an insurgency. They were replaced by other PHCCs through random selection. As shown in Table 2, mean scores of the 32 indicators ranged 1.5-2.77, with average 2.16. The standard deviation of the scores ranged 0.33-0.83, with average 0.6. Validity and reliability of the scorecard Experts review suggested high face validity of the scorecard. Table 3 presents the result of the EFA with PROMAX rotation. Five factors had eigenvalues more than 1, and screeplot shows flattened line at the Factor 7. Given that the commonality was higher for the six-factor model than for the five-factor model, and that our qualitative and literature review described above suggested a six-factor model, we chose the sixfactor model dropping 12 indicators with loadings less than 0.4. Since all items except for S5, S22, S31, and S32 had uniqueness higher than 0.50, we kept the items with uniqueness higher than 0.50 as long as their factor loadings were 0.40 or above. Also, three cross-loading items with values ≥ 0.32 on at least two factors were dropped (S9, S17, S20) from this model. As a result, 17 indicators were kept for analysis. Table 4 Table 4. For the ICC, the correlation among mean ratings for each team of judges is 0.94, showing high inter-rater reliability.

Discussion
We developed a novel scorecard that measures management practices in PHCCs in Nigeria. We highlighted financial management, community and stakeholder engagement as key additional elements of management practices for PHCCs in LMICs in addition to the Management Practices Measurement tool developed by Dorgan et al. [6], Bloom et al. [7], and McConnell et al. [8] for use in high income countries. Our scorecard also introduced a more specific definition of scoring criteria than the original instrument, and questions for non-OIC health workers to enable local data collectors to rate practices and to reduce social desirability bias. These are new and original features of the scorecard that would facilitate its adaptation to capacity-constrained contexts in LMICs.

Original scorecard vs. EFA results
The developed scorecard was further refined through the EFA. The EFA reduced the number of items from 32 to 17. It also provided a different grouping of items from the originally proposed management practices scorecard based on our qualitative study [5] and literature review. Table 5 compares the originally proposed management practices scorecard with findings based on the EFA results. There are a few notable differences. First, although community engagement is to some extent covered by the latent factor A and B in Table 3, a set of items related to building the relationship with and attracting patients (S1-S4) were not included in the EFA results (see Table 5, right-hand column 'New Groupings'). These dropped items were however highlighted as key differentiating factors of PHCC performance under the related qualitative study [5]. This may suggest that there are slight differences between factors that relate to PHCC performance and factors that represent health center management (suggested through EFA). Hence, the factors that explain health center management on the one hand and health center performance on the other, may be overlapping but not identical. For example, drugs and financial management are not a factor that directly differentiated high and low performers in the qualitative case study [5], whereas this is an important element of health center management based on the EFA. Likewise, outreach, household visits, and strategies to attract patients may not be a direct element of health center management, though they are key specific approaches that influence the performance of the PHCC. It is noted that community/client engagement is not included in the management practices measurement tool by Dorgan et al. [6], Bloom et al. [7], and McConnell et al. [8], and synthesized key elements of critical primary health facility management (Table 1). Another possibility is that the indicators for community/client engagement did not measure the practices sufficiently well. These indicators were developed specifically for this scorecard and could have been flawed. For example, the frequency of outreach last week (S1) may be too short a time period to get a reliable picture of outreach, or this measure may put too much emphasis on frequency and not enough on the quality of outreach. Further formative research, elaboration and testing of the scorecard questions may be needed in this area.
Second, most of the items related to Staff Management in the original scorecard were dropped, and the items kept were assigned to separate groups (i.e. 'A. Stakeholder engagement' and 'D. Performance management'). This is not consistent with the synthesized key elements of critical primary health facility management (Table 1) where activities to assign appropriate roles and responsibilities, create opportunities for learning, motivate and coach health workers, and promote cohesion and teamwork were highlighted as a key element of health facility management. The Management Practices Measurement tool also has 'Talent management' in the instrument [6][7][8]. This may suggest the challenge of scoring such activities in the scorecard, and points to the need for further review and adaptation. At least, however, some dimensions of staff management, such as responsiveness to staff feedback as a part of broader stakeholder engagement, and handling of poor performing staff as a part of performance management are covered in the final factors.

EFA results vs. literature
The EFA results are consistent with the developed management practices scorecard and literature in other settings. Latent factors 'C. Update of plan and target', and 'D. Performance management' and the items grouped in these factors are consistent with the Management Practices Measurement tool. Also, the factor 'F. Drugs and financial management' is consistent with the synthesized key elements of critical primary health facility management (Table 1), as well as the key management practices for the health facilities to manage the PBF  [19]. The latent factor 'E. Staff attention to plan, target, and performance' is a different grouping from the original management practices scorecard. However, this demonstrates the importance of communication, involvement, and incentives to motivate staff to be attentive to plan, target and performance, which is consistent with findings in the qualitative case study [5] and the Communication element of the synthesized key elements of critical primary health facility management (Table 1).

Value and use of the research
This research added significantly to the literature on health center management in developing countries. A careful review of prior studies and application of existing instruments with adjustments, expert review of the scorecard, and high inter-rater reliability are signals of the validity and reliability of the developed measurement approach. The EFA also provided a refined management practices scorecard, despite some differences between the results that it offered and findings from the literature and the related qualitative case study [5].
Capacity building of health facilities is included in most primary health care interventions in developing countries. However, there has been no instrument to help assess management practices and provide critical feedback to improve health facility management todate. Recent systematic reviews of researches on primary health care systems in LMICs suggest that major research gaps exist in how to improve facility management [31], and that routinely used performance measurement and management strategies are implemented without sufficient knowledge of their effects [32]. This scorecard can help address these critical gaps thus strengthening primary care services. The resulting scorecard is relatively simple, encompassing just 17 different indicators, and includes clear scoring criteria, meaning that it would be relatively straightforward for the central and local government officials to apply the scorecard as part of routine supervisory visits, and not just as part of a research project. This scorecard was used in Nasarawa state of Nigeria to measure baseline and follow-up management scores of the PHCCs under PBF funded by the World Bank to design/guide and measure the result of management strengthening interventions. This indicates high acceptability of the scorecard. Wider application of this scorecard would in turn help to further strengthen the scorecard and guidance associated with it.

Limitations and areas for further study
As suggested above, one of the limitations to this research and the scorecard is that some of the scorecard questions and scoring criteria, notably those related to community/client engagement, and staff management would benefit from further investigation and refinement. Given the limited literature seeking to assess management practices quantitatively, we were unable to compare our findings to other studies from LMICs.
The scorecard was designed to serve the needs of primary health care facilities under PBF or similar schemes that provide autonomy and funds for the health facilities to improve health services. and it was designed for use in the Nigerian context, drawing in particular on a qualitative case study previously conducted in Nigeria [5]. In order to understand how this scorecard may apply in other contexts, both with and without PBF, further studies may be required using confirmatory factor analysis (CFA) to assess the model fit of the scorecard. Adaptations would also be necessary to assess management practices in settings where there are more limited management autonomy and discretionary funds. Differences in health system structure and function, for example the structure of drug supply systems, or the extent of decentralization, may also influence items and constructs to be included in the scorecard.

Conclusion
While the management scorecard presented here is undoubtedly an initial attempt to develop a measurement tool that can be used across primary health care settings in low resource environments, we believe that further investment in this objective is warranted. The review by Rowe et al. [2] suggests that management approaches consistently had moderate to large effects on health worker performance. It is time to dismantle and investigate the black box to better understand facility management, how it affects performance, and how it may be strengthened. Ideally management scorecards would be used on a repeated basis, so that primary health care managers as well as central and local government policymakers can see how performance improves over time. Such repeated use may warrant reconfiguration of the scorecard at different time points to respond to the dynamic changes in management practices impacting performance over time. Learning from the related literature on balanced scorecards (e.g. Peters et al. [33], Khan et al. [34]) may be relevant in this regard. Health Care Development Agency (SPHCDA) colleagues who took time to advise authors and discuss the scorecard with authors: Dinesh Nair, Benjamin Loevinsohn, Ayodeji Oluwole Odutolu, Gyorgy Fritsche, Amaka Okechukwu Opara, Olalekan Olubajo, Adamu Ohagenyi, Mathias Murekezi, and Segun Oguntoyinbo. Hanovia Limited team collected and compiles data for this study.

Author contributions
• Shunsuke Mabuchi is the corresponding author, and lead literature review, developed the management practices scorecard, administered data collection and wrote the paper as the main author. • Kunle Alonge advised on research methods, including the research design, analyses (in particular exploratory factor analysis) and interpretation of the results. He also reviewed and revised the manuscript significantly. He also drafted the abstract. • Yusuke Tsugawa advised on research methods, and carried out the exploratory factor analysis and reliability analysis using Stata. He also reviewed and revised the manuscript significantly. • Sara Bennett advised on research questions, research methods, analyses and interpretation of the results as the main advisor for the corresponding author. She also reviewed and revised the manuscript significantly, including the substantial write-up of the introduction and conclusion parts.

Disclosure statement
The health results innovation trust fund (HRITF) managed by the World Bank provided generous funding to this research. The funding allowed data collection in Nigeria including the hiring of a local consulting firm to collect data. The authors have not been paid to write this article by any of the agencies. We have full access to all the data in the study and had final responsibility for the decision to submit for publication.

Ethics and consent
The IRB review was exempted -moved to the NR/NHSR/ NE state (IRB00006836).

Funding information
The health results innovation trust fund (HRITF) managed by the World Bank provided generous funding to this research.

Paper context
In low/middle-income countries, there is no validated and reliable measurement tool to measure health facility management, and many interventions to improve primary health care (PHC) facilities are designed without an evidence base on what management practices are critical. This paper developed and validated a scorecard to measure management practices at PHC facilities in LMICs. The scorecard is applicable for routine supervisory visits to PHC facilities and can help accumulate knowledge on facility management. Staff did not provide any opinion in the last one month.

ORCID
Less than a month ago.
Less than a week ago.

13
(Continued ) There is no update on the business plan since it was developed, or it has not been updated more than a year. The business plan was updated within 12 months.
The business plan is updated within last 3 months.

19
Business plan content (Observation of the business plan) Does the plan describe targets, problems in achieving the targets, key activities to address them, timeline, and budget for them?
The business plan includes none of them. The business plan includes some of them. What are the key issues and activities explained in the business plan?
The staff do not know about the business plan. The staff have some knowledge about the activities in the business plan (e.g., priority activities) The staff is fully aware of the content of the business plan (at least target, issues, and key activities).

Target update
When is the last time the targets were updated?
There is no update on the targets since they were set, or it has not been updated more than a year. The targets were updated within last 12 months.
The targets were updated within last 3 months.  Quantity and quality of services are recorded regularly (monthly) and visualized (e.g., on a wall as a graph or There is no regular performance review meetings at the PHCC. The performance review meeting is held at the PHCC every month or quarter. However, WDC and community leader, and other stakeholders do not participate or their participation is not regular. The performance review meeting is held at the PHCC every month or quarter, and staff, WDC, and community leaders participate in the meeting.

Performance review discussions
To review the contents of the meeting: -What was discussed in the last meeting? (Probe: performance? agreed actions to improve performance?) There is no discussion on performance, and actions to improve performance. The OIC explained clearly how performance was reviewed, but agreed actions to improve performance were not very specific.
The OIC explained clearly how performance was reviewed, and agreed actions to improve performance were very specific. The financial records are not updated. The financial records are updated with some delays, and monthly balance is not available.
The financial records are updated each time there is expense, and revenue and expenses are balanced every month.

31
Financial record content (Observation) Can we see the financial record of the PHCC? -Check whether incomes and expenses for the last month are recorded clearly, and total balance of the month or quarter is calculated correctly.
The incomes and expenses are recorded poorly or not recorded, and the calculation does not match. The incomes and expenses are recorded OK.
The incomes and expenses, and the total balance of the month or quarter are calculated correctly.