Estimation of entropy for inverse Weibull distribution under multiple censored data

ABSTRACT Entropy is a measure of uncertainty in a random variable which quantifies the expected value of the information contained in that random variable. This article estimates the Shannon entropy of the inverse Weibull distribution in case of multiple censored data. The maximum likelihood estimator and the approximate confidence interval are derived. Simulation studies are performed to investigate the performance of the estimates at different sample sizes. Real data are analysed for illustration purposes. In general, based on the outcomes of study we reveal that the mean square errors values decrease as the sample size increases. The maximum likelihood of entropy estimates approaches the true value as the censoring level decreases. The intervals of the entropy estimates appear to be narrow as the sample size increases with high probability.


Introduction
It is logically expected that smaller samples include less information as compared with larger samples; however, few attempts have been made to quantify the information loss due to the use of sub-samples rather than complete samples. Measuring of entropy is an important concept in many areas such as statistics, economics, physical, chemical and biological phenomenon. More entropy is referred to less information that found in sample.
Every probability distribution has some kind of uncertainty associated with it and entropy is used to measure this uncertainty. The concept of entropy was introduced by Shannon in 1948 as a measure of information, which provides a quantitative measure of the uncertainty.
Let X be a random variable with a cumulative distribution function (CDF) F(x) and probability density function (PDF) f (x). The Shannon entropy, denoted by H(x), of the random variable was defined (see Cover and Thomas [1]) by: Many researchers discussed the entropy in case of ordered data. Wong and Chan [2] showed that the amount of entropy is reduced when the sample is ordered. Morabbi and Razmkhah [3] studied entropy in hybrid censoring data. Abo-Eleneen [4] studied the entropy and the optimal scheme in progressive censoring of Type-II. Cho et al. [5] provided the Bayesian estimators of entropy of Rayleigh distribution based on doubly-generalized Type II hybrid censoring scheme. Cho et al. [6] provided the Bayesian estimators of entropy of a Weibull distribution based on generalized progressive hybrid censoring scheme. Lee [7] considered the maximum likelihood and Bayesian estimators of the entropy of an inverse Weibull distribution under generalized progressive hybrid censoring scheme. Bader [8] provided an exact expression for entropy information contained in both types of progressively hybrid censored data and applied it in exponential distribution. Estimation of entropy for generalized exponential distribution via record values was studied by Manoj and Asha [9].
Many applications of Shannon entropy can be found in the literature. Lee et al. [10] discussed the entropy in prices of agriculture product for various situations so that the stability of farmers' income can be increased (as known, the price of agriculture products have a great variability as they are affected by various unpredictable factors such as weather conditions, this is the reason why it is difficult for the farmers to maintain their stable income through agricultural production and marketing). An entropy model was developed which can quantify the uncertainties of price changes using the probability distribution of price changes. Chang et al. [11] applied an entropy theory to determine the optimal locations for water pressure monitoring. The entropy is defined as the amount of information was calculated from the pressure change due to the variation of demand reflected the abnormal conditions at nodes. They found that, the entropy theory provide general guideline to select the locations of pressure sensors installation for optimal design and monitoring of the water distribution systems. Rass and König [12] discussed different concepts of entropy to measure the quality of password choice process by applying multi-objective game theory to the password security problem. More applications for entropy in physics and other fields can be found in [13][14][15][16][17][18][19].
In the most life testing experiments, it is usually to terminate the test before the failure of all items. This is due to the lack of funds and/or time constrains. Observations that result from that situation are called censored samples. There are several censoring schemes, such as; Type I censoring in which the test is terminated when a prefixed censoring time arrives or Type II censoring in which the test is terminated at a predetermined number of failures. However, Type I and Type II censoring schemes don't allow units to be removed from the test during the life testing duration. Progressive censoring schemes allow for units to be removed only under control conditions. However, multiple censoring allows for units to be removed from the test at any time during the life test duration. So, multiple censored is an interval censored where the time of censored not equal. According to Tobias and Trindade [20], multiple censoring may also occur when some units on test are damaged, removed prematurely, or failed due to more than one failure mode.
This study estimates the entropy of the inverse Weibull (IW) distribution when a sample is available from the multiple censored samples. Maximum likelihood (ML) estimators and approximate confidence interval of entropy are obtained. Application to real data and simulation issues is implemented. This paper can be organized as follows. Section 2 defines entropy for IW distribution. Section 3 provides point and approximate confidence interval of entropy for IW distribution under multiple censored data. Simulation issues and application to real data are given in Section 4. The paper ends with some concluding remarks.

Inverse Weibull distribution
The inverse Weibull distribution is one of the most popular models that can used to analyse the life time data with some monotone failure rates. It can be applied in a wide range of situations including applications in medicine, reliability and ecology. Keller et al. [21] obtained the IW model by investigating failures of mechanical components subject to degradation. The IW with shape parameter α, and scale parameter β has the following PDF and CDF. and, Many applications of the IW distribution can be found in the literature. Calabria and Pulcini [22] dealt with parameter estimation of the IW distribution. Jiang et al. [23] obtained some useful measures for the IW distribution. Sultan [24] derived the Bayesian estimators of parameters, reliability and hazard rate functions using lower record values. Bayes estimators under a squared error loss function in case of Type-II censored data have been discussed by Kundu and Howlader [25]. Hassan and Al-Thobety [26] considered optimum simple failure step stress partial accelerated life test (PALT) for the population parameters and acceleration factor for IW model. Hassan et al. [27] discussed the constant stress-PALT for IW model based on multiple censored data.
The Shannon entropy of IW distribution can be obtained by substituting (2) in (1) as follows The integral (4) can be written as follows To compute the entropy in the above expression we need to find I 1 , I 2 and I 3 as follows To obtain I 2 , let u = βx −α then du = −αβx −(α+1) dx and log(x) = −1 α log u β , so I 2 will be To obtain I 3 , let also, u = βx −α , then we have Hence the Shannon entropy of IW model takes the following form This is the required expression of entropy of IW distribution which can be seen as a function of parameters α and β.

Entropy estimation
Let n components are put on life-testing experiment at time zero. X 1 hours later, all the components are examined and failures are removed. Let's r 1 failure are found, then n-r 1 components go back on test. At time X 2 , after X 2 −X 1 more hours of test, r 2 failures are found and nr 1 -r 2 units go back on test. The process is continued at time X 3 , X 4 , and so on to the end of the experiment. The likelihood function of the observed values of the total lifetime x (1) < . . . < x (n) under multiple censored data (see Tobias and Trindade [20]) is given by.
where, ε i,f = 1 the failed unit 0 otherwise and where, n f and n c are the number of failed and censored units, respectively, and K is constant. The likelihood function of IW distribution, based on multiple censored sample, is obtained by inserting (2) and (3) in (11), as follows.
It is usually easier to maximize the natural logarithm of the likelihood function rather than the likelihood function itself. Therefore, the logarithm of likelihood function is The partial derivatives of the log-likelihood function with respect to α and β, are obtained as follows and, The ML estimators of the model parameters are determined by setting ∂lnL/∂α and ∂lnL/∂β to zeroes and solving the equations simultaneously. Further, the resulting equations cannot be solved analytically, so numerical technique must be applied to solve these equations simultaneously to obtain α andβ. Hence, by using the invariance property of ML estimation, then ML estimator of H(x), denoted byĤ(x), becomeŝ Furthermore, a confidence interval of entropy is the probability that a real value of entropy will fall between an upper and lower bounds of a probability distribution. For large sample size, the ML estimators, under appropriate regularity conditions, are consistent and asymptotically normally distributed. Therefore, the twosided approximate confidence limits forĤ(x) can be constructed, such that where z ν 2 is 100(1 − ν/2) the standard normal percentile, and ν is the significant level. Therefore, an approximate confidence limits for entropy can be obtained, such that where are the lower and upper confidence limits for H(x) and σ is the standard deviation. The two-sided approximate confidence limits for entropy will be constructed with confidence levels 95%.

Application
This section assesses the performance of the estimators and provides an example to illustrate the theoretical results.

Numerical study
A simulation study is carried out to investigate the performance of the entropy estimators for IW distribution in terms of their relative bias (Rbias), mean square error (MSE), average length (AL) and coverage probability (CP) based on multiple censored samples. Simulated procedures can be described as follows: • 1000 random samples of sizes n = 30, 50, 70, 100 and 150 are generated from IW distribution based on multiple censored sample.  (14) and (15), then the ML estimate of H(x) is obtained by substitutingα andβ in (16). The approximate confidence interval of H(x) using (18) are constructed with confidence level at ν = 0.05. • Compute the average estimates ofĤ(x), Rbias, MSEs, CP, and AL.

Numerical results
Here, some observations can be detected about the performance of the entropy estimators according to Tables 1, 2      • The MSE ofĤ(x) at CL = 0.5 is smaller than the MSE ofĤ(x) at CL = 0.7.
Generally as seen from above tables that the coverage probabilities is very close to their corresponding nominal levels in approximately most cases. The maximum likelihood of entropy estimates decreases as the scale parameter of distribution decreases.

Real life data
The real-life data are a failure time of aircraft windshields that given by Blischke and Murthy [28]. They prove that the IW distribution fits this data. The failure time of aircraft windshields that measured in 1000 h are listed as follows (Table 3).
Using this data to estimate the entropy at CL = 0.7, 0.5 and complete sample (i.e. CL = 0). We find that H(x) = 3.95 at CL = 0.7,Ĥ(x) = 3.185 at CL = 0.5 and H(x) = 2.513 in case of complete sample.

Conclusion
This paper provides an estimation of Shanon entropy for inverse Weibull distribution using multiple censored data. The maximum likelihood estimators and approximate two-sided confidence limits of entropy are obtained. The performance of the entropy estimators for IW distribution is investigated in terms of their relative bias, mean square error, average length and coverage probability. Application to real data and simulation issues are provided. From simulation results we conclude that, as the sample size increases, the mean square error of estimated entropy decreases. Regarding the average length of estimators, it can be observed that, as a sample size increases the length of the interval of ML estimate of entropy decreases and coverage probability increases. As censoring level increases, the mean square error of entropy estimate increases.

Disclosure statement
No potential conflict of interest was reported by the authors.