Product of spacing estimation of entropy for inverse Weibull distribution under progressive type-II censored data with applications

This paper seeks to estimate the entropy for the inverse Weibull distribution using progressively Type-II censored data. To reach this objective, the entropy is defined through three entropy measures, namely, Rényi, q-entropy and Shannon entropy, and two estimation methods are used to estimate them. The first approach to estimate these quantities is the method of maximum likelihood. Furthermore, and for the first time, we consider the method of maximum product of spacing to estimate the mentioned entropy measures. Also, a simulation study is carried out and two real data sets are analysed. The numerical outcomes showed that the maximum likelihood provides good point estimates while the interval estimates based on the maximum product of spacing method have the shortest confidence interval lengths.


Introduction
Entropy is the expected quantity of information for the random variable. Measuring entropy is a very important role in many areas such as statistics, physics, chemicals, economics, insurance, financial analysis and biological phenomenon. More entropy is referred to less information found in a sample. Using the approaches of probability and statistics, the concept of entropy was defined by [1] as a measure of information. This concept was enhanced by numerous other entropy measures, arising from different real applications, see for complete survey [2]. In this study, we focus our attention on three of the most popular entropy measures, namely, Rényi entropy (RE) by [3], the q-entropy (QE) by [4] and Shannon entropy (SE) by [1].
Let X be a random variable with a probability density function (PDF) f (x, ), where is the vector of the unknown parameters. Then, the RE, QE and SE of X are defined, respectively, as follow and In most of the studies, at least is unknown as well as the entropy. For this reason, estimating the parameters and the entropy has been the object of all the attention in various studies. See for example [5][6][7][8][9][10][11][12][13].
The inverse Weibull (IW) distribution occupies an essential place in representing the lifetime of components and investigating different extreme circumstances retaining rainfall, sea waves, wind speeds, earthquakes, floods queues in supermarkets, etc. The IW distribution furnishes a useful fit for non-monotone failure rates, such as decreasing and unimodal failure rates. The random variable X has the two-parameter IW distribution, denoted by IW(α, β), if its PDF and cumulative distribution function (CDF), are respectively, given as f (x; β, α) = αβx −(β+1) e −αx −β , x ≥ 0, β, α > 0, (4) and where β > 0 is the shape parameter and α > 0 is the scale parameter. This study gives a contribution to the estimation of the RE, QE and SE for the IW distribution based on progressive Type-II censoring (PT-IIC) data. We first use the maximum likelihood (ML) estimation method to estimate the entropy measures based on the invariance property of the ML method. The approximate confidence intervals (ACIs) for these quantities are also obtained. Besides this approach and for the first time, we consider using the maximum product of spacing (MPS) estimation method to estimate the entropy measures and to construct the corresponding ACIs. To compare the performance of the different estimates, a simulation study is implemented. Finally, to show the applicability of the proposed methods, two real data sets are investigated.
The rest of this paper is organized as follows: The expressions of the three entropy measures are derived in Section 2. The ML method is investigated in Section 3. In Section 4, the estimates of the entropy measures are obtained using the MPS approach. A simulation study is conducted in Section 5, and two real data sets are studied in Section 6. Lastly, the paper is concluded in Section 7.

Expressions of entropies
Let X be a random variable following the IW distribution, then, using (1) and (4) with = (β, α) , the RE of X can be written as with ζ ≥ 0, ζ = 1 and ζ ≥ 1 1+β . Figure 1 displays the plot of the RE for the IW distribution for different values of β and ζ with α = 1. Figure 1 shows that the RE of the IW distribution is decreasing as β and ζ increase.
Similarly, the QE of the random variable X can be obtained based on (2) and (4) as with q ≥ 0, q = 1 and q ≥ 1 1+β . Figure 2 shows the plot of the QE for the IW distribution for different values of β and q with α = 1. It is noted from Figure 2 that as β and q increase, the QE of the IW distribution decreases.
Also, the SE of a random variable X can be obtained from (3) and (4) as After some simplifications, the SE of IW model is given by where γ is Euler constant. Figure 3 depicts the plot of the SE for the IW distribution for different values of β with α = 1, which indicates that the SE is decreasing as β increases.

Estimation of entropies using ML method
In this section, the ML method is used to obtain the point and interval estimates of the unknown parameters of the IW distribution as well as the entropy measures based on a PT-IIC sample.

ML estimation
Due to rapid advancements in technology, researchers often want to save costs and the total time on the test. Therefore, a general censoring scheme called PT-IIC scheme can be considered. In the PT-IIC scheme, n items are put on a life test with a predetermined censoring scheme R 1 , . . . , R m , where m is the effective sample size. For more details about PT-IIC scheme, see for example [14][15][16]. See also for more details [17][18][19]. Let x = (x 1 , x 2 , . . . , x m ) be a PT-IIC sample of size m with progressive censoring scheme (R 1 , . . . , R m ). Then, the likelihood function (LF), ignoring the constant term, can be written as From (4), (5) and (9), the natural logarithm of the LF, L( ) = log ( ), can be given as The first partial derivatives of (10) with respect to α and β are given, respectively, by and The maximum likelihood estimates (MLEs) of β and α denoted byβ andα can be computed by equating (11) and (12) to zero and solve the two normal equations simultaneously. Now, based on the invariance property of the MLEs, the MLEs of the entropies H ζ , H q and H S for the IW distribution can be obtained directly from (6), (7) and (8), respectively, as folloŵ with q ≥ 0, q = 1 and q ≥ 1 1+β , and

ACIs using MLEs
To construct the ACIs of the entropy measures H ζ , H q and H S , we should first obtain the approximate asymptotic variance-covariance (VC) matrix of the MLEs. In this case, the approximate asymptotic VC can be computed through the inverse of the observed Fisher information matrix as follows The elements of (13) are given by Here, we employ the delta method to get the approximate variances of the entropy measures. Let ζ = (∂H ζ /∂α, ∂H ζ /∂β), q = (∂H q /∂α, ∂H q /∂β) and S = (∂H S /∂α, ∂H S /∂β), which are obtained at the MLEs of α and β, where and where ψ(.) =´ (.) (.) gives the digamma function,´ (.) is the ordinary derivative of (.) and φ = q . Now, the approximate estimates for the variances of the entropy measures H ζ , H q and H S can be obtained, respectively, as Therefore, the two-sided ACIs for H ζ , H q and H S at confidence level 100(1 − τ ), are given, respectively, bŷ and where z τ 2 is the 100(1 − τ/2) standard normal percentile.

Estimation of entropies using MPS method
Cheng and Amin [20] introduced the MPS method and many other authors used this method because the MPS estimators (MPSEs) retain most of the properties of the MLEs including the invariance property, see [21,22]. For more details about the MPS method, one can refer to [23][24][25][26]. Based on a PT-IIC sample, the MPS function can be written according to [27] as follows

MPS estimation
The MPS function of the IW distribution can be obtained from (4), (5) and (17) as follows The natural logarithm of (18), denoted by P( ) = log p( ), takes the form The MPSEs of α and β, denoted byα andβ, respectively, are obtained by solving the following normal equations simultaneously and Any numerical method can be implemented to getα andβ from (20) and (21), respectively. Utilizing the invariance property of the MPSEs, the MPSEs of the entropies H ζ , H q and H S can be acquired from (6), (7) and (8), respectively, as with q ≥ 0, q = 1 and q ≥ 1 1+β , and

ACIs using MPSEs
In order to construct the ACIs of H ζ , H q and H S , we first obtain the approximate asymptotic VC matrix based on the MPSEs as follows The second derivatives of the (19) with respect to α and β are given by Now, we use the delta method to approximate the variances of H ζ , H q and H S . We first obtain the quantities , where the elements of these quantities are given by (14)- (16). Therefore, the approximates variances of H ζ , H q and H S can be computed, respectively, as Several authors have elicited the asymptotic equivalence of the MPS and ML methods, Refs. [20,22,28] revealed that the MPS method also exhibits asymptotic properties like the ML method. One can also refer to [29]. Therefore, the two-sided ACIs for H ζ , H q and H S at confidence level 100(1 − τ ), are given, respectively, bỹ and

Monte Carlo simulation
To investigate the behaviour of H ζ , H q and H S , an extensive Monte Carlo simulation study is conducted. Using different combinations of n, m and R, we generate 1,000 PT-IIC samples under two sets of the    For each setting, the root mean square errors (RMSEs) and relative absolute biases (RABs) are computed for each entropy measure and reported in Tables 1 and 2. Also, the confidence lengths (CLs) as well as the coverage probabilities (CPs) of these measures are also obtained and presented in Tables 3 and 4. We refer to the RE entropy as H * ζ and to QE as H q in the simulation and real data outcomes. All numerical computations are implemented via "maxLik" package, proposed by [30], in the R statistical programming language software which using Newton-Raphson method of maximization in the computations.
For the parameter estimation, the outcomes of the simulation study showed that the ML approach provides better estimates than the MPS approach in terms of minimum RMSEs and RABs. On the other hand, the MPS approach gives more accurate interval estimates in terms of minimum ACLs. From the simulation results established in Tables 1-4, in respect of RMSEs, RABs, ACLs and CPs of the proposed estimates, we can obtain the following considerations. In general, it can be seen that the MLEs and MPSEs of H S , H ζ and H q are very good in terms of minimum RMSEs, RABs and ACLs as well as highest CPs. As n (or m/n) increases, the RMSEs, RABs and ACLs of all investigated estimates decrease while CPs increase as expected. Therefore, to get better estimation results, one may tend to increase the total (or effective) sample size. As α and β increase, the RMSEs associated with all proposed estimates decrease while the RABs increase. In most cases, when α and β increase, it is observed that the ACLs constructed based on LF and MPS approaches decrease while the corresponding CPs increase. Further, for each setting, it is evident that the RMSEs associated with MLEs decrease for both H ζ and H q when ζ and q increase. Similar behaviour is observed in the case of MPSEs of H ζ and H q . As ζ increases, the corresponding RABs of all estimates increase when (α, β) = (5,5). Also, as q increases, the corresponding RABs of all estimates increase when (α, β) = (5, 5) while these decrease when (α, β) = (3,3). In addition, for each test, ζ and q increase, the CPs using both LF and MPS approaches are mostly below (or near to) the specified nominal level.  Comparing the proposed censoring schemes I, II and III, it is observed that the MLEs (based on Scheme-I) and MPSEs (based on Scheme-III) performed better than other censoring schemes in terms of the lowest RMSEs and RABs for both true value sets. Also, the interval estimates, using observed likelihood samples, performed better based on Scheme-III compared to the other censoring schemes in terms of the shortest ACLs and highest CPs. It is also noted that the ACI estimates using the MPS approach worked effectively based on censoring schemes I and II when (α, β) taken as (3,3) and (5,5), respectively. To sum up, simulation results showed that the likelihood approach has performed superior to the other in the case of point estimation while the product of spacings approach has performed superior to the other in the case of interval estimation.

Real-life applications
To show the adaptability and flexibility of the proposed methodologies to a real phenomenon, we shall provide two numerical applications using real-life data sets in this section. The first data set (referred to as Data-I) represents the vinyl chloride data (in mg/L) obtained from clean-up-gradient monitoring wells. The data was first used by [31] and recently analysed by [32]. The second data set (referred to as Data-II) shows the times (in min) to breakdown of an insulating fluid between 19  Table 9. The ACIs (with confidence lengths in parentheses) for the real data.
ACI-LF ACI-MPS electrodes recorded at 34 kV. The Data-II was reported by [33] and recently analysed by [34,35]. The ordered values of Data-I and -II are presented in Table 5. We first fit the IW distribution to the two data sets. For this purpose, the Kolmogorov-Smirnov (K-S) and Cramér-von Mises (CvM) goodness-of-fit test statistics with associated p-values are reported in Table 6. The MLEs and the corresponding standard errors (SErs) and the goodness of fit statistics are displayed in Table 6, which show that the IW distribution is a suitable model to fit the two data set. Now, from Table 5, three PT-IIC samples are generated using different choices of m and R. These generated samples are provided in Table 7. In short, the censoring scheme R = (3, 0, 0, 3) is referred as R = (3, 0 * 2, 3). Using the generated sample, the MLEs and MPSEs of the unknown parameters as well as H S , H ζ and H q with their SErs are reported in Table 8 for the two data sets. Also, the 95% ACIs of these quantities with their lengths are listed in Table 9. From Tables 8  and 9, it is observed that the estimates of H S , H ζ and H q using the MPS approach performed better than those using the ML approach in terms of minimum SErs and confidence lengths. The data analysis shows that based on the MLEs, Sch-III performs better than other schemes. The estimated entropies have the smallest values in this case, which indicates that the data obtained using Sch-III provides more information than other schemes. On the other hand, using the MPSEs, we can observe that the data acquired employing Sch-I gives more information than other schemes. Based on these results and when using the PT-IIC data, one can estimate the entropy to decide which scheme gives more information.

Conclusion
This article considers the estimation of the Rényi, qentropy and Shannon entropy for inverse Weibull distribution based on progressively Type-II censored data. The methods of maximum likelihood and maximum product of spacing are used. The point estimators are acquired through the invariance property and the approximate confidence intervals are also computed. To examine the efficiency of the offered estimators, a simulation study and two real data sets are investigated. The numerical analysis shows that the maximum likelihood provides an acceptable approach to estimate the entropy measures and the maximum product of spacing is a good choice when the researcher wishes to obtain confidence intervals for these measures. For future work, the Bayesian approach can be considered to estimate the entropy measures employing both maximum likelihood and maximum product of spacing methods. Another future work is to perform the bootstrap confidence intervals of the entropies using the both estimation methods.