Study on development, validity and reliability of a risk-screening questionnaire for alcohol and drug use

ABSTRACT Objective: The objective of this study is to develop a risk-screening questionnaire appropriate for cultural characteristics in detection of alcohol- and drug-use level through utilization of Addiction Profile Index (API) and perform the reliability and validity work thereof. Methods: The study was carried out on the sample of two previously made separate studies. Both samples were selected from inmates in prisons. API, CAGE Scale, Alcohol Use Disorders Identification Test (AUDIT), Drug Abuse Screening Test (DAST-10), Drug Use Disorders Identification Test (DUDIT) and clinical interview form structured for DSM-IV-TR (SCID-I) were employed in the study. Results: BAPİRT-alcohol and BAPİRT-drug questionnaires evaluating alcohol and drug abuse separately and each of which consisting of six questions were developed. Cronbach's alpha coefficients were found as 0.70 and 0.88 in the internal consistency analysis made with sample 1 data of BAPİRT alcohol and drug scale respectively. BAPİRT alcohol scale consists of two components while BAPİRT drug questionnaire comprises a single component. BAPİRT-alcohol questionnaire was found to correlate with BAPİ (Bağımlılık Profil İndeksi, Addiction Profile Index), AUDIT and CAGE by 0.94, 0.92 and 0.78 respectively. BAPİRT- drug questionnaire was found to correlate with BAPİ, DUDIT and DAST by 0.96, 0.89 and 0.81 respectively. BAPİRT for alcohol had sensitivity and specificity scores of 93.8%, and 72.5%, respectively, when using the cut-off score of 3 while BAPİRT for alcohol had sensitivity and specificity scores of 91.7% and 92.3%, respectively, when using the cut-off score of 4. Conclusions: These findings support that APIRS questionnaires are reliable and valid drug abuse screening instruments in Turkish patients with alcohol and drug use. Further studies need to be done in different clinical populations.


Introduction
One of the important methods of raising awareness as to the issues posed through alcohol and drug use and making widespread of the treatment services provided is the utilization of screening questionnaires [1,2]. It has been revealed in numerous studies that utilization of screening questionnaires to this end is effective [3]. The most important reason supporting the making widespread of screening for alcohol-use disorders is the need for quick and easy diagnostic tools which can be employed through experts who do not work in the field of addiction, whereas encounter alcohol and drug usage issues [4].
Receipt of training rate in the field of alcohol-and drug-use disorders in medical education prior or subsequent to graduation is low in our country. A study conducted in our country has revealed that physicians do not know the risky alcohol-use levels, do not have time to tackle with alcohol problems of their patients while half of the physicians have stated that they find it difficult to talk with patients as to use alcohol [5]. It has been noted that risk-screening questionnaire and brief intervention are of great importance in developing countries where addiction services are not widespread [6].
Screening questionnaires are tools utilized to identify individuals with high risk for a disorder. These questionnaires, which are also employed in the field of addiction, are utilized to identify individuals at risk for drug-use disorders and determine the course of treatment depending on the dimension of the experienced issues [7]. Screening forms are not intended to diagnose [8]. More in-depth research should be performed in positive cases revealed through screening [9].
Screening forms are useful for recognition of situations where individuals generally do not seek treatment who are unrevealed and for cases in which the symptoms are not very apparent. As such, they have to be used in cases of general psychiatric patients, prisons, emergency services and primary healthcare services [10]. It is important for a good screening form to include high sensitivity and specificity. Appropriateness of screening forms for cultures and populations is also of great significance [11].
There are many scales used to this end. The most commonly used tests to determine the risk of alcohol-use disorders is Alcohol Use Disorders Identification Test (AUDIT) consisting of 10 questions and CAGE test consisting of only 4 questions [12,13].
Different scales have been developed to screen druguse disorders. One of them is Drug Use Disorders Identification Test (DUDIT) developed similar to the AUDIT scale and consisting of 11 questions [14]. Drug Abuse Screening Test (DAST-10) consisting of 10 questions is modeled after the scale of Michigan Alcoholism Screening Test (MAST) [15]. The CAGE-AID scale adapted from CAGE scale utilized to determine the level of alcohol consumption consists of 4 questions like the CAGE scale. CAGE-AID scale adapted for drug use focuses more on addiction and has been reported not to detect problematic or risky use in non-dependent individuals [16]. It is stated that the DAST scale also includes similar issues [15].
Alcohol, Smoking and Substance Involvement Screening Test developed through the World Health Organization has been presented as a questionnaire that determines the usage risk level of all drugs including alcohol [17].
It has been reported that there is no sufficient information as to the results of the use of developed screening scales in different cultures and there are problems regarding their cultural validity [18]. For example, the question "Does your spouse (or your parents) complain of your intention to drugs?" in the DAST-10 scale or the question "Did a relative or a friend, a doctor or a nurse, or anybody else tell you their worry about your drug use or that you have to stop using drugs?" in DUDIT can be foreseen that they will marked as "yes" by everyone from our culture. Hence, it is important to develop appropriate scales for the cultures.
The objective of this study is to develop a riskscreening questionnaire appropriate for culture in detecting alcohol-and drug-use level through utilization of Addiction Profile Index (API), [19] and perform the reliability and validity study thereof.

Samples
The study was conducted with analysis of two different data used in two separate studies carried out previously. Information as to the sample from which the data were obtained is provided below. The reason for conducting the study with two different samples is the use of different questionnaires in each one of the two studies. It was aimed to develop the questionnaire with sample 1, and determine the validity and reliability with samples 1 and 2. Both samples consist of inmates in prisons. Features of samples are provided in Table 1.
Sample 1: The first sample consists of the data obtained from a survey conducted in 2012 in 10 prisons located in different cities of Turkey. The prisons included in the research were located in Ankara, Istanbul, Antalya, Kocaeli, Manisa, Eskişehir and Adana. Two of the prisons were for women, one was F-type prison (high-security prison with cells for one to three inmates based on strict isolation conditions) and the others were different types of prisons for men. A total of 1125 prisoners selected randomly from four wards in each prison were included in the study. The number of the convicts in the ward varied according to the prison. A short form of API, CAGE and AUDIT scales were administered in this research. Sample 2: The second sample consists of the data of a survey conducted in 2013 in a T-type prison located in Umraniye district of Istanbul. This prison is an institution where male convicts stay. A total of 202 convicts selected randomly from 24 wards were included in the study. A short form of API, DUDIT and DAST-10 scales were used in this research. In addition, addiction diagnosis was made through an expert using the clinical interview form structured for DSM-IV-TR.

Measures
Addiction Profile Index API is a questionnaire consisting of 37 questions developed through Ögel et al. [19] for evaluating different aspects of addiction and measuring the severity of addiction. The scale consists of five subscales including drug-use characteristics, diagnostic criteria for dependence, impact of drug use on the life of the person, craving to use and motivation to quit drugs. The short form developed later contains 22 questions. Psychometric properties of the long and short forms are similar. The correlation coefficient between API short form and API was found to be 0.96. The short form includes two factors, which include diagnostic criteria for dependence and craving questions, and impact to life and motivation [20,21].

CAGE scale
Reliability study of the scale developed by Ewing was conducted through Arikan et al. [13,22,23]. The scale consists of four questions and is answered as "Yes" or "No." The "Yes" answer given to two or more items is considered as risky use.

Alcohol Use Disorders Identification Test
AUDIT is a screening scale developed to identify risky and harmful alcohol use [12]. Saatçioğlu et al. [24] carried out validity and reliability study of the Turkish version. There are a total of 10 items in the scale and first three questions of this scale show hazardous alcohol use, the questions between 4 and 6 show dependence symptoms and the last four questions show harmful alcohol. Total score of the scale is 40 and 8 or more are recommended as indicators of hazardous and harmful alcohol use. AUDIT scores of 20 or above clearly warrant further diagnostic evaluation for alcohol dependence.

The Drug Abuse Screening Test, DAST-10
This is a test consisting of 10 questions measuring the severity of the drug-use problem as well as the magnitude of the problems [15]. Evren et al. have made the adaptation thereof to Turkish [25,26]. It has been developed for clinical screening and treatment. Cronbach's alpha coefficient value of the scale is 0.92. It is a one-component scale. When the cut-off point is taken as 4 and above DAST-10, sensitivity and specificity scores have been found to be 0.98 and 0.91, respectively.

Drug Use Disorders Identification Test, DUDIT
This is a scale consisting of 11 questions developed for screening individuals who have drug-use problems [14]. It has a Cronbach's alpha value of 0.93 and is a psychometrically reliable drug abuse screening scale. It consists of a single component. When the cut-off point is taken as 10 and above, DUDIT sensitivity and specificity scores have been found to be 0.96 and 0.94, respectively. Evren et al. [25] studied the psychometric properties of the Turkish version.

SCID-I, Structured Clinical Interview for DSM-IV Axis I Disorders
This is a configured interview schedule developed by First et al. [27] to diagnose Axis I disorders according to DSM-IV-TR diagnostic criteria. The clinical version of SCID-I was used in our study. SCID-I was translated into Turkish by Çorapçıoğlu et al. [28] to study the psychometric properties of the Turkish version.

Statistical analysis
All statistical analyses were performed using the SPSS 17.0 for Windows. Stepwise linear regression analysis was performed in the development of questionnaires. Cronbach's alpha coefficient was calculated for reliability analyses. Factor structure of the questionnaires was examined by virtue of descriptive factor analysis and varimax rotation. Pearson's correlation analysis was performed in related validity analysis of the questionnaire, while t-test was used for comparison of the mean values. Receiver operating characteristics (ROC) curve and calculation of the area under the curve were used to determine the cut-off point sensitivity and specificity of the questionnaires. ROC curve, AUDIT and DUDIT cutoff points have been taken as reference for control group, alcohol risk-screening questionnaire and drug risk-screening questionnaire, respectively. AUDIT cutoff point was taken as 8 points, while DUDIT cut-off point was taken as 10 points. These analyses were performed on two different data. The survey data utilized in each analysis have been indicated in the text.

Sociodemographic characteristics
One fourth of the total sample 1 consists of female participants, while there are no female participants in sample 2. It was noteworthy that almost half of the samples in both samples were single and more than half of the samples were primary school graduates. The number of unemployed people in sample 2 is more than five times that of sample 1.

Development of the scale
There is more than one question in the short API form, which examines the prevalence of drug use. These questions examine the frequency of use of different drugs. All questions examining the incidence of drug use have been turned into a single question by taking an average in order not to ask too many questions.
Data of sample 1 were divided into two as alcohol and drug use to distinguish alcohol and drug use. API total score examined the correlation between questions and questions showing correlation above 0.80 were selected to the risk-screening form. It has been determined that seven questions were showing sufficient correlation for alcohol and drug use. However, it has been determined that the questions varied for alcohol and drug abuse.
Questions showing high correlation for alcohol are as follows: the amount of alcohol drunk at one time during the day, the presence of daytime alcohol use, alcohol-use frequency, excess amount of alcohol usage at one time (heavy use), concern of family and society due to alcohol use, giving up a variety of activities due to use of alcohol and desire to quit the use of alcohol. Questions showing high correlation for the drug form are as follows: use of drug during daytime hours, effects of use of drug on the lives of people, drug-use frequency, emergence of withdrawal symptoms due to drug, physical and mental health problems due to drug use and giving up a variety of activities due to the use of drugs.
Stepwise linear regression analysis was conducted to develop a model for both questionnaires. AUDIT scale was taken as the basis for alcohol form in sample 1, while DUDIT scale was taken as the basis in sample 2. Two different models were obtained for use of alcohol and drug ( Table 2). The model prepared for the alcohol form calculates 49% of the total variance and consists of 6 questions. A model consisting of six questions was created in the drug form and it was determined that this model calculated 68% of the total variance.
The questionnaire which is planned for making risk screening for alcohol use was named as to Addiction Profile Index Risk Screening (APIRS)-alcohol, while the questionnaire which is planned for making risk screening for drug use was given the name of Addiction Profile Index Risk Screening (APIRS)-drug. Both of the questionnaires consist of the triple Likert response options and 6 questions and the highest possible score to be obtained is 12. Questionnaires are provided in the appendix.

Reliability analyses
Cronbach's alpha coefficients were found to be 0.70 and 0.88 in the internal consistency analysis made with sample 1 data of BAPİRT (Bağımlılık Profil İndeksi Risk Tarama Ölçeği, Addiction Profile Index Risk Scanning Scale)-alcohol and -drug questionnaires, respectively. APIRS-alcohol questionnaire item-total correlation coefficients were determined between 0.64 and 0.68 (Table 3). APIRS-drug questionnaire item and APIRS-total score correlation coefficients were found between 0.86 and 0.88.

Factor analyses
Explanatory factor analysis was performed to both of the questionnaires in sample 1 with main components method using varimax rotation. Two factors with eigenvalue above 1 were obtained in the explanatory factor analysis for APIRS-alcohol and this explains the 66.6% of the total variance (Table 4). All items were involved in one factor with their load factor greater than 0.30. The first factor is more related to drug-use features and the second factor includes questions as to abuse and addiction. One factor with eigenvalue above 1 was obtained in the explanatory factor analysis for APIRS-drug and this explains the 64.7% of the total variance. All items were included to one factor with load factor greater than 0.30.
When we examined the correlation of APIRS-alcohol questionnaire with other scales in sample 1, it was found to be 0.94, 0.92 and 0.78 with API, AUDIT, and CAGE scales, respectively. When we examined the correlation of APIRS-drug questionnaire with other scales in sample 1, it was found to be 0.96, 0.89 and 0.81 with API, DUDIT and DAST scales, respectively. All correlations were statistically significant at p < .01 level.
The average of the scores of APIRS questionnaire between men and women were found different in the analysis made with sample 1 data. APIRS-alcohol questionnaire average score was found to be 3.57 ± 3.44 in men and 1.32 ± 2.52 in women (t-value 12:38; SD = 721; p < .001) APIRS-drug questionnaire average score was found to be 3.47 ± 3.71 in men and 1.60 ± 3.15 in women (t-value 8.65; SD = 621; p < .001).
However, a difference could not be determined in the APIRS-alcohol and drug questionnaire scores between men and women who use alcohol or drug. APIRS-alcohol questionnaire average scores were found to be 6.73 ± 2.58 and 6.34 ± 2.62 in men and women, respectively, indicating that they use alcohol (t-value 0.86; SD = 41; p = .38). APIRS-drug questionnaire average scores were found to be 6.06 ± 3.21 and 6.03 ± 3.69 in men and women, respectively, indicating that they use drug (t-value 0.84; SD = 572; p = .93).
Average of APIRS-alcohol and -drug questionnaire scores who were previously treated for alcohol or drug use was compared according to sample 1 data. Average of APIRS-alcohol questionnaire was 4.53 ± 3.0 and 6.47 ± 2.8 in previously untreated and treated people, respectively, and the difference between them was statistically significant (t-value = 6.83; SD = 176.3; p < .001). Average of APIRS-drug questionnaire was 4.15 ± 3.4 and 8.30 ± 2.8 in previously untreated and treated people, respectively, and the difference between them was statistically significant (t-value = 14.03; SD = 185.4; p < .001). In the analysis performed by sample 1 data, the average of the APIRS-drug questionnaire was found to be 3.46 ± 3.62 in those not using drugs intravenously, while it was found to be (n = 47, 4.7%) 7.08 ± 3.81 in those using drugs intravenously and the difference between them was statistically significant (t value = 3.43; SD = 968; p < .001).
Average of APIRS-drug questionnaire scores in those having drug-use disorders and not having drug-use disorders as a result of the interview made with SCID-I was compared on the sample 2 data. Average of APIRS-drug questionnaire scores in those not having drug-use disorders was 1.84 ± 2.82, while it was 8.64 ± 2.75 in those having drug-use disorders (t value = −16.87; SD = 199, p < .001).

ROC analysis
Sensitivity and specificity were evaluated by ROC analysis in order to determine the operability of Alcohol and Drug APIRS-questionnaires. When AUDIT scale score 8 is taken as basis in the analysis made in sample 1 in the area under the ROC curve (AUC) in the cut-off point for 3 points for BAPİRT-alcohol is  When DUDIT scale was taken as the basis in the analysis made in sample 2 in AUC the cut-off point for 4 points for APIRS was 0.97% (p < .001), sensitivity was 91.7%, specificity was 92.3%, PPV was 86.8% and NPV was 95.2%. When DAST scale was taken as the basis in the analysis made in sample 2 in the AUC the cut-off point for 4 points for APIRS was 0.90% (p < .001), sensitivity was 86.5%, specificity is 72.6%, PPV was 77.5% and NPV was 88.2% ( Figure 1).

Discussion
This study was carried out to develop questionnaires that can make risk-level screening for alcohol and drug use and which are appropriate for the culture. Although the developed APIRS-alcohol and -drug questionnaires consist of six questions, it can be said that their psychometric properties are in a satisfactory level. It is known that scales consisting of few questions are more widespread and can be easily implemented. We believe that APIRS-alcohol and -drug questionnaires can be easily implemented in practice in their current form.
Internal consistency coefficient of APIRS-drug questionnaire is quite high while the coefficient is around 0.70 in alcohol questionnaire. Although this coefficient is not too low, it has been reported that reliability coefficients in psychometric studies are acceptable up to the level of 0.60 and reliability coefficients may be low when the number of items in scales are low and therefore the low reliability coefficients in scales with low number of items should be considered in this context [29,30]. Thus, it can be said that APIRS questionnaires have a good internal consistency in their current form.
It was determined that APIRS-drug questionnaire consists of a single factorial component according to the results of the factor analysis made, while the APIRS-alcohol questionnaire consists of two factorial components. In this form, it can be said that APIRSdrug questionnaire has similarities with DUDIT and DAST [26]. When it was taken into consideration that AUDIT scale consisted of more than one component [31], APIRS-alcohol questionnaire consisting of two components shows similarity and is understandable. It can be said that APIRS-alcohol and -drug questionnaires show high correlation with the scales taken as reference and therefore they have the characteristics of the cited scales.
The average scores of APIRS questionnaires are higher in men when a comparison is made between genders, while there is no difference between the scores of men and women who use drugs or alcohol. These findings indicate that no difference was observed between the genders after starting to use drugs or alcohol. Similar findings have been found in some other studies and although there are differences between men and women in terms of alcohol use, risky use levels have been determined to be similar [32].
It is also known that factors such as having inpatient or outpatient treatment, intravenous drug use show the severity of the alcohol-and drug-use problem [33]. It is observed that APIRS questionnaires are differential when comparison is made in terms of both variables. Similarly, APIRS-drug questionnaire average is significantly different in those who are diagnosed and who are not diagnosed with drug-use disorder with SCID. These findings can be considered as a sign that APIRS questionnaires have the feature to distinguish the severity of alcohol-and drug-use problem.
Conducting the study in the prison population can be considered as a limitation. It is known that alcohol and drug use have higher frequency in prison populations. However, a significant part of population without alcohol and drug use also stays in prisons; when it is considered that the purpose of the questionnaire is to determine the alcohol-and drug-use level, it can be said that the data obtained are sufficient for the use of the questionnaires. The research in which sample 1 was claimed to reflect the country's population therefore it can be considered that the questionnaires reflect the country's representative sample [34]. On the other hand, when it is considered that prisons are an important area of risk-screening scales, it cannot be said that selection of samples of the study from the prison is a big limitation [35]. It has been shown that risk-screening scales give similar results in different medical settings [36]. Nevertheless, we believe, it would still be useful to test the questionnaires in different populations and especially in primary health care services [9].
APIRS questionnaires have other positive aspects in addition to the appropriate psychometric properties. APIRS questionnaires consist of six questions and therefore their use by experts working outside the field of addiction would be easy. It is easier for individual to respond since response options are composed of three-point Likert scale. The APIRS scale is not an adaptation of another scale, which was developed in another culture and regulation of questions and expressions are arranged in a completely culturally appropriate manner. This scale may be easily used by the doctor, nurse, psychologist or other experts who do the clinical application. However, we believe, it would be useful to develop a guide on how to use.
As a result, we believe that APIRS-alcohol and -drug questionnaires have satisfactory psychometric properties, they are valid and reliable and their sensitivity and specificity values are satisfactory and they can be used as a risk-screening instrument in this form.

Disclosure statement
No potential conflict of interest was reported by the authors.