“Every time that month comes, I remember”: using cognitive interviews to adapt grief measures for use with bereaved adolescents in South Africa

Objective: To assess standard grief measures through cognitive interviews with bereaved adolescents in Free State, South Africa, and make recommendations designed to improve the measurement of grief in this and similar populations. Methods: Twenty-one parentally bereaved adolescents participated in semi-structured cognitive interviews about the Core Bereavement Items (CBI) questionnaire, Grief Cognitions Questionnaire for Children (GCQ-C), or Intrusive Griefs Thoughts Scale (IGTS). Results: Interviewees offered valuable insights for improving grief measurement with this population (e.g., consensus that not thinking frequently about a deceased loved one was shameful, aversion to terms including “died”). Participants were better able to apply response options denoting specific frequencies (e.g., “once or twice a week”) versus general ones (e.g., “a little bit of the time”). Questions intended to gauge grief commonly elicited responses reflecting the impact of loss on adolescents’ basic survival instead of psychological wellbeing. Conclusions: The need for psychological support is high among orphans and vulnerable children. Tools for measuring psychological outcomes can provide evidence of programme effects and guide decision making about investment. Grief measures used with adolescents in South Africa should account for the issues raised by cognitive interviewees in the study, including question and response option complexity, linguistic preferences, and cultural norms.

Psychologists have long observed that in some individuals grief fails to progress from the acute stage to resolution, even after six months or more (Boelen & van den Bout, 200;Rando, 2012,). However, manifestations of grief are culturally specific (Rosenblatt, 2008). Research from sub-Saharan Africa highlights distinct mourning rituals, ancestry beliefs, acceptable expressions of grief and other social customs that vary significantly from Western practices (Demmer, 2006;Rosenblatt & Nkosi, 2007). As such, grief scales and other measures predicated on Western norms are likely to lack contextual relevance in different settings. This assertion is consistent with previous research highlighting the importance of cross-cultural adaptation in the measurement of psychological problems including anxiety, trauma and depression (Shoeb, Weinstein, & Mollica, 2007;Sweetland, Bekin, & Verdeli, 2014;Myer et al., 2008).
Cognitive interviewing is one established technique for pre-testing survey questions, and has been recommended as a key tool for improving the measurement of psychological states in HIV research (Simoni et al., 2011). Participants are guided through a semi-structured interview regarding how they think and feel about particular survey questions and response options. The use of cognitive interviews as part of survey pre-testing is based on a theoretical model suggesting that individuals responding to a question must comprehend the question, recall the information needed to provide a response, decide if and how to use that information, and choose or generate an answer (Fowler & Cosenza, 2008;Willis, 2004;Willis, Lessler, & Caspar, 1999). In South Africa, cognitive interviews have been used to assess survey questions related to sexual violence, palliative care outcomes, risk-taking among adolescents, and children's health-related quality of life (Harding et al., 2010;Karnell et al., 2006;Sikweyiya, Jewkes, & Morrell, 2007). Another study exploring age disparity, sexual connectedness, and HIV infection in disadvantaged South African communities reported that the use of cognitive interviews allowed researchers to assess: "the clarity, comprehensibility, and cultural sensitivity of the proposed questions…improve the phrasing of questions, incorporate meaningful slang, and define suitable categorical answer options that maximise the precision and accuracy of responses" (Delva et al., 2011, p. 3).
Reports of cognitive interviewing used as a means to improve the measurement of grief and other psychological constructs, especially among children, are rare in scientific literature. One study engaged the bereaved caregivers of terminally ill cancer patients in cognitive interviews to assess a 31-item instrument measuring quality of death and dying, but did not address grief specifically (Hales et al., 2012). Another study interviewed children ages 8-17 years to obtain feedback on items proposed for a patient-reported measure of physical functioning, emotional health, social health, fatigue, pain, and asthma-specific symptoms (Irwin et al., 2009). Both these studies, however, emanated from a Western context. Increased understanding of how young people in sub-Saharan Africa respond to bereavement can offer important insights on how best to safeguard their psychological wellbeing. This study responds to this gap in the literature by engaging bereaved South African adolescents in cognitive interviews designed to provide evidence for the improved measurement of grief symptoms in this context.

Materials and methods
Potential interviewees were identified with assistance from Child Welfare Bloemfontein and Childline Free State (CWBFN & CLFS), a South African organisation offering a range of services to orphaned and vulnerable children, including school-based multi-session grief support groups led by social workers -a programme called Abangane ("Friends"). All cognitive interviewees were recruited from the pool of participants in the Abangane groups for parentally bereaved adolescents conducted by CWBFN &CLFS in 2013 and2014. Programme staff provided basic information and consent forms detailing the study purpose, procedures, and participants' rights to each potential participant and his/her parent or guardian. Forms contained telephone and email contact information for the research team, and invited parents or participants with questions or concerns to reach out at any time. Only adolescents whose parent or guardian signed and returned the form indicating consent were considered eligible to assent. The interviewer reviewed a separate assent form individually with participants just before the interview, and those who opted to sign it were allowed to proceed. Protocols were also established for interviewees exhibiting emotional distress or risk of harm. Specifically, these cases were referred to CWBFN & CLFS for follow-up by a programme social worker, who by prior agreement with the researchers ensured contact with the participant within 24 hours.
Interviews took place in office and classroom spaces at four secondary schools in both urban (Bloemfontein) and rural (Ficksburg) areas of Free State province. Twenty-one Grade 9 students aged 14-17 years, 5 male and 16 female, took part in individual cognitive interviews lasting approximately 30 minutes. Each participant was asked to answer and discuss survey questions about the loss that affected them the most and how long ago it happened, along with the items in one of three standard psychological instruments: the Core Bereavement Items (CBI) questionnaire, the Grief Cognitions Questionnaire for Children (GCQ-C), or the Intrusive Grief Thoughts Scale (IGTS). Participants were assigned to instrument type in sequence at each study site to ensure variation.
The 19-item CBI questionnaire was designed by researchers at the University of Queensland, Australia (Burnett, Middleton, Raphael, & Martinek, 1997). It was developed using questions administered to recently bereaved spouses, adult children and parents, with items derived both from the literature on grief/bereavement and from the authors' clinical experience. The original article on the development of the CBI notes that: "all the scales are essentially culture-bound" (p. 51), such that local adaption is necessary and appropriate. The GCQ was developed in 2003 by researchers in the Netherlands, "to facilitate the assessment of potentially problematic cognitions after bereavement" (Boelen & Lensvelt-Mulders, 2005, p. 291). A 20-item version designed for use with children was developed and validated separately in 2014 (Spuij, Prinzie, & Boelen, 2014). The nine-item IGTS was developed for the Family Bereavement Program run by the Program for Prevention Research at Arizona State University (Program for Prevention Research, 1999). It measures the extent to which bereaved individuals experience disruptive thoughts about a loss, and has been used to assess the impact of support groups on adolescents' levels of grief and grief-related conditions (Sandler et al., 2010). Example items and response options from each of the tools used in the cognitive interviews are provided in Table 1.
Study participants were led through a series of semi-structured prompts that called for guided reflection on questions from the grief instruments, a technique called verbal probing. The interviewer offered participants frequent, focused probes directed toward gaining specific insights related to question comprehension (e.g., "What does the phrase "to pine for" mean to you?"), paraphrasing (e.g., "Can you repeat the question in your own words?"), confidence judgment (e.g., "How sure are you that 2009 is the year your father passed away?"), and recall (e.g., "Can you tell me how you remember that you thought about your mother five times in the past four weeks?"). Each grief instrument was professionally translated by two separate translators from English into Sesotho and then back-translated by a third professional before use in the cognitive interviews. Interviews were conducted in participants' preferred language(s), typically but not exclusively Sesotho, by one of the authors who is a multilingual, native Sesotho-speaking educational research consultant experienced in psychometry and working with adolescents. Only the interviewer and participant were present during the interview. All interviews were audiotaped using Olympus DS-2 digital voice recorders. Digital interview files were transcribed verbatim and then translated into English for review and identification of emergent themes by the research team. The research was approved by the Human Research Protection Office at Tulane University in the United States and the Ethics Committee at the University of the Free State in South Africa.

Participant reactions to the cognitive interviewing process
Orientation to the cognitive interview task was markedly limited in this population of South African adolescents. Most interviewees had difficulty verbalising their thought processes when presented with a survey item, even in response to a prompt, although a few appeared to gain fluency as the interview progressed. Adolescents were informed that the purpose of the interview was to: "find out how young people describe their thoughts and feelings after losing someone close to them… to help design surveys on this topic that can be easily understood by the people who take part in them". Despite this introduction and re-orientation to the response task throughout the interview, participants often persisted in offering very brief answers to the interviewer's probes, and frequently responded to uncertainty or confusion about a question or the response task with silence. Repeated probing was necessary to obtain more extensive responses and explanations.
Sensitive subject matter and the in-depth nature of the cognitive interviews also placed a high emotional burden on some respondents. Six out of 21 participants in the study (29%) were referred by the interviewer to CWBFN & CLFS social workers for follow-up because of mild to moderate emotional distress. In these cases, the interviewer initially responded using several interpersonal methods designed to put participants at ease: suggesting that participants take a short break, offering verbal reassurances and encouragement, and reminding interviewees that they could discontinue their participation at any time without consequence.

Lexical issues
Language concerns affected nearly every aspect of the cognitive interview protocol. The psychological instruments under study were translated into Sesotho, because of its predominance as a home/first language in the study population. However, we found that many adolescents who had spoken Sesotho since birth with their families at home still experienced notable difficulty completing a cognitive interview exclusively in Sesotho. Compounding the issue, direct translation of the grief scales from English into Sesotho reportedly yielded instruments with relatively advanced vocabulary and high conceptual complexity. Most of the interviews were ultimately conducted in a mix of Sesotho and English; some participants also introduced other local languages, including Setswana and/or isiXhosa, into their responses. Particular words used in the scales presented as unreliable, or with conceptual overlap. The word "distressed" was sometimes understood by interviewees to mean "depressed", "sad" or "sorrowful", suggesting some difficulty with the translation and/or the existence of a relatively broader concept in Sesotho. Pining/yearning was most often described by respondents as synonymous with "hurt" or "miss". Several respondents also felt that "remembering" and "missing" expressed an identical concept. "Since [the deceased] died, I think of myself as a weak person…" also evidenced conceptual problems. Whereas "weak" might be synonymous with ineffectual or helpless in English, to these adolescents the Sesotho version connoted someone who was emotionally strained or simply changed by the experience of loss: Interviewer: What does it mean to say that a person is weak? Adolescent: It means he may be suffering from stress or depression.
Adolescent: It means your life has been changed by what happened. In contrast, several words and phrases from the standard scales translated demonstrably well. For questions about "images" and "thoughts" related to a loss, participants were generally able to identify and describe a difference between images as visual and thoughts as cognitive, shown in the following example: Interviewer: How is an image different from a thought? Adolescent: With an image, you can say that you see that thing, a thought is just to think without picturing. Interviewees expressed strong preferences for and against particular terminology for referring to the loss of a loved one. We found that direct translations of questions about death and the deceased originally formulated for Western audiences may be offensive and alienating to survey participants in this population. Many participants stated that "passed on" is always preferable to "died", because "died" is used exclusively to connote the death of animals and is insulting to use in reference to a person. For example: Adolescent: To die is a word that is used for animals…when it comes to people it doesn't sound good, it makes it as though the person asking these questions is rude. Adolescent: [Died] is a bad word…like dogs died. When a person is said to have died, it is bad. It is better to say passed away. A final language-related implementation issue pertains to the interpretation of information collected during the cognitive interviews. To facilitate their review and use by non-Sesotho speaking members of the research team, interview transcripts were professionally translated into English. However, this translation carried the potential to obscure the precise words and phrases being discussed, limiting the value of the data collected. For example, participants were sometimes asked to suggest an alternative (Sesotho) word for a particular notion or concept reflected in an item from the grief scale under consideration. The transcript of such an interview translated word-for-word into English would, of course, leave the reader unable to know which Sesotho word(s) were being discussed, and therefore unclear what to recommend as part of the scale's adaptation for use with Sesotho speakers. To mitigate this, translators were requested to selectively retain original words and phrases from the Sesotho versions of the instruments and the discussion when translating the interview transcripts -an unusual assignment requiring considerable operational nuance combined with a sophisticated understanding of the research objectives.

Identifying the focal loss
Questions about grief from bereavement require a reference point to a deceased loved one. In South Africa and other contexts where even children are likely to have experienced multiple bereavements it is necessary to narrow the respondent's focus to one particular loss at the beginning of the questionnaire. We explored an approach asking respondents to identify "the loss that affected you the most", anticipating that participants would reflect on the relative magnitude of their sadness and other negative emotions. Instead, those who had experienced multiple losses often based this decision on the deceased person's importance as a resource provider.
Participating adolescents also reported basic material concerns to be a primary motivator of their responses to questions about grief cognitions -unpleasant or unsettling thoughts related to the loss.
Interviewer: How did you decide that your grandmother is the loss that affected you the most?" Adolescent: …everything I needed I got from her. Now she is not here; I often don't get the things I need.
Interviewer: Which kinds of things did you include in your answer about how different things would be [if the deceased were still alive]? Adolescent: We were able to eat, there was food and things like that, every two days when [he] got paid. Other interviewees interpreted the questions about their most affecting loss differently, but still not primarily in terms of the person's emotional influence. Instead, participating adolescents focused on the level of harmony they had experienced in their relationship with the deceased: Interviewer: How did you determine that her passing affected you the most? Adolescent: She is the one I got along with most.
Interviewer: Can you repeat the question I just asked? Adolescent: Has there been a person who died with whom you got along. Psychological responses to bereavement evolve over time, making the recency of loss an important measure. Adolescents in this study were highly adept at reporting how much time had elapsed since the loss that had affected them the most. Almost without exception, participants were able to recall or calculate with certainty when their loved one had passed away. This was somewhat unexpected, as the researchers hypothesised that participants' concepts of time might be unreliable due to the psychological effects of loss, different cultural standards for timekeeping, and since many respondents were discussing a death that had occurred during their early childhood. Participants, however, used an impressive variety of response strategies that included subtracting their age at the time of the loss from their current age, recalling dates from a gravestone or funeral, and associating the death with an event that happened in the same month. Interviews included the following exchanges: Interviewer

Manifestations of grief
Many of the questions used in the GCQ-C scale that reflected a sense of guilt or responsibility did not resonate well with participants and/or were considered irrelevant to their experience. Asked how frequently they thought, "I should have seen to it that [the deceased] would not have died", several respondents indicated that since they were very young when the death happened, this question did not make sense. Interviewees asked to consider how often they blame themselves for not having "cared for [the deceased] better than I did", either indicated they did not have such responsibilities or needed clarification as to whether this included emotional or physical care or both.
Similar statements about whether they felt guilty letting go of their feelings were also confusing, as reflected in the following example: Interviewer: It is not nice toward him/her, when I will begin to feel less sad. Have you felt this way hardly ever, sometimes, or always? Adolescent: For a dead person? Findings also highlighted the influence of social desirability bias on participants' responses to questions from the CBI and IGTS about how often they perseverate on a loss. Asked, "Do thoughts of [the deceased] come into your mind whether you wish it or not?" and "[How often] do you think about [the deceased]?" respondents provided significant insights about this potential for bias. Several expressed a firm belief that thinking about or remembering someone who has died is an important sign of respect. They further acknowledged that they would be unwilling to admit not having these thoughts, even if it was the case, because, "It means [the deceased] meant nothing to your life." For this group of adolescents, "to think about the death of someone" was usually unambiguously positive -reflecting the strength of the relationship between the respondent and the deceased.
Interviewer: What does it mean to you that you think of this person's death? Adolescent: It means I loved this person.
Interviewer: If you were not thinking about the death of your father in the past four weeks, how easy or hard would it be to say so? Adolescent: That is embarrassing. Interviewer: How is it embarrassing? Adolescent: You were talking about a person who loved you. Respondents were comfortable answering simple questions about the frequency of common feelings and behaviours related to the loss. For example, items within the CBI and IGTS asking about how often they experienced loneliness, crying, bad dreams, and difficulty falling asleep were easily understood. However, longer, complex and multi-part questions posed significant problems with recall and comprehension. Many respondents seemed to be answering a different question than what was asked, typically focusing on the last part of a lengthy or multi-part question. Respondents seemed both unable to parse question components or to recall the earliest parts of a question when composing their responses to these items. Many interviewees also offered yes/ no answers to questions about the frequency of a feeling or experience, and required extensive probing and/or persistent reorientation to the response task before settling on a standardised response option.

Response options
While some response categories adapted from the Western grief scales performed well crossculturally, others lacked mutual exclusivity and/or were widely misunderstood. Matching their responses to "a lot of the time, quite a bit of the time, a little bit of the time, or never" proved particularly problematic for the South African adolescents in this study. Numerous respondents interpreted the options as referring to the duration of an experience, instead of its frequency: Interviewer: If something happened a little bit of the time, how often is that -how many times? Adolescent: When it happened for 10 or 15 minutes.
Interviewer: How is "a lot of the time" different from "a little bit of the time"? Adolescent: It is from time to time. Furthermore, some respondents felt that the length of time since the loss was somehow linked to the response options: Interviewer: When something happens a little bit of the time, how often does it happen? Adolescent: It is when it happens after months.
Interviewer: And quite a bit of the time, how often is that? Adolescent: It is when days pass and you are still thinking about it. Participants also reported difficulty using the response options "hardly ever, sometimes, or always". While "always" was intuitive for most respondents, "sometimes" was defined by several to mean "rare" or "uncommon". In addition, "hardly ever" was incomprehensible to a large proportion of interviewees, as these examples illustrate: Interviewer: What is "hardly ever"? Interviewer: What do you think "sometimes" means? Adolescent: Sometimes is when [something] happens once but not always. Interviewer: And hardly ever? Adolescent: "Hardly ever" is a mistake. By contrast, the five response options from the IGTS -several times a day, about once a day, once or twice a week, less than once a week, or not at all -were generally well-understood by participants. Many were able to provide a very specific accounting of how often they thought about the death of a loved one using these options, and to connect their estimates to actual events, as in the following examples: Interviewer: Sometimes people think about someone's death even when they don't want to. In the past four weeks, how often did you think about [deceased's] death even when you didn't want to? Adolescent: I thought of him once, because when I went to write exams he used to wish me luck.
Interviewer: How many times would you say you thought about your father's death in the past four weeks: a few times a day, once a day, once or twice a week, or less than once a week? Adolescent: About three days a week. Adolescent: Every Sunday I think about him. Interviewer: Why every Sunday? Adolescent: Because we used to go to church together. When I go to church and he's not around, then I think, this person used to be around. However, respondents had difficulty recalling and applying a time frame of reference included as part of the questions. Adolescents frequently seemed unsure how to respond to questions about their feelings in the past four weeks if the loss had happened long ago. In other instances, participants could not recall the time period in question when trying to formulate a response. The following interview excerpts illustrate these difficulties. Interviewer

Discussion
This study offers a rare example of cognitive interviewing conducted with bereaved adolescents in South Africa. To the best of our knowledge, this is the first example of cognitive interviews being used to adapt grief measures for cross-cultural use. Findings from this study provide unique insights about the measurement of grief among bereaved adolescents in South Africa as well as lessons for the implementation of cognitive interviews in this context.
Adolescents' reluctance to speak openly and in detail about their perceptions of candidate survey questions presented a major implementation challenge, and may have reflected a combination of social inhibition and limited ease with the response task. These issues are not confined to South Africa: poor task/interview orientation among cognitive interviewees in multi-cultural contexts has been noted by other researchers. A report by researchers from the US Census Bureau report on conducting cognitive interviews with non-English speakers observed that the "challenges include how to familiarise respondents…with the goals of a cognitive interview", and "the procedure in developing interview protocols and probes" (Pan, 2004, p. 4863). Findings from a recent review of cross-cultural cognitive interviewing noted similar concerns, suggesting that the efficacy of focused probes such as paraphrasing appears limited for some groups (Willis, 2015).
In South Africa, social norms favouring emotional suppression and deference, especially from children, may also affect cognitive interviews. An exploratory study of grief among professional caregivers in South Africa reported that children's feelings about death were rarely if ever discussed (Demmer, 2007). Nearly one-third of participants in this study (6/21) were referred to a social worker due to concerns about psychological distress. This high referral rate could be related to the extensive probing that often characterises cognitive interviews and the singular focus on one emotionally difficult subject. Orphaned adolescents may also be especially sensitive to grief assessment, even when responding to questions about a different focal loss (13 of the 21 parentally bereaved adolescents in this survey reported that the loss of someone other than a parent had affected them the most). Adolescents' difficulties discussing grief may also compromise the use of grief questions in surveys by provoking refusals or item non-response. Future efforts should aim to clarify the aspects of both cognitive and survey research about grief in adolescents that exacerbate or mitigate emotional burden, particularly for orphans and other vulnerable groups, to improve the respondent experience and data quality.
Furthermore, since cognitive interviewees are tasked with explaining how they experience and interpret specific words and phrases, polyglot contexts represent an exceedingly complex environment for research implementation. Willis (2015) remarks on the difficulty inherent in locating interviewers who are skilled in cross-cultural cognitive interviewing, and notes that "it is not clear at what point it is best to convert cognitive testing results from target language interviews back into the source language for consumption by (monolingual) project leaders", suggesting that this decision be based on whether lexical or conceptual issues predominate (p. 388). Likewise, we found that implementing a cognitive interview protocol with adolescents in South Africa required unusual attention to language concerns in both fieldwork and data analysis. The historical dominance of English as an instructional language in South African schools and the tendency for speakers of all backgrounds to use a mix of languages in their everyday speech, known as code-switching (Ncoko, Osman, & Cockcroft, 2000), led to interviews that included not only the target language but also English and often several other languages. In addition, several interviewees were from families that had migrated to South Africa from Lesotho, where Sesotho may be imbued with regional distinctions. We conclude that it may be advisable to adopt a multilingual approach for cognitive interviews from the outset in some settings, allowing participants to respond to the interview prompts in the language of their choice. Findings may then be used for decision making about the most appropriate language, or mix of languages, for use in the survey itself.
This study was principally undertaken to inform the development of grief measures for use in an evaluation of support group programming for bereaved adolescents in South Africa. Results helped finalise selection of suitable scales and adaptation of grief-related survey questions for use with this population by providing valuable insights about how specific words and phrases influence validity and reliability, which items may be subject to social desirability bias or other sources of error, and which response categories are most intuitive. Components of the CBI and IGTS that reflected concrete feelings and behaviours related to the loss were best understood by participants, and were adapted for use in the subsequent survey. Important modifications were made to ensure respondents' comprehension and acceptance of survey items, including the substitution of "passing" for "death", simplification of multi-part questions, and usage of appropriate local language terms reflecting states and feelings. Furthermore, frequent reiteration of the time period of reference and exclusive use of response options with concrete time periods identified as acceptable to this group of respondents were applied. Given respondents' concerns that "never" thinking about someone who had passed was disrespectful, this response option was modified to "almost never or not at all".
The cognitive interviews also offered important insights for the interpretation of survey findings. Questions meant to identify the respondent's most significant loss, and to gauge grief cognitions, did not consistently reflect the psychological consequences of bereavement. Instead, respondents often referred to the economic consequences associated with loss. These findings echo results from other qualitative research highlighting that "grief is a luxury" for many South Africans who have lost loved ones to AIDS, as the loss provokes concerns about basic survival first and foremost (Demmer, 2007). While the grief measures we tested may effectively capture psychological responses to bereavement in the more advantaged populations for which they were developed, in this context they appear less likely to reflect strictly emotional states. Results also suggest that grief symptoms may be over-reported in cultures where thinking frequently about the deceased is an important sign of respect, suggesting a possible need for higher thresholds for distinguishing problematic grief.
As psychological programming evolves in response to the specific needs of orphans, the application of research methods designed to improve psychological outcome measurement will remain important. Erroneous conclusions about participants' needs and programme effectiveness can result from survey questions that unwittingly reflect something different from what was intended intended, mean different things to different respondents, or pose response tasks participants are unable or unwilling to engage in. Cognitive interviewing and associated question pretesting techniques therefore hold significant promise for improving both programme and research quality, especially for psychological outcomes where measurement consensus may be lacking and cross-cultural variation is likely. Jeffrey W Rozelle for editing assistance. The Bangula Lingo Centre Team, led by Hannietjie Sapire and Marcus Sapire, worked tirelessly to produce high-quality interview transcripts and translations, for which we are grateful. Most importantly, we thank the study participants who shared intimate details of their lives with the research team, and their parents and caregivers for allowing us to learn from them. Support for this project was provided by USAID Southern Africa under the President's Emergency Plan for AIDS Relief (PEPFAR) through a Cooperative Agreement [AID-674-A-12-00002] awarded to Tulane University. The views expressed in this document do not necessarily reflect those of USAID or the United States government.