Justified true belief theory for intelligence analysis

ABSTRACT This article explores how leading theories from a key area of epistemology, the philosophical study of knowledge and belief, can support best practice in intelligence analysis. The article specifically examines how the three leading theories for establishing justification for beliefs: evidentialism, process reliabilism, and indefeasibilism, could be utilized for practical use in intelligence analysis. The consideration of these theories of Justified True Belief (JTB) in an intelligence context offers an opportunity for adjusting and building upon existing recommended methodological approaches, specifically the Assessment of Competing Hypotheses (ACH).


Introduction: intelligence, truth, and epistemology
The pursuit of truth and knowledge is widely accepted as a fundamental part of intelligence. Almost all definitions of intelligence include the goal of acquiring knowledge in some capacity, irrespective of whether absolute knowledge is considered a plausible goal, and what actions the knowledge is intended to enable. 1 However, the cost of analytical failure when it comes to intelligence can be devastatingly high. It is no surprise that a significant proportion of research in intelligence studies has focused on intelligence failure, and how it can be avoided. Analytical judgements are a key component of most identified ways in which intelligence failure can occur: the influence of cognitive bias, the inadequate collection of information from which judgements are formed, an inaccurate focus of analytical efforts, and the inaccurate communication of analytical judgements. Different solutions have been suggested for combating analytical failure in intelligence. Some argue for a methodological approach using bespoke structured analytical methods. 2 Others argue that intuition or critical rationalism should be relied on instead. 3 This division of opinion is known colloquially as the art versus science debate. Research in this area has drawn on psychological studies from the late twentieth century, information theory and anthropology, with some attempts to adapt theories from the philosophy of science. 4 The science aspect of this debate arguably holds greater weight. The most important argument for using structured analytical methods rather than relying on intuition in intelligence analysis is that intuition is significantly prone to error. Whilst intuition may in some cases allow a greater efficiency in decision-making, it also leads to inaccurate judgements and distorted perceptions of reality. Whilst using structured analytical methods might not be enough to counteract cognitive biases, by making the justification for our judgements explicit, errors in judgment become more visible. Justification for individual and collective beliefs is a crucial aspect in ensuring best practice in intelligence analysis. If no justification is provided for analytical judgements, there is no rational basis to accept them. Providing justification gives others an obvious basis for taking our beliefs to be true.
However, guidance on what valid justification for beliefs includes, and how much weight different kinds of justification can carry in the reduction of uncertainty in intelligence is an underdeveloped area of academic research. On what basis should the credibility of information used to inform judgements be judged? How much confidence can be placed on intelligence assessments when they are being used to inform decision-making? Outside of an intelligence context, these are key questions that philosophy tries to answer, and the answers that philosophy has come up with have strong potential to inform intelligence analysis best practice. The philosophical field of epistemology is dedicated to identifying standards for establishing truth, knowledge, and justification for our beliefs. Although the potential benefit of drawing from epistemology for intelligence analysis best practice have long been recognised, attempts to explore how this could be done represent an extremely discrete area within intelligence studies. 5 The main questions that epistemology looks to answer are: what are the sources of knowledge, what are the mechanisms by which knowledge can be gained, what is the structure of knowledge, and when can we be justified in believing something to be true? Sources of knowledge look at identifying the types of information on which beliefs are based, such as the perceptive (empirical) senses, reason, memory, human testimony, and introspection. 6 Epistemological theory in this area is mainly focused on identifying which of these sources of knowledge can be taken as being more reliable in providing justification for beliefs.
Epistemological research focusing on the mechanisms by which knowledge can be gained looks at which individual methodologies that have been devised for particular types of truth can be taken as philosophically valid, such as mathematics, logic, and the empirical sciences. Research focusing on the structure of knowledge attempts to identify if there is a type of knowledge that does not require validation, that can stand as a foundational set of truths.
Efforts at exploring epistemological theory in an intelligence context so far have focused on the implications of research looking at the structure of knowledge or mechanisms through which knowledge can be gained. James Bruce in 2008 explored different mechanisms by which knowledge can be gained in intelligence, and intelligence sources of knowledge and justification. 7 Herbert, and Hoffding and Vrist Ronn have explored whether epistemological arguments regarding the structure of knowledge can be applied in an intelligence context. 8 Questions about the structure of knowledge in epistemology centre upon determining whether there is a class of basic beliefs which do not require any justification for believing to be true. This is to avoid a never-ending requirement of justification for beliefs which the truth or falsity depends on the truth of a separate belief. 9 Vrist Ronn further explores epistemological arguments regarding the structure of knowledge, exploring Susan Haack's theory of foundherentism as a suitable approach for intelligence. 10 There has yet to be a substantive attempt at exploring the possibilities for best practice advice from epistemological theory looking at the fourth key question in epistemology: what standards can be used to determine when we can be justified in believing something to be true? 11 This area of epistemology, known as Justified True Belief (JTB) theory, seeks to identify general standards for determining a sufficient degree of justification for true beliefs that can be applied to all types of sources of information and all types of mechanisms through which knowledge can be gained. The focus on a sufficient degree, rather than an absolute degree, recognises the near impossible task of identifying pure knowledge -truths that can be guaranteed not to be proved false. Absolute truth is an impractical goal. Knowledge is understood as something that is imperfect and open to revision. We have to make decisions based on the closest approximation we can find to the truth, accepting that it might be wrong.
Ensuring appropriate justification is identified for analytical judgements allows decision-makers to determine how much confidence can be placed on them. Knowledge acquisition in intelligence covers a wide range of topics and can draw from all types of information. As such, a general rule of thumb for determining when beliefs are sufficiently epistemically justified that can be adapted to suit a wide range of analytical needs offers potential benefit for supporting best practice.
There is a strong case for looking to epistemology for guidance on determining justification for beliefs in intelligence analysis. The standards promoted in epistemology as being the most appropriate for determining truth apply universally and should arguably be seen as aspirational standards in all areas of knowledge acquisition both within and outside of philosophical discourse. However, knowledge acquisition in intelligence faces significant challenges, for example: deception, misinformation, the fabrication of intelligence for financial gain, and active attempts by adversaries to keep information hidden. Whilst these issues are not unique to intelligence, it is arguable that they pose a higher degree of challenge in intelligence than in other contexts. Intelligence analysis presents less than ideal circumstances for gaining knowledge or justification for beliefs. Implementing the standards promoted in JTB theory in an imperfect reality inevitably poses several challenges. This article outlines these challenges and suggests pragmatic solutions for how they can be overcome. Whilst some of these challenges may be insurmountable, the potential benefit of overcoming them makes the attempt worthwhile.
The first part of the article provides an introduction to JTB theory and the potential benefit that JTB has to offer intelligence analysis, outlining the implications of each in an intelligence context, and discussing key challenges and objections to each theory. The second part of the article explores ways in which these obstacles could be overcome, how the recommended methods of determining epistemic justification in these JTB theories compare to existing and currently recommended approaches, and what suggestions they offer for new or different approaches to supporting best practice in intelligence analysis. The third part of this article outlines suggestions for improving existing training and guidance for intelligence analysis and an initial proposal for a new methodological approach based on these theories, centred on improving the structured analytical technique, the Analysis of Competing Hypotheses (ACH).

Part one: justified true belief theory in an intelligence context
To be able to know something, we need to have an agreed understanding of what is true from what is false, and how we can be certain that what we believe to be true is in fact true. The most obvious truism we can say about knowledge is that knowledge has to be true. But this truism does not allow us any insight into how to determine the standards by which we can legitimately say we know something to be true. It is by no means an easy task to identify what the standard for knowledge is, although it may be easy to see when we fail to achieve it.
A second truism about knowledge that is widely accepted is that we have to believe that something is true in order to claim to know it. However, it is also possible that an individual can believe something to be true when it is actually false. Belief on its own cannot stand as a criterion for knowledge. One of the main arguments that has been put forward in epistemology is that justification is essential to knowledge: You have to have an adequate basis for your beliefs to be justified in believing them to be true. This principle is the foundation of JTB theory. 12 The notion that knowledge requires some form of justification for beliefs is widely accepted within epistemology. 13 Furthermore, when knowledge cannot be obtained, justified beliefs are looked to for informing behaviour and decision-making. Some philosophers have argued that the requirement for attaining knowledge is so high as to be virtually impossible to reach, and therefore justified beliefs are more important as an epistemological standard than pure knowledge. 14 As such, JTB theory does not face significant theoretical challenge. 15 It looks for a sufficient standard for truth, rather than demanding an absolute one, offering a pragmatic approach to seeking knowledge. However, there is significant disagreement as to what the criteria for justification for beliefs should look like.
The notion of justification covers a lot of conceptual ground, though not all concepts or types of justification are relevant to knowledge, for example, consequential justification and veritable justification. Consequential justification is when a subject claims or is claimed to be justified in believing that a proposition is true because the consequences would be beneficial to the believer or other individuals. Veritable justification is where a subject is justified in believing that a proposition is true because it actually turns out to be true. However, it is theoretically possible to be justified in believing that a proposition is true even if the consequences of the belief were not positive for yourself or others, or if the belief turned out to be false. The type of justification that epistemology is concerned with is epistemic justification. Epistemic justification is where an individual has good reason for believing a proposition to be true. Whilst many questions that philosophy concerns itself with can seem a world away from the realities of intelligence communities, the question of epistemic justification is of fundamental importance. The purpose of intelligence analysis is to determine what is happening or what might happen in the real world that is of defence and security concern, based on the best information available: It seeks to identify the best approximation to truth from available evidence. Epistemic justification matters to intelligence analysis because it is the link between the ideas of truth and evidence. Intelligence analysis is fundamentally an exercise in epistemic justification. But can best guidance from the world of philosophy help inform or support epistemological standards in intelligence analysis?
A large part of contemporary epistemological research has focused on trying to identify the requirements for being justified in your beliefs without it being possible to acquire true beliefs by chance. 16 Epistemologists' attempts to provide this have taken different approaches. 17 The current leading theories are evidentialism, process reliabilism and indefeasibilism. These theories are not without flaws and require development to be successfully utilised for an intelligence context. Not all have obvious or practical solutions, and some may be insurmountable. A degree of pragmatism is required for adapting the epistemological ideals they promote to provide real world solutions.
Reliabilism is the most well-developed justified true belief theory. Alvin Goldman developed the first version of the reliabilist theory of knowledge in 1976. 18 He argued that an individual is justified in believing a proposition to be true if their belief was acquired by a reliable method. 19 If you can demonstrate that the process by which you arrived at a belief is reliable in providing true beliefs, then you can be justified in taking your belief to be true.
Most versions of reliabilist theory are concerned with the processes by which beliefs are caused and are known as process reliabilism. 20 Process reliabilism seeks to establish the reliability of the method by which a belief is caused. 21 In practice, this refers to the cognitive processes of belief formation. For example, if a belief was based on empirical information and the cognitive process used to process the empirical data was inductive logic, then process reliabilism will apply to how reliable in general we can consider inductive logic in providing truth. Process reliabilism would look to assign a degree of general reliability in comparison with other cognitive processes, such as mathematical calculations, deductive inferences or logic, and intuition. It is not contentious to work on the assumption that some cognitive processes will be generally more reliable at producing true beliefs than others.
Process reliabilism faces several objections as a philosophical theory. The most significant objection is that it is too vague: it provides no guidance on what degree of reliability a cognitive method must provide to qualify as being reliable. There are methods that are theoretically reliable in general, but in specific instances fail to provide correct information. The notion of a cognitive process being 'reliable' is too vague to be used as a standard of justification for beliefs without additional precision. However, requiring a cognitive method to be totally reliable imposes an impossible standard. An agreed standard needs to be established for what degree would make a cognitive method sufficiently reliable if the expectation of being totally reliable is impractical.
Most objections with process reliabilism in epistemology concern beliefs that are formed purely through a process of introspection. It's very complicated to try and establish how such beliefs can be deemed reliable without justification relying solely on beliefs held by the individual in question. 22 Whilst it is unlikely that intelligence analysis would face this particular issue, the wider implications of this objection affect how process reliabilism can be of use to intelligence. The term cognitive process refers to any mental process involved in gaining knowledge and comprehension. This encompasses a wide range of higher-level brain functions, including, for example, thinking, imagination, perception, problem-solving, language, memory and judgment. The mental processes to which process reliabilism refers face a significant degree of individual difference. No two brains are the same: One person's memory may be more reliable than another, for example. More importantly, process reliabilism is exclusively concerned in determining the reliability of processes that are not open to external observation. Some occur at a subconscious level rendering them beyond internal observation. In an intelligence context, applying process reliabilism in its original form would require a way to measure whether reliable cognitive processes had been followed in the formation of assessments. However, the inability to witness and objectively measure human cognitive processes, and the high likelihood that human cognitive processes will be subject to subconscious error and bias, poses a challenge to practical implementation. 23 How do you measure the rationality of a human mind and form objective processes for ensuring a reliable cognitive process is followed for every belief the mind forms and holds true?
It might be tempting to solve this problem by measuring how often analysts turn out to be correct in their judgements: to confer a truth ratio to their analytical capabilities. 24 This approach would be an application of verification of truth and has been incorporated into some versions of process reliabilism. However, this approach would not only place analysts in an unfairly pressured work environment but would be theoretically invalid from an epistemological perspective. Correct predictions do not necessarily entail that they were formed by reliable cognitive processes -being right does not mean that your beliefs were justified.
Another key issue facing the practical implementation of process reliabilism is how to deal with counter-arguments that have also been formed through a reliable cognitive process. 25 For example, two intelligence analysts arrive at substantially different judgements based on the same body of intelligence information. Both analysts' judgements are logically valid, but both cannot be true at the same time. How would process reliabilism decide which judgment has greater epistemic justification?
Although many process reliabilists' have tried to get around this issue by adding the qualification that a belief can only be considered to be formed through a reliable process if it cannot be defeated by counter-argument (borrowing a key element from the theory of indefeasibilism), there is no practical guidance yet provided within epistemology on how to distinguish when a counterargument can be objectively considered to have defeated a belief if both the original belief and the counter-argument were formed through a reliable cognitive process. To do this you would need to find a way to rank different types of reliable cognitive processes. This problem is also pertinent in an intelligence context for having to deal with a high risk of deception and misinformation, issues which render determining the credibility of competing sources of information as particularly challenging. If two pieces of information can be said to have been formed through a reliable cognitive process, but directly contradict one another, then there has to be a way to determine which one has a higher degree of justification for being believed to be true. For example, two separate pieces of human reporting providing witness accounts of the same event, but that provide directly opposing accounts.
Adapting process reliabilism for an intelligence context clearly faces challenges. A reasonable accord between epistemological theory and practical implementation could be found. However, process reliabilism cannot be adapted in isolation: it requires synthesis with evidentialist and indefeasibilist principles to account for its theoretical and practical shortcomings.
Evidentialism argues that for an individual to know a belief is true, they must have sufficient evidence to support the truth of the belief. This is arguably an obvious truth. It is generally agreed both within and outside of philosophy that truth is at least in part, and if not exclusively determined by how closely it matches and accounts for an independent reality. However, being an obvious truth doesn't mean there is an obvious and easy way to implement it for intelligence analysis. Adapting evidentialism for intelligence faces similar challenges to the adaption of process reliabilism.
The greatest challenge that evidentialism faces in a philosophical context is the current lack of agreement within epistemology as what constitutes evidence, and whether a difference of degree can be permitted to the term: The nature of evidence has yet to be determined, let alone whether it can be held to allow a difference of degree. This can be seen in the use of the concept outside of philosophy. The term 'evidence' is interpreted in different ways, depending on the type of judgements being formed. For example, evidence in the natural sciences takes the form of empirical data. However, in terms of legal concerns, evidence can include forensic information and witness testimony. In everyday terms, the concept of evidence covers all sources of information, and therefore covers a range of quality and degree of confidence in terms of whether something can be taken as truth or not.
However, for evidentialism to be applied in terms of sufficient evidence, where a total body of evidence is not practical, evidence has to be able to be recorded in degrees. As such, the greatest challenge of applying evidentialism in an intelligence context is agreeing what qualifies as evidence, and what constitutes enough evidence to justify beliefs as being qualified as truth, without specific guidance from philosophical theory. This is quite a substantial undertaking. However, it is by no means impossible. Academic fields have established their own standards for what stands as sufficient evidence, allowing the distinction between theory and fact. The challenge is how such standards could be developed, adopted, adapted, and agreed for intelligence.
A second key challenge to establishing a standard for sufficient evidence is the issue of comprehensiveness of available information. Judgements made in intelligence analysis are highly likely to contain gaps in information, even if based on sources of information that are accepted as providing evidence. Further, it may be difficult to distinguish a degree of evidential status for competing sources of justification in cases of intentional deception, where methods of direct independent corroboration are not feasible. If two pieces of information from different sources provide contradicting accounts, but both sources count as evidence, how do you choose which one is of higher quality in providing justification for believing to be true?
In an intelligence context, evidentialism is concerned with which sources of information can be accepted as generally providing sound information, and which can be said to provide a higher quality of soundness than others. For example, it could be argued that information gained through empirical processes would qualify as evidence. This is similar to the stance taken by classical empiricism, whereby evidence is taken as sensory perceptions or sensory data. 26 In an intelligence context, this could apply to information obtained from human sources, and information gained through the interception of human communication. However, it is questionable whether these types of sources of information should be taken generally providing a higher quality of evidence over other sources, due to the risk of deception, errors in perception, and the limited opportunities for intelligence collectors to corroborate the information received through these sources.
Some philosophers argue that certain types of knowledge, such as mathematics, also count as evidence, thereby arguing for introspection and reason to be included as evidential sources of justification. However, it is questionable whether introspection and reason could be considered to be sources of information that reliably produce information of a quality that could be classed as evidence. Our cognitive faculties are far from infallible. The same argument also applies to the inclusion of memory as an evidential source of information -our memories are unlikely to be fully accurate. Some sources of information cannot be reasonably held to be reliable producers of evidence. Sources of these types would need to have independent corroboration from sources of information that are considered to provide reliable evidence. Further, a bar would need to be set for determining how much independent corroboration would be needed to stand as sufficient justification.
The indefeasibilist response to the question of what constitutes justification for beliefs is that to be justified, a belief should not be susceptible to valid counterarguments: that it is undefeatable, or indefeasible. 27 Indefeasibilism also has specific implications for intelligence. In philosophical terms, a valid counter-argument is a theoretically valid example in which the argument under consideration is clearly not true. Counter-arguments in philosophy can be, and often are, purely hypothetical. However, using hypothetical counter-arguments to falsify beliefs outside of philosophy is not viable. As most propositions in an intelligence context are conditionally true -their truth being dependent on what actually happens in the world -valid counter-arguments have to be based on contradicting or inconsistent evidence, or valid alternative explanations for the evidence available, rather than thought experiments. Outside of philosophy, indefeasibilism comes down to how well a belief fits available evidence against competing valid hypotheses. The issue poses a similar problem to ranking counter-arguments in process reliabilism -how to rank and assign different degrees of quality to competing arguments or interpretations of evidence when all competing arguments are valid.

Part two: what can justified true belief theory offer intelligence?
The three JTB theories presented each enshrine a different approach for establishing epistemic justification for beliefs. Process reliabilism advocates for established reliability in producing true propositions from the cognitive processes that form them. Evidentialism advocates for a sufficient degree of evidential support. Indefeasibilism advocates for the inability for beliefs to be defeated by valid counter-arguments. Each theory presents challenges for practical utilisation. Taken in isolation, each theory is arguably an inadequate method for setting standards for epistemic justification in intelligence analysis. However, combined, these three JTB theories offer some benefit. They are not mutually exclusive approaches. The key to overcoming most of the individual challenges outlined in this article lies in combining their approaches. There are several inter-connections between the challenges faced by the three theories and their potential solutions for practical use as epistemological standards for intelligence analysis.
Process reliabilism requires a way to allow the cognitive processes that form beliefs to be subject to independent corroboration, to 1) determine the reliability of these methods and 2) to compare competing valid propositions against available evidence. This can arguably be achieved by the combination of two approaches, both based in philosophical theory. The first approach is to enshrine formal logic as a necessary component of intelligence analysis and assessment. The second is to combine process reliabilism with evidentialist theory. Whilst it is unfeasible to identify if the underlying cognitive processes that formed an analytical judgment could be objectively deemed reliable, formal logic can stand as an external validation method. Using formal logic prevents arguments from being automatically false and provides an objective measure of rationality. 28 It provides a neat and comprehensive way of establishing the reliability of the underlying process of forming propositions. Logic is already widely recognised as a reliable method of identifying true propositions, providing that logically valid propositions can be shown to be an accurate reflection of the real world. Logic alone cannot establish truth. Logic is purely a tool for examining the linguistic structure of formal arguments. It is not and cannot be a measure of whether an argument is sound. This is where the second proposed approach comes in. Process reliabilism requires a form of evidentialism to establish the reliability of cognitive processes: reason requires independent corroboration from available evidence. Combining the best elements of process reliabilism -a focus on how propositional beliefs can be rationally validated -and evidentialism -a focus on establishing a beliefs' evidential support -has been explored in epistemology. The theory of evidentialist process reliabilism 29 argues that justification for propositional beliefs requires both a reliable cognitive method and supporting evidence. Combining this approach with formal logic avoids the challenge of objective observation of internal cognitive processes.
Indefeasibilism requires a way to identify the degree to which competing valid beliefs are supported by available evidence. Resolving this issue could be achieved by a combined approach of formal logic, evidentialism, and eliminative induction. As validity in philosophy ultimately refers to logical validity, indefeasibilism would advocate for the inclusion of formal logic as a necessary part of intelligence analysis. If a competing hypothesis is logically valid, then it needs to be given equal consideration. Formal logic can establish which competing hypotheses are required for inclusion in an analytical process. However, for indefeasibilism to work, the identification and inclusion of logically valid competing hypothesis needs to be a necessary part of the analysis and assessment process.
Indefeasibilism is also inextricably connected to evidentialism. Valid counter-arguments outside of philosophical thought experiments are competing hypotheses that explain available evidence. Following the principles of indefeasibilism in intelligence comes down to the interpretation of available information judged to be sufficiently credible to be used to validate assessments: how well analytical judgements explain the relevant credible information. This requires the evidentialist approach. If reliabilism and indefeasibilism cannot parse between competing hypotheses, then a degree of explanatory power between hypotheses and their sources of justification is also required: how well does the available evidence fit alternative explanations. In this context, when the standards of reliabilism and evidentialism have been satisfied, that a belief is logically valid and that its truth is sufficiently supported by available evidence, it is a combination of indefeasibilism and evidentialism that determines which competing belief has greater degree of justification from its evidential support. When the degree of evidential support is equal between competing valid hypotheses, evidentialism plays yet another role. Establishing truth in these circumstances requires attempts to falsify competing propositional beliefs by seeking new evidence that the proposition cannot account for. This approach combines evidentialism with the principle of eliminative induction, which prioritises attempts to disconfirm hypotheses over attempts to confirm them. This ensures efficiency in efforts to establish epistemic justification through evidentialism. It is far easier to identify if a belief is false than to prove a belief is true, as it only takes one instance of a disconfirming fact to establish falsity, whereas establishing truth requires all relevant evidence that can ever exist to support the truth of the belief. However, this approach still requires a definition of what qualifies as a sufficient degree of evidence to stand as epistemic justification for believing a proposition to be true or to be false.
Evidentialism requires a way of reliably assessing and communicating the comparative degree of quality of information. In an intelligence context this means providing the best quality guidance for how to assess the credibility (epistemic justification) of individual pieces of intelligence information (evidence), and how to identify when a piece of information provides evidential support for identifying the truth value of a proposition: whether a belief is true or false. The biggest challenge in grounding intelligence analysis in the standards of epistemic justification espoused by the three JTB theories presented centres on the ability to find a satisfactory middle ground between the ideal of evidentialism and the challenges faced by intelligence that render this ideal impractical or impossible. Intelligence analysis poses significant unique challenges that are not encountered to the same degree in other forms of analysis: poor quality and incomplete evidence basis, deliberate attempts to conceal or provide false information, and limited options for independent corroboration. Further, the standards of determining epistemic credibility for beliefs in evidentialist terms need to be able to account for multiple types of sources of information. Beliefs formed in intelligence analysis can be based on any type of information, and any combination of information types, and could therefore require a holistic approach to evidentialist epistemic justification.
For intelligence analysis to follow the principles advocated by the JTB theories presented here, the following would need to happen: 1) Making full use of formal logic as a tool for forming and testing the epistemic reliability of beliefs; 2) Enshrining the consideration of more than one hypothesis as a necessary component of intelligence analysis and assessment; 3) Enshrining the approach of disconfirmation when seeking additional intelligence information; and; 4) Ensuring that the advice provided to intelligence practitioners in establishing the epistemic justification for available evidence -the credibility of individual sources of information -is of the highest possible quality. But do any of these suggestions provide novel approaches for intelligence communities?
Using formal logic as a way of judging the epistemic quality of intelligence assessments is not a novel idea. Previous guidance documents for intelligence analysis produced by intelligence communities have attempted to utilise classical logic as a form of verifying intelligence judgements. 30 However, the information provided in internal guidance documents often reflects a misunderstanding of logical theory and does not go into sufficient depth to explain how the process of establishing logical validity works in practice. 31 The best utilisation of logic to date in analytical guidance has been providing examples of logically valid and invalid syllogisms, for analysts to use to try and identify illogical arguments in their own work. 32 However, for formal logic to be of use to intelligence, it needs to be applied in a less superficial capacity. Logic only applies to beliefs that take the form of arguments: beliefs that have a conclusion that is logically entailed by at least two premises. Logic cannot be used to apply to beliefs that are presented only as conclusive statements without the underlying premises made clear. However, it is likely that analytical judgements will often be comprised of multiple conclusions without explanation of their underlying premises. Outside of philosophy, the examination of arguments by unpacking and examining the premises that support them is not a common approach. It is questionable that intelligence assessments would be produced or communicated in a way that made the premises of each individual conclusion explicit.
Expecting intelligence analysts to apply formal logic in their assessment work, or for formal logic to be properly utilised in peer review processes is contentious. Examining beliefs for logical validity is a time-consuming and complicated process that requires formal training in classical logic. It would be impractical to require intelligence analysts and practitioners involved in peer review processes to train and work as classical logicians. However, practical alternatives are feasible. The rules of classical logic can be incorporated into software applications that automate this process. The development of such information technology for use in intelligence communities would be a substantial project. However, the intellectual capital required for such an effort is already established, and technological advancements could easily utilise knowledge of logical processes into analytical software tools. If assessments can be produced or examined for logical validity in the drafting stages using information technology, the peer review process would be left to focus on challenging evidential epistemic justification, rendering the process more targeted and efficient. The real challenge in such a project would be getting around the likely security restrictions regarding the development or introduction of new technological software within an intelligence community. These are logistical challenges, and are arguably feasible to overcome, providing intelligence communities are willing to embrace the idea.
The consideration of multiple hypotheses and the principle of disconfirmation in intelligence analysis and assessment is also not a novel idea. The consideration of multiple hypotheses is recommended in multiple analytical guidance and training documents. Further, the consideration of multiple hypotheses and the principle of disconfirmation are founding principles of a key structured analytical technique that has been recommended and taught in Western intelligence communities since the 1990ʹs, the Analysis of Competing Hypotheses. 33 However, there is no evidence to indicate that the consideration of multiple hypotheses or the principle of disconfirmation have been made a compulsory part of intelligence analysis and assessment processes within intelligence communities. This would require substantial cultural and bureaucratic shifts that would have significant consequences for intelligence collection as well as analytical approaches.
Guidance on judging the credibility of intelligence information has evolved considerably in modern history and has seen significant contribution from academic thought. 34 Whilst much of the tradecraft used by intelligence communities in verifying sources of information is not available for public scrutiny, some scrutiny can be applied to the guidance given for processing finalised source information to form holistic assessments. Some training guides that have been produced internally by Western intelligence communities have been released as open-source documents. However, many of these documents do not provide specific or detailed guidance about judging the credibility of intelligence information. Further, those that do contain several theoretical flaws from an epistemological viewpoint. 35 Only two publicly available Western intelligence training manuals currently provide instructions on how to determine the credibility of information: Krizan (1999) and the United Nations Office on Drugs and Crime (UNODC) (2011). These include: the use of verification of truth as a measure of reliability of individual source's reporting history, 36 and inaccurate guidance on assessing logical validity. 37 This indicates that there are opportunities for improving the quality of advice provided within intelligence communities on judging the credibility of information from an epistemological perspective.

Part three: suggestions for new methodological approaches based on JTB principles
As we have seen, looking to the key principles of indefeasibilism, evidentialism and processreliabilism for ways to improve intelligence analysis best practice provides offers four specific recommendations. The first is to make better use of use formal logic as a way of externalising and verifying that intelligence assessments are formed through a reliable cognitive process. Second, is to ensure multiple logically valid hypotheses are taken into consideration in forming intelligence assessments. Third, is to ensure that the principle of eliminative induction is followed in attempting to seek intelligence information to compare against equally valid hypotheses when existing information provides an equal degree of evidential epistemic justification. And fourth, is to ensure that guidance on judging the epistemic justification of intelligence information is of the highest theoretical quality.
These approaches can be utilised as a way of improving current training in intelligence analysis provided in Western intelligence communities. They can be combined to provide a new methodological approach centred on developing a new and enhanced version of the ACH structured analytical technique. ACH is arguably the most widely recommended methodological approach to intelligence analysis. It is widely advocated within intelligence analysis and in defence and security companies in the private sector. It is one of two structured analytical techniques that has been recommended for use in intelligence analysis that involves the process of evaluating information against hypotheses. 38 However, its current versions have significant issues from a JTB perspective. JTB can provide suggestions for improving on existing versions of the technique and for developing new versions.
The original ACH method has eight basic steps. The first, is to develop at least three mutually exclusive competing hypotheses that aim to cover all reasonable possibilities and create a matrix or table where each hypothesis forms the individual columns. The second step is to identify evidence that is relevant to the hypotheses. The third step is to score each piece of evidence against the hypotheses. ACH does not dictate which scoring system or systems should be used to do this, allowing any scoring system to be used, provided that it matches relevant information against specific hypotheses. The fourth step is to refine the matrix or table by adding new hypotheses, or combining hypotheses together, where appropriate. The fifth step is to interpret the scores. The hypothesis that is consistent with the largest volume of information is deemed the one that has the highest degree of epistemic justification, and the hypothesis that is inconsistent with the largest volume of information is deemed to have the least degree of epistemic justification. The sixth step is to review and readjust the scoring if required. The seventh step is to report the conclusions of the ACH. The eighth step is to identify indicators for future observation. This requires a set of two indicators, one on future events that would support the hypothesis assessed to be most likely to be true, and a list of indicators that would reduce the degree of epistemic justification that this hypothesis is true. 39 There are some basic improvements to guidance on using this original version of ACH that JTB would advocate. Most current versions of ACH either do not include credibility of information as a ranking system in the ACH process, or do not incorporate credibility of information scores into the overall scoring system. In this capacity, most versions of ACH do not adequately take epistemic justification into account. 40 The use of ACH in intelligence communities should be on the basis that the recommended versions of the technique necessarily include two ranking systems: one that assesses the evidential quality (credibility) of information and one that assesses the degree to which the information supports the truth value of the hypotheses.
Formal logic should be incorporated into the ACH design. Hypotheses included in the technique should first be vetted for logical validity. This requires competing hypotheses to be broken down into their underlying premises. Only logically valid competing hypotheses should be included for consideration. Further, there should be no requirement that the hypotheses included be mutually exclusive. This requirement restricts the use of ACH to exam questions that only examine one possible answer. Removing this clause widens the utility of ACH for application in answering questions that have multiple choices, as the truth values of each choice would be examined as separate hypotheses. Finally, the ranking systems of evidential quality, and degree of evidential support should be applied to the truth value of the individual premises that logically entail the hypotheses. The current versions of ACH do not require the premises that support each hypothesis are made explicit or examined separately. Provided that the hypotheses included in the process are logically valid, true premises will be unable to logically entail false conclusions, even if the truth of the conclusion is conditional upon being true in the real world.
Reformulating a version of ACH with these suggestions offers a pragmatic way to align intelligence analysis with the ideals enshrined in JTB theory for how best to establish epistemic justification for beliefs. However, to be able to provide the best alignment with these principles, this approach needs to be paired with adequate guidance on how to judge the degree of evidential epistemic justification of source information. Arguably the greatest challenge facing the development of new methodological approaches is the current lack of appropriate guidance provided in ACH manuals or training and guidance documents as to what methods or standards should be used to assess the credibility of information. This is not an issue with the concept of using credibility of information as a scoring system for processing information in an analytical task, but whether the guidance on how to identify epistemic justification for information is theoretically appropriate. This is not an issue that is specific to ACH but applies to intelligence analysis and assessment in general.

Summary and conclusions
JTB theory provides aspirational standards for improving analysis in intelligence as well as outside of the intelligence world. However, whilst it is widely agreed that epistemic justification is a necessary component of having the right to be sure in our beliefs, there is limited agreement in JTB theory as to the best approach to establish it. Process reliabilism, evidentialism and indefeasibilism each offers a strong argument for what principles should be used to establish epistemic justification: that the method that produces the beliefs in question is reliable in its ability to identify truth values; that beliefs should have sufficient support from available evidence, and; that justified beliefs should have a higher degree of evidential support than valid competing hypotheses.
Looking to JTB theory for practical use in intelligence analysis and assessment faces several challenges. Rather than trying to choose one of these three approaches, or trying to identify which should have primacy, all three can be equally embraced as fundamental guiding principles of best practice in intelligence analysis. Taken in unison, the combined strengths resolve many of the individual issues that have been identified with the three theories in philosophical discourse. Blending these with the additional methodological approaches of formal logic and eliminative induction resolves even further issues.
Looking to the three leading JTB theories provides four specific recommendations for improving the quality of best practice for intelligence from a philosophical perspective: to make better use of use formal logic as a way of externalising and verifying the reliability of assessments; ensuring that the formulation of intelligence assessments takes into consideration multiple logically valid hypotheses; the utilisation of eliminative induction in comparing evidence against competing logically valid hypotheses, and ensuring the highest quality of guidance on judging the epistemic justification of intelligence information. The first three of these recommendations can be utilised to improve existing recommended approaches to intelligence analysis, specifically in the creation of a revised design of the Analysis of Competing Hypotheses (ACH) methodological approach.
However, the fourth recommendation from JTB theory poses a challenge that is less simple to overcome: the challenge of ensuring that the highest possible quality guidance in judging the epistemic quality of intelligence source information is provided to intelligence practitioners. Identifying ways in which epistemology can assist this challenge should constitute a key part of future research agendas focusing on integrating intelligence with philosophical theory. Intelligence." 9. Hoffding and Vrist Ronn, "The Epistemic Status of Intelligence." Hoffding and Vrist Ronn's article also put forward the argument that it is possible to acquire knowledge through intelligence analysis, and that it shouldn't be necessarily assumed that all attempts at acquiring knowledge in intelligence will fall short of epistemological standards for identifying truth. 10. Vrist Ronn, "(Mis)Informed Decisions." 11. Ibid. Vrist Ronn touches on justification theory, arguing that criteria of justification for beliefs in intelligence should be a system of degree, rather than a binary choice between unjustified and justified beliefs, but does not explore what such a system would look like in detail. 12. JTB is alternatively known as the Tripartite Analysis of Knowledge. 13. Every traditional account of knowledge includes belief as a necessary component, and almost all traditional accounts of knowledge include justification as a necessary component. 14. This is a central component of Alfred J. Ayer's, The Problem of Knowledge. In intelligence analysis the debate as to whether knowledge is possible or impossible is framed in the debate regarding the distinction of 'secrets' and 'mysteries', where secrets are held to be knowledge that is possible but difficult to obtain, and mysteries are held to be knowledge that is impossible to obtain. For further reading on this debate see Marrin, "Evaluating the Quality of Intelligence Analysis." 15. There are two notable outlier arguments that do not accept the requirement for justification for beliefs. 20. The other main formulation of reliabilism, modal reliabilism, focuses on applying reliabilism in a multiverse (alternative universe) context. This approach has limited utility for application to intelligence analysis, as it is predominantly concerned with conceptual metaphysical theory. 21. For further reading, please see Goldman, "What Is Justified Belief?," and Goldman and Beddor, "Reliabilist Epistemology." 22. For in-depth discussions on this topic, please see Vogel, "Reliabilism Levelled"; Cohen, "Basic Knowledge"; Bon Jour, "Externalist Theories"; Cohen, "Justification and Truth"; Pollock, "Reliability and Justified Belief"; Feldman, "Reliability and Justification"; and Foley, "What's Wrong with Reliabilism?." For a general overview, please see Goldman and Beddor, "Reliabilist Epistemology." 23. For further reading on cognitive biases, and how they might impact intelligence analysis, please see Heuer,

Notes
Psychology of Intelligence. 24. This approach was originally proposed as a way of establishing process reliabilism. 25. Beddor, "Process Reliabilism's Trouble with Defeat." 26. For further reading on classical empiricism, see Quine, "Two Dogmas"; and Quine, "Epistemology Naturalized." 27. For a detailed discussion of this topic, please see Paxson, "Knowledge: Undefeated True Belief." 28. Using formal logic also removes the individualistic capacity of the original formulation of reliabilism, allowing a measure of rationality to be conferred to beliefs formed by group assessment. 29. For leading arguments supporting the synthesis of reliabilism and evidentialism, please see Comesaña, "Evidentialist Reliabilism"; and Goldman, "Toward a Synthesis." 30. For example, the United Kingdom's Cabinet Office, "Quick Wins." 31. For example, the 6 × 6 system for judging the credibility of information in the United Nations Office on Drugs and Crime (2011) includes 'not illogical' as a separate criterion from logical validity (United Nations Office for Drugs and Crime, Criminal Intelligence Manual for Analysts, 26-28). Another common error present in guidance documents is the classification of abductive reasoning as a form of classical logic alongside inductive and deductive syllogisms. Inductive and deductive reasoning are the two forms of logic originally devised by Aristotle. Abduction is a theory of scientific method and is concerned with how to determine the ability of competing hypotheses to account for and explain available evidence. Abduction has not yet been successfully reduced to a system of logic. 32. For example, the United Kingdom's Cabinet Office, Quick Wins. 33. For example, the United Kingdom's Cabinet Office (2015) recommends multiple structured approaches for generating and evaluation multiple hypotheses (Cabinet Office, Quick Wins). The ACH method, developed by Richards J. Heuer Jr. for the Central Intelligence Agency in the 1990ʹs was inspired by the principle of disconfirmation as developed by Karl Popper in weighing evidence against competing hypotheses. For further details on the ACH method, please see Heuer and Pherson, Structured Analytical Methods. 34. A key example of this is demonstrated in the contribution to intelligence analysis best practice by Sherman Kent, a professor of history who served at the US Central Intelligence Agency. 35. For a detailed analysis of the epistemic quality of advice provided in these training guides, please see Whitesmith,Cognitive Bias,[81][82][83][84][85][86][87][88][89][90][91][92]. Both Krizan's "Intelligence Essentials for Everyone" and the UNODC' publication in 2011 use reliability as a measurement applied to the credibility of human source reporting. The measure of reliability that both recommend is judged by the accuracy of a human source's reporting history, with no reference to the cognitive processes which produced the human source's claims. This is veritable justification: where a subject is justified in believing that a proposition is true because it actually turns out to be true. Veritable justification does not stand as an acceptable measure of epistemic justification, as it does not follow that just because an individual's belief actually turned out to be true means that he or she formed that original belief based on an adequate justification. It is theoretically possible to be justified in believing that a proposition is true even if the belief turns out not to be true (Krizan, "Intelligence Essentials," 26, and UNODC, Criminal Intelligence Manual for Analysts, 26-28). 37. Please refer to endnote number 31 for details. 38. The second SAT recommended that includes evaluating evidence against hypotheses is subjective Bayesianism.
However, this technique is considered irredeemably flawed within philosophy as it equates truth values with probability calculations. This is a highly contentious claim and is widely rejected. For further reading, please see Talbott, Bayesian Epistemology; Hájek, "Interpretations of Probability," and Joyce, "Bayes Theorem." I would not recommend using Subjective Bayesianism as a method of judging the epistemic justification for beliefs. Objective Bayesianism does have some utility as a suitable method of probability calculations for future events in intelligence analysis. However, it is not without limitations. For details of these argument please see Whitesmith, Cognitive Bias, pp. 77-81.) 39. Heuer and Pherson, Structured Analytical Methods, chapter 7, part 6. 40. Whitesmith, Cognitive Bias.

Disclosure statement
No potential conflict of interest was reported by the author(s).

Funding
The author(s) reported there is no funding associated with the work featured in this article.

Notes on contributor
Martha Whitesmith Visiting Fellow at the Department of War Studies, King's College London.