Towards Precedence that Justifies the Knowledge Claims of Design Methods

: This paper analyses the relation between the precedence of a design method and the justification of the method. A design method is assumed to advance two knowledge claims concerning its domain of application: that it is an effective means for designing and that it is an efficient means for doing so. It is argued that precedence of a method can justify these knowledge claims under two conditions. The first is that precedence, in addition to descriptions of successful design projects realized with the method, also comprises (i) negative precedence of unsuccessful design projects, and (ii) comparative precedence of design projects carried out with other methods rival to the method concerned. The second condition is that the two knowledge claims of a method can be justified by precedence about only exemplar design tasks that represent the full application domain of the method.


INTRODUCTION
Successful design projects play in design methodology at least three roles: first they are analysed for deriving new methods to design with (or tools, or approaches, etc.), second they are used for illustrating how design methods work, and third they are presented for justifying the methods. These roles are visible in, for instance, the literature presenting the newer design methods. Successful design projects are analysed and then used for abstracting the methods from (e.g.: Verganti, 2009), or methods are described in parallel with giving explanatory successful cases (e.g. : Brown, 2009;Dorst, 2015;Hekkert and Van Dijk, 2011;Plattner et al, 2009). And in this literature the success cases are presented for giving the promise: 'adopt this method, and similar design successes are in your reach.' The first two roles of successful design projects make research-methodological sense: descriptions of design projects form a central starting point in design research; and it is explanatorily beneficial to illustrate methods with examples. The third role is useful too for displaying the successes design methods can bring. Yet this role is research-methodologically problematic since it can be argued that success cases hardly justify methods. This paper considers the role of design projects in justifying design methods. These methods are taken as advancing two knowledge claims, namely that they are effective means for doing design projects in their domain of application, and that they are efficient means for doing so. Design projects successfully carried out with design methods are in design research taken as the precedence of these methods, and the aim of this paper is to determine how this precedence can be related to the justification of the two knowledge claims of methods. The upshot is that for establishing this relation in a research-methodologically sound way, one should first take precedence in a broader sense than merely successful design projects, and second limit justification of methods to their performance on exemplar design tasks representative for the methods' application domains.
The analysis is general and intended to hold for all design methods regardless of the specific application domains they are meant for. A drawback of this generality is the abstractness of the analysis.
In Section 2 it is argued that standard precedence cannot justify design methods. In Section 3 the notion of precedence is expanded to include also negative and comparative precedence. In Section 4 it is shown that precedence may justify methods when this justification is based on their performances on exemplar design tasks. Section 5 gives research-methodological backgrounds. And Section 6 discusses the presentation of design methods in this approach.

EFFECTIVENESS AND EFFICIENCY
For exploring how precedence can justify the effectiveness and efficiency of a method, it is useful to model the knowledge represented by these two claims. For keeping the analysis general some informal notation is introduced.
Let D refer to a designer, M to a design method and N M to methods that are taken as rivals of M (boldfaced symbols refer to sets, so N M may refer to a number of rival methods N, N', N'', …). Let E M refer to the design expertise a designer is supposed to have for applying method M, and let R M refer to the maximum resources the designer is supposed to use for carrying out a project with M. Let, finally, T M refer to the application domain of design tasks for which the method M is meant.
It is here assumed that proponents of a design method M define N M , E M , R M , and T M . If, for instance, M is a method for innovation by design thinking, the rivals N M could be other design thinking approaches. The expertise E M of a designer could be the type of education the designer has had and the number of years of his or her professional experience. The resources R M for a project may be the time and money that may be spent on a design project, and the tasks T M may be designing a specific type of products, say, energy infrastructures or computer games.
The knowledge claims of effectiveness and efficiency can then be modelled as:

Effectiveness of design method M:
A designer with expertise E M can resolve a design task in T M with design method M with a specific success rate S M , using resources R M or less.
Efficiency of design method M: If a designer with expertise E M resolves a design task in T M with a design method N in N M rival to M, then the designer uses regularly more resources than R M .
The success rate S M part of the formulation of effectiveness captures to what extent a method M allows for failures. This rate may be high, meaning that designers regularly resolve tasks in T M with M, or it can be low, as when the method gives merely solutions with a specific probability (the analysis remains qualitative, so talk about quantitative probabilities is avoided).In the formulation of efficiency it is assumed that a method M can be called more efficient than a rival N if applying N regularly requires more resources than M. The possibility that a method M is only sometimes more efficient than N is not included, for that possibility implies that using method M regularly requires more resources than method N and that N is thus more efficient than M.
The formulation of effectiveness does not rule out that designers with less or different expertise than E M can also (occasionally) resolve a task in T M with the method M, or that designers with more expertise than E M may resolve tasks in T M with M with higher success rates.
Precedence consisting of successful design projects that are carried out with a method M gives inductive support for the effectiveness of M, but hardly justifies this claim. It establishes that there exist some tasks in T M for which the method M is effective, but does not give much information about the other tasks in T M . And precedence leaves open whether M is a method with a high or low success rate S M ; it merely shows that there were occasions at which applying the method M led to successful projects, which still may mean that M often does not lead to success.
Precedence also does not justify the efficiency of a method M; evidence that there were occasions at which M led to successful design projects does not rule out that rival methods N in N M can do so with resources less than R M . Only in the special case that T M contains tasks no other method than M can address, then precedence of M lends support to the efficiency of M relative to other methods (an example would be the frame creation design method of Dorst (2015) that is presented as being able to resolve 'networked problems' other approaches cannot deal with).

PRECEDENCE
For preserving the relation between the precedence and the justification of a design method M, one can consider including as precedence projects that failed when carried out with M, as well as projects in the domain T M of the method M that were carried out with other methods N in N M that are rivals to M. Let positive precedence be the usual type of precedence consisting of design projects successfully carried out with M. Two new types of precedence can then be negative precedence being projects that failed with M, and comparative precedence consisting of projects that are carried out with methods N rival to M: Positive precedence of design method M:  Designer D' with expertise E' equal to E M resolved design task T' in T M with design method M, using resources R' equal to or less than R M .
Negative precedence of design method M:  Designer D'' with expertise E'' equal to E M did not manage to resolve design task T'' in T M with design method M, using resources R'' equal to or less than R M .
Comparative precedence of design method M (two subtypes): with a design method N in N M rival to M, using resources R'''' equal to or less than R M .
The first subtype of comparative precedence includes projects that did not lead to results at all when carried with a rival method N.
Negative precedence supports conclusions about the success rate S M of a design method M, since if such precedence exists, one has evidence that this success rate S M cannot be high. Negative precedence may moreover reveal ways in which a method M can be improved, by suggesting what additional expertise or resources designers should have for applying the method. Hence, it may point to ways to reformulate the method M for arriving at one that does have a high(er) success rate.
Comparative precedence gives information about how a design method M fares relative to its rivals N M . It supports the efficiency of M if designers using those rival methods, regularly need more resources than R M when carrying out design tasks in the application domain T M of M. Reversely, if those rival methods do regularly lead to successful outcomes using less recourses than R M , one has reason to limit the domain T M at which method M is claimed to be efficient.
With negative and comparative precedence included, the relation between precedence and justification is still not established. For having justification of the effectiveness of a design method M one should minimally have that its positive and negative precedence in some way exhausts the whole application domain T M of the method M. And even with abundances of positive precedence supporting that the success rate S M is high, subsequent experiences may generate large amounts of negative precedence suggesting a lower rate. The problem one encounters here is that of induction: logically and practically a series of predominantly successful design projects may still be succeeded by a series of mostly unsuccessful projects, as tossing five times 'heads' with a coin may be succeeded with tossing four times 'tails.' The justification of the efficiency of a method M leads to similar problems, even when efficiency is only claimed relative to one rival method N. Again the comparative evidence should exhaust in some way the application domain T M of the method M, and again the problem of induction turns up.
Adding the two new types of precedence does have the advantage of relating precedence more broadly to the knowledge claims inherent to design methods: negative precedence connects with the claim of effectiveness, specifically to the success rate S M , and comparative precedence gives a connection with the claim of efficiency. Still precedence of a method in this broader sense does not fully justify the method.

JUSTIFICATION
The observed gap between the support that precedence gives to design methods and what is needed for their justification, is in part due to the assumption that justification of design methods means full verification. By this assumption justification means evaluating effectiveness and efficiency of a method M for all the design tasks in its application domain T M . Hence, for bridging the gap one may try relaxing this assumption. This can be done on the basis of recent design research on method justification, called the validation of design methods (e.g.: Blessing and Chakrabarti, 2009;Frey and Dym, 2006;Frey and Li, 2006;Seepersad et al, 2006). This gives at least four options for justifying a design method M through considering only some design tasks in its application domain T M .
Three options can be found in (Frey and Dym, 2006) who explore possibilities to validate design methods similar to how medical treatments are evaluated. First, using the analogy to medical in vitro experiments on animal models of medical disorders, design methods could be validated in controlled lab circumstances. In design research this would involve focussing on design tasks for which the possible design actions and their results can be upfront modelled, say by means of computer simulations. The effects of applying a design method M to one of these tasks can then be monitored and assessed in an unambiguous way (an example of such a model design task is given in Frey and Li 2006). A second option draws on the analogy with clinical trials and involves defining design tasks that can act as benchmarks for the validation of methods. Designers can apply different methods M, M', … to these benchmark tasks, and the results of the different projects can then be compared. The third option Frey and Dym discuss is doing field research on design methods, now in analogy to field research on the effects of medical treatments. This option entails determining the overall success of firms that have adopted a specific design method and comparing this success between firms using different methods M, M', … . This third option means that only those design tasks are considered that firms are actually carrying out.
A fourth option can be found in (Seepersad et al, 2006). They propose to validate the effectiveness of design methods by means of four steps making up four quadrants of what they call the validation square. First, the structural validity of a design method M is to be established by showing that the method is logically rigorous, internally consistent, mathematically correct, and applicable to its intended domain T M . Second, example design tasks should be defined for verifying its performance, as well as an argument be given why these example tasks are appropriate for, e.g., representing the full application domain T M .
Third, it should be demonstrated that the method gives successful results when applied to the example problems and that these results are due to using the method. Fourth, an argument should be given that the usefulness of the method for the example tasks may be generalized to the full application domain T M .
There are differences between these four options. The three that Frey and Dym are providing assess and compare design methods and thus validate effectiveness and efficiency. The fourth by Seepersad et al focusses on single methods and therefore seems to be about effectiveness only. What the options share is that validation of a design method M does not mean giving evidence that the method M successfully resolves all the design tasks in its application domain T M ; the evidence required is that designers can do so for a few design tasks representative to this domain T M , ranging from lab design tasks to example design tasks.
Let us call these design tasks that are representative to the application domain T M of a design method M the exemplar design tasks of the method M, and let TX M be the set containing these exemplars. Let us adopt the described validation approach followed with the four options, and take justification of a design method M as showing that M is effective and efficient on only its exemplar design tasks. Positive and negative precedence about the exemplars of the method M can then sufficiently cover the exemplar domain TX M , and hence justify that M is effective with a certain success rate S M . Comparative precedence about the exemplars can similarly justify efficiency of the method M. A straightforward relation between precedence of a method M on its exemplar design tasks and the justification of the method is then achieved.

RESEARCH METHODOLOGY
The approach to justifying a design method M by showing that M is effective and efficient for its exemplar design tasks TX M can research-methodologically be understood as a move from justification by verification of the design method M's knowledge claims to justification by falsification (Vermaas, 2014). Verification indeed means showing for all tasks in its domain T M that the method M is effective with a certain success rate S M and more efficient than its rivals methods in N M . The sketched approach limits the (initial) focus on the full application domain T M of the method M to a focus on the smaller domain TX M of M's exemplars. Hence, the exemplar domain TX M of the method M declares what design tasks can be used, in first instance, for testing M, meaning also that the method can already be rejected if effectiveness and efficiency is proved not to hold for the exemplars. This possibility of a quick rejection of a design method is similar to falsification in the Popperian sense, although in falsification it is also the evaluator who may choose the design tasks relative to which he or she wants to put the method M to the test.
Considering in addition to effectiveness also the justification of the claim of efficiency of a design method M can research-methodologically be understood as a second move from justification by Popperian falsification to justification by (sophisticated) falsification as described by Lakatos (1978). In falsification as associated to Popper (1968), a scientific theory, or design method in our case, is put to the test with the aim to either corroborate or reject it. In this testing the theory, or method, is considered in isolation, and if the outcome is negative then the theory or model indeed has to be thrown away. In philosophy of science Lakatos argued that such rejections in isolation do typically not occur in science. Scientists do not take single counterevidence as reason to drop a theory; that theory may be, for instance, the only theory scientists have, and provides at least some means to understand and predict phenomena. In science such theories are rather kept, and the counterevidence is set aside as anomalies to be dealt with later (or not at all). Yet, if rival theories are available and scientists have the opportunity to switch to an alternative second theory when the first is falsified, then theories are indeed rejected or, better put, replaced by their rivals. By including efficiency in the justification of design methods, rivals are explicitly introduced in design research. Hence, when effectiveness and efficiency of a method M is falsified for a few (exemplar) design tasks, design researchers become in the position to actually reject the method M and replace it by one of its rival methods N in N M .
This prospect may give a new impetus to design research (Vermaas, 2014). In analogy to Lakatos' scientific research programmes consisting of series of competing scientific theories that in time are replacing each other, one can imagine the emergence of design research programmes of series of competing design methods that in time are replacing each other. Engineering design methods (e.g.: Gero, 1990;Pahl and Beitz et al, 2007;Suh, 1990) may form such a design research programme, and so may design-thinking methods (e.g.: Brown, 2009;Plattner et al, 2009;Verganti, 2009).

PRESENTING DESIGN METHODS
In this paper the roles of precedence in presentations of design methods have been discussed, focussing on the role of justifying the methods concerned. For letting precedence play this latter role in a research-methodologically sound way, two new types of precedence were defined -negative and comparative precedence -in addition to the standard positive precedence. If justification is understood as testing that a method is effective and efficient with respect to exemplar design tasks that represent the method's full application domain, precedence of a method indeed may justify the method.