Using Design Methods to Improve Design Quality: Verbalizers vs Visualizers

Abstract Although a wide variety of design methods are used, two questions have been little investigated: whether using many methods improves the outcome quality and who benefits more from using them. We conducted a quasi-experiment in a classroom employing a 2 (Design Method: More vs Fewer) x 2 (Style of Processing: Verbalizer vs Visualizer) between-subjects design. We obtained two findings from the data. First, the students using more design methods generated better outcomes than those using fewer design method. Secondly, verbal-oriented students generated better outcomes than visual-oriented students. Our obtained two findings will be discussed in the context of design process.


Introduction
Historically, design research came to the fore as the study of design methods in the 1960s. While design researchers worked on research "into" design to build theory, design practitioners conducted research "for" design (i.e. making artefacts by focusing on figures and aesthetics (Frayling, 1993)). Now, contemporary design researchers focus on the specific methods that designers utilize when working on a project. They view design research as research "through" design, i.e. designers' unique ways of navigating through a project (Roth, 1999;Cross, 2001;Koskinen et al., 2011;Adams et al., 2011). Since design research is "tied to practice and is driven by its needs" (Roth, 1999, pp. 19-20), designers' intellectual property is honed through the activities involved in design practice (Cross, 2001) and research needs to employ methods and processes from design practice as a legitimate method of enquiry (Zimmerman et al., 2010). That is, design research encompasses the physical embodiment of man-made things, construction as a human activity, the embodiment of configuration, a systematic search for and acquisition of knowledge related to design and design activity (Bayazit, 2004, p.16).
A wide range of design methods and applications are used and taught in practice and education (Roth, 1999). Recently, design research or applying design ways have been highlighted to solve complicated problems (Brown, 2009;Martin, 2009) in the social innovation area (NESTA, 2014;Kimbell, 2014) and in the corporate environment (Lafley and Charan, 2008;Clark and Ron, 2008). In particular, design methods, such as cultural probing, shadowing and quick-and-dirty prototyping, empower designers to become engaged up front in a project and help non-designers to approach a problem in an innovative way (Liedtka and Ogilvie, 2011;Junginger, 2007;Kimbell, 2012).
Despite the rapid increase in using design methods to tackle (non-) design tasks, there is little research to substantiate the benefits of utilizing design methods. When non-or novice designers use design methods, whether the design methods improve their outcomes. Rather, people often adopt random design methods without considering that the outcome quality differs depending which design methods are applied to the project. Further, although designers' thinking has been emphasized (Kimbell, 2011(Kimbell, , 2012, the design methods that identify problems have been extensively discussed whereas the design methods that embody concepts and realize them in tangible forms have been relatively ignored. In addition, design researchers belittle the utilization of design methods; for example, user-engagement approaches are often blamed for hindering disruptive solutions (Beverland, 2010;Verganti, 2009). Therefore, this paper aims to identify how to use design methods to improve outcomes. We believe that a more strategic approach to design methods is needed to maximize the value of design methods in design quality; the application of design methods needs more understanding and better guidance. In order to achieve this aim, we test two hypotheses in the present work. In essence, addressing these hypotheses will help managers and teachers to educate their employees and students about how design methods can be employed, depending on their underlying capabilities.

Design Methods
A wide variety of design methods have been devised in a human (user)-centred approach and participatory culture is highlighted (Sanders, 2002). Studies or books about design (thinking) methods and tools (Kumar, 2013;Hanington and Martin, 2012;Liedtka and Ogilvie, 2011;IDEO, 2003 etc.) show four phases of a design process which are categorized into two modes: exploration (understanding contexts and framing insights) and exploitation (conceptualization and realization). Some are applied in the research phase and others are employed in the exploitation phase (Buchanan, 2001;Forlizzi and Battarbee, 2004). However, most studies note that a single design method cannot create a design, rather multiple design methods are needed to achieve a very good design (Hanington and Martin, 2012). To improve design, two phases -research and exploitationneed to be integrated; design seeks to demonstrate the results of a "systematic enquiry as a tangible design product" (Swann, 2002, p. 52).
Depending on the methods or techniques that researchers use, the types of understanding and findings (Sanders, 2002;Button, 2000), the results and ideas (Kulkarni et al., 2012;Lim et al., 2008) they obtain will be predetermined (Boeijen et al., 2013). As illustrated in Figure 1, designers use 'interviews' ('saying' techniques including surveys: which are classically used in social and natural science studies) to obtain explicit knowledge, 'observation' ('seeing' techniques such as ethnography and a 'fly on the wall') to obtain observative knowledge or "doing" techniques (e.g. generative sessions and co-prototyping) to obtain tacit and latent knowledge.
Regarding 'observation' techniques, since these are less contentious, emphatic observation is a good way to interpret human behaviour (Lunenfeld, 2003). Kolodner and Wills (1996) note that observation plays a key role: problem reformulation, serendipitous recognition, criteria and constraints emerge across diverse design activities. Observative knowledge incorporates new product development (Sanders and Stappers, 2012;Mintzberg and Westley, 2001) to improve the current usage of design; this aspect relates to the usability and usefulness of design quality. Sanders and Stappers assert (2012) that 'doing' with users enables them to reflect on their past in a natural manner and elicit the future and find their unmet needs by working with a researcher (Sanders and Stappers, 2012;Button, 2000). In particular, it can be asserted that the pivotal roles which Sanders notes within 'doing' conform with the role of 'making' methods, such as visualizing and prototyping. These 'making' methods are useful to help participants reflect on memories or initiate participants into the process of ideas generation. Regarding 'making' methods, there are diverse perspectives, e.g. to understand and identify users' unmet needs (Sanders and Stappers, 2012), to explore and stimulate a design (Houde and Hill, 1997). That is, this role of 'making' methods relates to visceral experience or disruptive technology (Sanders and Stappers, 2012;Mintzberg and Westley, 2001). On the other hand, 'making' methods make it possible to embody ideas within stakeholders, and this enables them to experience tangible forms through communicating ideas (Dow et al., 2011) and to appreciate the feasibility of and have a common perspective on a project (Koskinen et al., 2011); this role can increase feasibility and viability (Sanders and Stappers, 2012).  Sanders and Stappers 2012, p. 67) To summarize, as Mintzberg and Westley (2001) posit, interview and observation methods combine diverse elements for creative solutions, and making methods can be applied to find a solution to an ill-defined problem. For social or collective creativity, three types of knowledge generated by using the three methods in Figure 1 are required (Sanders and Stappers, 2012) during a project. Thus, since saying (interview) techniques per se are used not only in design research but also in social science research, this paper focuses on identifying the differences in applying two types of designers' methods during a project: observation and making (doing) techniques, whose efficacy is unprecedented. In this paper, making techniques convey other techniques of thinking by doing (Saloner, 2011), which enhance the defining of problems and generating solutions.
H1: The designers who use more design methods generate better outcome than the designers who use fewer design methods.

Style of Processing (verbalizer and visualizer)
Psychologists have long studied how people process visual information and have developed three different measurements (Childers et al., 1985;Sheehan, 1967;Richardson, 1977). First, researchers focus on 'imagery vividness' or the clarity of the mental image that an individual evokes. This is measured by the Vividness of Visual Imagery Questionnaire (VVIQ) developed by Marks (1973) and enhanced in Bett's Questionnaire on Mental Imagery (Sheehan, 1967). Next, researchers became interested in 'imagery control' or the individual's ability to self-generate a mental image or perform certain manipulations, such as mental rotation. In order to assess this facet of imagery processing, Richardson (1977) revised Gordon's Test of Visual Imagery Control (VIC). It includes 12 items in a true-false response format. Finally, more recently, researchers have studied 'imagery style' or the individual's willingness to engage habitually in visual-versus verbal-oriented processing. Processing style is conceptualized as a preference and the propensity to engage in a visual and/or verbal modality of processing. Richardson (1977) developed the VVQ (Verbalizer and Visualizer Questionnaire) and Childers et al. (1985) revised it to develop Style of Processing (SOP) to assess the propensity of/ preference for visual-information processing compared with verbal-information processing. It comprises 22 items and four response scales.
According to previous psychological research, visualizers primarily rely on imagery processes when attempting to perform cognitive tasks; verbalizers prefer to process information by verbal-logical means. Therefore, visually-oriented individuals gather, process and extract insights from information differently to verbally-oriented individuals. Darley (2010), for instance, demonstrates that SOP relates to media perceptions and information exposure. Visually-oriented individuals are more favourably predisposed to the medium of television and report greater exposure to this medium than verbally-oriented individuals. Verbally-oriented individuals report greater exposure to printmedia information than do visually-oriented individuals. Yoo and Kim (2014) demonstrated that visualizers experience greater elaboration of mental imagery than verbalizers when exposed to a concrete picture of a product. These findings are relevant to multimedia learning within design. Mayer and Massa (2003) investigated the multifaceted dimensions of visualizers and verbalizers, including cognitive style, the capabilities of verbal and visual, and learning preferences.
In the same vein, design researchers have investigated the relationship between visual-information processing and creativity. Visual triggers, such as nature, open spaces or pictures, are found to either enhance or foster creativity (Goldschmidt and Smolkov, 2006;McCoy and Evans, 2002).
In sum, our review of visualizer-verbalizer differences suggests that visualizers tend to gather information and discover insights from visual sources, whereas verbalizers tend to do this from verbal sources. Therefore, we expect that while visualizers are able to identify insightful needs without any external help in a design project, verbalizers may need supportive methods to perform the task at the same level. Therefore, we argue that design methods will benefit verbalizers more than visualizers in terms of identifying needs and developing outcomes.
H2a: When designers are verbalizers, those who use more design methods generate better outcome than those who use fewer design methods.
H2b: When designers are visualizers, those who use more design methods do NOT generate better outcome than those who use fewer design methods.

Participants and Procedure
We collected data from students during a six-and-a-half-week team project in a visual environmental design studio class doing a Visual Communication Design course in Korea. Sixty students were randomly assigned to twenty teams of three students each. They were in the second semester of the third year and had no prior experience of design research. Each team was asked to develop a new product or service with a new brand name, which improves the mental and/or physical health of the people in their twenties. Prior to beginning their projects, students were provided with a set of IDEO method cards which contain 51 methods to inspire design under four categorizations: learn, look, ask and try (IDEO, 2003). We chose the IDEO method cards because they include not only businessoriented, conventional research methods (e.g. desk research, competitor analysis and questionnaire) but also design-oriented, new research methods (e.g. shadowing, cultural probe and quick-and-dirty prototyping) (see Table 1 for more detailed information about IDEO method cards). In class, the whole students learned and practised the two methods, affinity diagram and customer journey map. After that, they were asked to use more than three methods available from the IDEO method cards for their own projects. Students were informed that the number of design methods they used would not influence their course grades. Once they finished their own projects, they submitted booklets including workbooks and guidelines. They illustrate how they defined problems, how they developed solutions, and how their solutions could be applied in the real world.
We conducted an experiment with a 2 (Design method: More vs Fewer) x 2 (Style of Processing: Visual vs Verbal) between-subjects design. First, we divided 20 teams into two groups depending on whether a team used more than the four methods including both observation and making or not in their research and exploitation phases (see in Appendix 1). Note that we counted the types and the number of the design methods each team used from their submitted booklets. Self-reporting methods (booklets in this paper) are widely and methodologically used in behaviour studies (Farrington et al., 1996;Spector, 1994). Second, we also divided 20 teams into two groups depending on whether a team was visually-oriented or verbally-oriented. To do this, we collected individuallevel Style of Processing (SOP) scores by conducting an online survey of individuals and then averaged them to derive a team-level Style of Processing score. Note that we did not treat the verbal and visual components of SOP scales as independent dimensions because the authors say that they "prefer to compute a single score representing a point on a continuum ranging from verbally oriented to visually oriented processing. Scoring the scale in this manner is more consistent with the nature of the construct it was designed to measure -preference for a style of processing" (Childers et al., 1985, p. 131).

Measurement
We recruited six design experts to evaluate the submitted 20 design outcomes. They were three design researchers and three design practitioners, all with more than seven years' experience. They evaluated each design outcome based on the three evaluation criteria: novelty, usefulness and commercial appeal. We borrowed the three criteria from the prior research on the topics of creativity (Amabile, 1996), new product development (Cooper and Kleinschmidt, 1987) and design (Dahl et al., 1999). Judges evaluated each criterion by answering three questions on a seven-point Likert scale. For instance, novelty was measured by uniqueness (ordinary=1 vs unique=7), originality (commonplace=1 vs original=7) and freshness (routine=1 vs fresh=7). Usefulness was measured by usefulness (useless=1 vs useful=7), effectiveness (ineffective=1 vs effective=7) and worthwhileness (worthless=1 vs worthwhile=7). Finally, commercial appeal was measured by appeal (unappealing=1 vs appealing=7), likeability (not likeable=1 vs likeable=7), and desirability (undesirable=1 vs desirable=7).
In sum, we collected three data sets from 20 teams, including design methods (More vs Fewer), Style of Processing (Visual vs Verbal), and design quality (novelty, usefulness and commercial appeal).

Findings
We obtained evidence that using more design methods does not improve visualizers' design quality but doing so significantly improves verbalizers' design quality. First, we obtained a marginally significant interaction effect of design methods and style of processing on the novelty of design projects (F(1,16) = 3.127, p = .096). Experts evaluated that the design projects submitted by verbaloriented teams were more novel when using more design methods (5.14) than when using fewer ones (4.26; simple effect, F(1,16) = 5.815, p = .028). However, the novelty of the visual-oriented teams' design projects did not differ depending on whether they used more design methods (4.76) or fewer design methods (4.78; simple effect, F(1,16) = 0.004, p = .952).
Second, we obtained an identical interaction effect for design methods and style of processing on the usefulness of design projects (F(1,16) = 6.242, p = .024). Verbal-oriented teams submitted more useful design projects when they used more design methods than when they used fewer design methods (More = 5.25 vs Fewer = 4.15; simple effect, F(1,16) = 8.454, p = .010). However, the usefulness of the visual-oriented teams' design projects did not differ depending on whether they used more design methods or fewer design methods (More = 4.77 vs Fewer = 4.99; simple effect, F(1,16) = 0.358, p = .558).
Finally, we obtained an interaction effect for design methods and style of processing on the commercial appeal of design projects (F(1,16) = 6.613, p = .020). Using more design methods improved the commercial appeal of the design projects submitted by verbal-oriented teams (More = 4.88 vs Fewer = 3.70; simple effect, F(1,16) = 7.517, p = .014). However, using more design methods did not improve the commercial appeal of design projects for visual-oriented teams (More = 4.20 vs Fewer = 4.57; interaction effect; simple effect, F(1,16) = 0.750, p = .399).

Figure 3. Design quality (measured by novelty, usefulness and commercial appeal) as a function of design methods (More vs Fewer) and style of processing (Visual vs Verbal)
In order to clarify the effect of design methods on design quality, we additionally tested whether using design methods in the research phase improves the novelty, usefulness and commercial appeal of design projects. We employed a 2 (Design Methods in Research Phase: More vs Fewer) x 2 (Style of Processing: Visual vs Verbal) between-subjects design. In this additional analysis, we divided teams into two groups depending on whether a team used both design methods -observation and making methods -or not in the research phase. We used the same Style of Processing scores obtained from the previous study.
Our analysis reveals that the design methods used in the research phase did not improve the design quality for visual-oriented teams, which is consistent with our previous findings. However, we found that using design methods in the research phase improved the usefulness of their design projects but did not improve their novelty and commercial appeal.
Statistically speaking, the interaction effect of design method and style of processing on the novelty of design projects among the verbal-oriented teams was not significant (F(1,16) = 1.955, p = .181).
The same interaction effect on the commercial appeal of the verbal-oriented design projects was not significant either (F(1,16) = 1.886, p = .189). However, the interaction effect on the usefulness of design projects of the verbal-oriented teams was marginally significant (F(1,16) = 4.083, p = .060). More specifically, verbal-oriented teams submitted more useful design projects when they used more design methods than when they used fewer design methods in the research phase (More = 5.26 vs Fewer = 4.30; simple effect, F(1,16) = 4.777, p = .044). However, usefulness did not differ among the visual-oriented teams depending on whether they used more design research methods or fewer design methods in the research phase (More = 4.73 vs Fewer = 4.99; simple effect, F(1,16) = 0.386, p = .543).
In sum, we obtained two interesting findings. First, design methods throughout a project improved the design quality for verbalizers but not for visualizers. When verbalizers used many design methods, they improved the novelty, usefulness and commercial appeal of their design projects. Second, design methods in the research phase improved the usefulness of the verbalizers' design projects. However, they did not improve the novelty or the commercial appeal of the verbalizers' design projects.

Summary of findings
Our data reveal interesting findings. First, design methods -observation and making -successfully improved three aspects -novelty, usefulness and appeal -of the design projects of verbal-oriented teams. However, they had no impact on the design quality of visual-oriented teams. Our findings suggest that design methods benefit designers who need to trigger insights and amplify outcomes.
Second, the impact of design methods on design quality depends on the phase. Using design methods in the research phase improved usefulness, but not the novelty or the commercial appeal of the design projects. Differently than these findings, using design methods in the exploitation phase showed improvements in the all three criteria for the design projects submitted by verbal-oriented teams, suggesting that the design methods used in the exploitation phase increased the novelty and appeal of design projects.
To summarize, utilizing design methods in the research and exploitation phases will help verbal designers to develop 'good design'.

Discussion
Traditionally, designers have adopted design methods without rigorously examining their effectiveness. We found that verbal-oriented designers benefitted more than visual-oriented designers when using design methods. Our findings can attribute to the different ways of processing information between verbalizers and visualizers. They needed to translate visual data into verbal form in order to synthesize the collected data to generate ideas. We speculate that verbalizers are more familiar with doing so than visualizers. Therefore, verbal-oriented students can accrue more benefits from using more design methods than visual-oriented students. Applying design methods to both the research phase and the execution phase may stimulate verbal-oriented students to generate influential outcomes. Specifically, using design methods in the research phase influence the usefulness aspect and using design methods in the execution phase drives verbal students to deliver influential outcomes in terms of the novelty and appeal aspects. In sum, design methods need to be applied throughout a project in order to deliver influential design outcome in diverse aspects.
Teaching design methods needs to be done carefully with a class and undertaken differently depending on students' characteristics. That is, verbal-oriented students need to be taught about applying design methods throughout a project in a more structured way to trigger students' idea generation and execution. On the other hand, visual-oriented students need to be differently treated to apply design methods to their projects. In addition, these days, since design methods are taught vigorously in business schools and corporations where verbal-oriented processes are dominated, this paper might be another good evidence of why business people need to learn about utilising design methods. This also suggests cues to business people in order to utilize design methods more strategically and maximize their impact on design quality; business people need to apply design methods not only in the research phase but also in the execution phase.
Therefore, in particular, the psychological characteristics of designers and the phase of design projects should be considered and treated differently. We elaborate the implications of our findings for the different groups below.
First, for design disciplines, we found that the number and types of design methods applied throughout the whole design process influenced the overall quality generated by verbal-oriented teams. Thus, design educators should encourage students who show verbalizer characteristics to use design methods -both observation and making techniques -throughout a process.
Second, for non-design disciplines, our findings suggest that non-design students who study materials in verbal form will benefit from employing design methods. In order for them to be comfortable with making techniques, we provide them with some guidelines to apply making techniques to their own projects.

Limitations and Future Research
First, we conducted a single experiment. Our findings need to be tested with different types of and in different contexts of design projects. For instance, our studies may not be replicated when designers work on incremental new products. Further, an identical pattern may not be found when individual designers independently work on their own projects. Visual designers may also need many design methods when they work by themselves.
Secondly, we did not manipulate but measured design methods and style of processing. In the future research, researchers should randomly assign participants to different conditions of design methods (More vs Fewer) as well as manipulate the way in which each team processes information (Visual vs Verbal) in a subtle way to obtain more robust findings.
Lastly, the results of this project can be determined by other factors. For instance, the quality of design outcomes may be high when teams' willingness to achieve the goal is high. Alternatively, when teams use certain design methods, their outcomes are more qualified. Future research needs to determine how much effort each team invests and which specific design methods they use for their projects.