Reconstructing the critique. Using inclusive formative feedback in face-to-face and online communities of practice to improve knowledge acquisition in design education

: Design teaching originated as a practice-based activity centred on reviewing the visual for its aesthetic appeal. This process has enabled students to produce opportune visual outcomes that lack research, academic rigour and rely on stylistic imitation of contemporary design trends. We identified that this occurs due to two underlying problems: 1) design pedagogies have developed without solid educational foundation; and 2) assessment remains linked to the end product rather than process. To address these issues, we reviewed learning design and reconstructed the design critique to shift the learning focus toward process. We developed four stages of engagement; Investigate, Apply, Explain and Share (IAES). These stages enabled us to create educational experiences that used reflective processes to improve knowledge acquisition. Student results and feedback show this application has improved understanding of key design processes and facilitated learning relevant to future design careers.


Introduction
Providing design education that meets the contemporary expectations of a designer is challenging and how to best balance pedagogies remains a conundrum for design educators. The knowledge and skills required to develop and produce compelling design solutions is multifaceted. Designers need to be innovators who can reflectively analyse research, understand design theory and combine this analysis with practical design know-how to create innovative solutions (Owen, 2007;Brown, 2008;Cross, 2006).
Traditionally design education has exposed students to fundamental design theory and practical skills and then used practice based projects to facilitate the proficiencies over time using situated learning (Lave & Wenger, 1991). Core to this process was the guidance offered by the expert, the teacher, often a designer. Students balance complex skill, expert judgement and creative intuition to generate design outcomes (Dineen, 2005;Frascara, 2007). Through this process the student gained comprehensive knowledge through observation and in turn demonstrated the acquired knowledge through practice.
This pedagogical approach, albeit less demonstrable, continues today. The success of the process relies heavily on dialogue, the design critique, to foster the ability to seamlessly integrate theory and practice. This paper discusses a pedagogical approach that acknowledges the critical importance of the design critique and redefines methodologies for using the critique as a learning tool. The methodology used is a multi-stage feedback structure. Grounded in authentic assessment the structure provides immediate feedback to work in progress. Techniques developed foster student engagement thus enabling a meaningful dialogue that questions, explains and reinforces research, theory and practice to develop a deep understanding across all aspects of the design process.

Learning through practice
Designing by its very nature is an iterative practice-based activity. However, it is the product of design that has long been valued for its aesthetic appeal, the inevitable visual response creating a focus on outcome. To the novice this creates a diminutive regard for the stages of development undertaken during the design process. Design education perpetuates this response. Educators commonly review and assess work on completion. This specific issue, is a compelling one for design educators everywhere.
"The final project product or artefact is typically used as the primary measure of learning, thus focusing students on the outcomes of the project rather than the process by which that outcome is achieved" (Ellmers, Foley, & Bennett, 2008 p.78).
This assessment approach encourages the student to focus efforts on finishing. Consequently, little attention is given to early stage creative processes that are fundamental to the development of effective design outcomes.
The visual nature of the work produced enables an immediate response to the aesthetic appeal. This suggests that it is the outcome not the process that is of primary importance. This problem is compounded by the design educator's ability, based on experience and knowledge, to make an immediate visual judgement. The rationale for judgement, often presented post project, misses the opportunity to engage students at a critical process juncture where a meaningful connection could be made, applied, reviewed and refined.
This approach fosters a culture in which students focus on producing highly-refined design outcomes, often mimicking contemporary trends or the known preferred aesthetic of the design educator to achieve a high grade. When interrogated the design outcome lacks rigour and the student when questioned is often unable to demonstrate research and knowledge in a meaningful way. Measures for determining understanding of the design process using this approach are scant making it difficult for the teacher to assess student learning holistically. When students fail to include critical steps that are fundamental to the design process, the final outcome may be aesthetically pleasing but demonstrate no evidence of process, which begs the question, what have they learnt (Kvan, 2001;Ehmann, 2005)?
This problem is compounded by the common practice of using multiple disparate projects each independently using end of semester assessment as the milestone. Timing issues aside, this approach further minimises interconnectivity by providing inadequate informative feedback at critical junctures. As a result, knowledge and skills are not feeding forward from work in progress to future application (Quinton & Smallbone, 2010). For this multiple-project curriculum to be effective, it is important students transfer their learning between projects as they progress through their program of study, however students are often left to connect their learning themselves. (Ellmers et al., 2008).
The failure of this end focus assessment compounds where design project results show that students are unable to transfer what they have learnt from one project to other subject areas and from one semester or level to the next.
Further investigation allowed us to identify that the measure of success, relational to final grade was only part of the problem. Through discussion and observation, we noted that students were progressing through units at a fast pace. The demands of society from both a social and technological perspective are forcing students to engage immediately with learning. This practice limiting the opportunities for students to become fully immersed in the practice of learning and understanding theory and practice (Roberts, 2013).

Using Feedback and Assessment for Design Learning
Anecdotal evidence within our School repeatedly highlighted this issue of poor knowledge transfer. To address this reoccurring theme the School had trialled a number of approaches to improve learning including the use of common terminology and shared resources. However, a solution to the problem was not emerging. 'Add on' departmental approaches were difficult to action and showed little impact.
Consequently, we developed a structure that would build relational connections and facilitate learning transfer engaging the critique to provide feedback directly linked to assessment. Making this link encouraged a shift in student focus and enabled a strong connection to the iterative nature of design development to form. This structure used four stages of engagement; Investigate, Apply, Explain and Share (IAES). At each stage the structure adapted the traditional design critique to focus learning through feedback style and encouraged reflective practice. The Explain and Share stages give particular emphasis to understanding through shared ideas in regard to process and practice.
This structure enabled theory, practice and outcome to interconnect; each aspect clearly defined and directly linked to formative and summative feedback. This approach shifted the emphasis away from outcome-based assessment and created parity within each stage of the design process, equally weighting theory, conceptual thinking, idea development and final production. Importantly the structure connected students with a process that in many cases had previously eluded them. Each stage needed to be purposeful to the design process and inform future development to prevent the perception that unnecessary work was being undertaken. Critical to the success of the structure was clear explanation of each stage and how it contributed to the design process. This approach validated the work requirements undertaken and through the process of explanation increased learning opportunities.
"Students have more opportunities to learn throughout the unit and respond to the brief in a continued method of learning with key assessment milestones throughout. From a teaching perspective I find that there are more opportunities to see the areas students are having difficulty with and adjust teaching to improve learning." [Sessional teacher, Typography] In 2012 we trialled our staged approach in two first year and two second year units (Photography and Typography). Enrolment in each of these units was in excess of 250 students per year. We developed a staged structure that reflected the design process identifying project milestones, connecting these to theory and practice through interlinked weekly learning tasks. These weekly tasks were reviewed requiring students to explain the theories that underpin their 'work in progress' and how they apply theory to subsequent design iterations. At each stage of the design process formative and summative feedback is provided. This enabled students to internally clarify their understanding and externally demonstrate this to the teacher through practice. This creates a critical connection between theory and practice and allows students to improve knowledge and outcome throughout the entire design process. Investigate, Apply, Explain and Share (IAES) breaks down the design process into four stages. The stages broadly represent the activities commonly used to problem solve and provide flexibility to accommodate a range of design projects. Within the four stages the project brief is 'chunked' as appropriate to the specific project. Design briefs are project based and use authentic assessment tasks that require students to demonstrate: critical thinking; problem-solving; innovation; and mastery of skills.

Interconnecting Stages
The learning process translates the traditional master apprentice method into a contemporary setting. The students are expected to acquire the knowledge through teacher instruction, mentoring and feedback. As a result, the structure remains focused on dialogue, making the critique the fundamental link between instruction and practice. At each stage aspects of theory are actively linked to practice. Stages are carefully considered to coincide with project milestones. The requirements at each stage of the process are made explicit with clear indications of expectations provided. Assessment criteria are outlined in detail in project rubrics. The tangible design outcome created at each stage then uses the design critique both in class and online to engage students in a dialogue about ideas and work in progress.
The specific requirements of each stage are clearly identified to ensure that the problem of minimal understanding of process is addressed. In stage one, Investigate, students investigate the application of design theory to conceptual development. They are asked to consider the problem broadly as it applies to the given context. From findings students then begin to develop an approach.
In stage two, Apply, students apply the theory repeatedly to the conceptual development over a broad range of outcomes. During this stage the student considers what they need to do. How will they go about actualising their proposed design? What design theory is being applied and what practical techniques are being used?
A Community of Practice (CoP) (Lave & Wenger, 1991) approach leads stages three and four. In these stages students explain and share their new knowledge. Stage three, Explain, uses a nonconfrontational approach to consolidate thinking. Students are asked to clarify their processes to the group and explain their creative thinking and proposed application. This stage requires students to stop and consider, effectively minimising the race to the final outcome. This pause brings with it the opportunity to reconsider and refine. This process focused approach uses both feedback and assessment to add weight to critical reflection and progressive iteration processes of design development. Students are encouraged to pay attention to these stages rather than skip through them as they might in an outcome based approach.
Stage three works in strong conjunction with stage four, Share. The opportunity to share work in a non-threatening environment provides increased opportunity to seek feedback from peers and continue the review process until a successful outcome is achieved. Throughout the project, stages three and four work together to form a resource for all students to review and interact with at multiple points in the semester, thus reinforcing the application of theory to their practice.

Facilitating the Engagement Process
This staged pedagogical approach was introduced to achieve two aims: 1) provide a solid grounding that enabled students to feed knowledge up and forward; and 2) inspire students to achieve at higher levels. The success of these two aims is measured using student results and student feedback in pilot units for the period of 2012-2015. From the student feedback we identified four highly regarded factors: 1) project brief; 2) project structure; 3) access to project examples; and 4) opportunity for quality feedback. Each factor is discussed and student comments presented, highlight the perceived value.

PROJECT BRIEF
The units utilised project briefs designed to allows student choice in subject matter to provide flexibility, scope and reduce the likelihood of imitation. We believed that adding personal interest would encourage a deeper connection with the subject matter and allow student focus to be maintained for the length of the semester. The projects while remaining flexible had to be authentic, and provide opportunities for students to engage in complex tasks, that enabled deep reflection and broad application perspectives within a real-world context (Reeves, Herrington & Oliver 2002).
"I loved this unit. I think the layout of assignments and weekly tasks was very effective and enhanced my learning. I think it was fantastic choosing an organisation and working with that the entire semester. I really enjoyed all aspects of both projects." [Typography student, 2013]

PROJECT STRUCTURE
Project assessment was structured and carefully designed to underpin theory and knowledge and link them to process. When submitting the first of six weekly tasks for assessment, students were required to show their understanding of the theories through application to demonstrate they had understood the learning. The later tasks required them to visually exhibit and articulate the previous stage's knowledge by solidify the connection between theories and processes. This compelled students' awareness of links between all stages, building a transferable knowledge base which can be relied upon in later years. This process empowers students because at any stage they can describe their learning and connect it to practice, which helps them to construct and retain vital knowledge. Student feedback demonstrates how this approach is engaging and accessible and positively influences their learning.
"This is the best unit I have undertaken at Swinburne so far. I have learnt so many skills, I cannot explain how well planned and organised this unit is. Finally, I have been satisfied by the skills I have been taught in a unit. It is current, relevant and fun." [Typography Student, 2013] "The units have been structured clearly and the projects were great. Without this I wouldn't have come to enjoy and understand the profession as much as I have." [Photography student, 2015] "Thank you for being a design unit that has weekly tasks, it is so much more productive to have relevant task sheets every week that are marked and help complete the final submission and we are not left at the end of semester with one submission worth 95%." [Typography student, 2015]

ACCESS TO PROJECT EXAMPLES
The CoP that was created in the classroom and beyond provided access to a student-centred ongoing learning environment. Face-to-face and online learning communities provided support and created opportunities to access broader views gaining a deeper understanding of application.
"I think that this is a great way to learn we can access anyone's work at any time. I'm learning from the people who are more advanced. Comments are always helpful because I'm getting feedback from someone else's perspective." [Photography student, 2012] "I thought the in class presentations were a great idea as they allowed not only the teacher but other students to comment and critique the work enabling it to be constantly improved." [Typography student, 2013] Expanding feedback to an online environment also encouraged opportunities to develop student confidence by providing a voice for students who may not effectively communicate in a traditional tutorial but are able to articulate in writing in an online environment. This is especially noticed with students with English as a second language who are often reticent in class because of language barriers.
"Yeah, the comments on flickr are really useful for me. They help me to understand when something is really professional. Usually, in the classroom everything is too fast and I don't really understand." [International photography student, 2013].

OPPORTUNITY FOR FEEDBACK
The design process feedback is aimed at facilitating a stronger understanding of this process by providing alternate ways to tackling the problem and challenge themselves to achieve quality results. Consequently, all tutors provide consistent and timely feedback specific to each submitted task. The immediacy and frequency of the feedback is highly useful to the student and provides opportunities for them to apply new knowledge to design development as it is emerging. This approach is consistent with Vardi (2013) whose study showed that "relatedness and proximity of task requirements; consistency of assessment standards between tasks; [and] the specificity of the feedback given" (p. 607) were key factors required to increase opportunities for students to feed knowledge forward in subsequent learning tasks.
"The knowledge and experience I gained in Lynette and Lucia units drove me to challenge myself to bring my design work to the next level. They constantly provided good design advice throughout my education and I have applied this throughout my professional career." [Past student, 2016] The knowledge developed by students provides a solid building block and once sufficiently developed is one that is flexible and transferrable. Consequently, knowledge learned through our process is reapplied in progressive units and developed to sophisticated levels.
"Having students explain what they are doing in first and second year has helped them understand more. They are making better compositions in third year studio units." [Tutor, Third Year Photography, 2016] 3.3 IAES -Working in Practice

INCREASED STUDENT ACHIEVEMENT
A review of average student results where the IAES system has been piloted shows an increase in student achievement levels. It should be noted that tutors, authors included, did not change during this period, suggesting IAES is the contributing variable in this process. Presented in Table 1 is a summary of grades from 2012 -2016 for first year and second year units. When combined on an annual basis these units have in excess of 300 students. The figures demonstrate that during this period the number of students who were able to obtain high level grades, 3.2 (80%), increased. Prior to the engagement of this pedagogical approach, in 2012, on average 7.5% of students would to be able to achieve this grade. By 2015 the success rate at this level had increased to 22.5% and has remained steady at this level. The influence of this structure is further substantiated by the fact that after its first year of introduction the most marked increase, 9.5%, was achieved. At the lower end of student achievement, the structure also had a positive impact. The number of fail grades in students in the units represented here were approximately 16% in 2012. In 2015, the fail rate has been dramatically reduced to only 3.5%.

STUDENT SATISFACTION
Student Satisfaction Data (2012Data ( -2015 that reflects the average of all end of unit questions in relation to curriculum, assessment and overall engagement is presented here. Table 2, shows the student satisfaction level of the four pilot units using IAES compared to the rest of the School. The results show that pilot units have consistently improved throughout this period and have achieved an overall satisfaction rating that is higher than the School average. These findings indicate that our pedagogical approach over the four-year period of the initiative is perceived to be valuable by the students. The positive impact of IAES is reinforced in student feedback. Comments made by students are overwhelmingly positive and demonstrate they are able to understand the rationale and significance of IAES. Importantly there is an acknowledgement of how using a structured approach has benefitted them. Student comments reflect an improved understanding of the role of theory as it relates to application in the design process now while realising the potential this approach has to influence future knowledge acquisition. "My skills have definitely improved however there is still room to grow. There are so many different things to consider. In the past I had never known the importance of or the difference that things (theory) can make to a photo. They are important even to the most simple subject matter." [Photography student, 2012] "I think the feedback has been great and am really happy that people have embraced the opportunity to not only comment but to give helpful tips to other students for future reference." [Photography student, 2013] "This unit is good because of the way we have been shown how to apply techniques and then applied them the next week. I still have a lot of room for improvement but I feel have been given all the foundations I need to work with." [Photography student, 2012]

TUTOR SATISFACTION
The success of the process has relied undoubtable on the engagement of the academic tutors. To facilitate this engagement our process has been inclusive. Tutors are integral to the delivery of the content have been consulted in the development process. Regular meetings that ensure a vital understanding of our pedagogical approach (IAES) is clearly understood. Feedback from tutors identifies that the structured learning design has enabled engagement and state that students respond well to the IAES framework.
"Students have more opportunities to learn throughout the unit and respond to the brief in a continued method of learning with key assessment milestones throughout." [Sessional typography tutor, 2016] Tutors have further identified that IAES has provided scope to action improved learning. The multiple feedback stages are identified as opportunities for observation. Tutors identify improved preparedness for the unit and increased capacity to identify clearly student learning shortfalls and address them appropriately in a timely manner.
"From a teaching perspective I find that there are more opportunities to see the areas students are having difficulty with and adjust teaching to improve learning." [Sessional photography tutor, 2016]

FUTURE DIRECTIONS
The approach has been recognised by design colleagues and our staged structure is now been implemented in other units, both on-campus and online, within the School of Design. Currently this approach is also being trialled in two existing units, Concepts and Narratives (Level 1) and Visual Narrative and Integrated Media (Level 3).
"Having taught in a variety of units with different teaching methods, I find Lucia and Lynette's way of teaching design much stronger. Now developing my own units, I am using their structure of small assessment task to analysis and apply their understanding of learning." [Full-time Academic, 2016] Our IAES structure is fluid and flexible and can be applied across a variety of discipline areas. As a result, the approach is spreading beyond our school and interest in adapting our method more broadly has been shown across the university. It is expected that this broader application will provide opportunities to understand more holistically the role of frequent meaningful feedback in non-design disciplines.