Learning modules: problem-based learning, blended learning and flipping the classroom

ABSTRACT In 2020, the author introduced a new learning and assessment programme to the compulsory Land Law course at the University of Auckland, Faculty of Law. This article introduces, explains and evaluates the programme design. The Learning Modules programme has five components: structured pre-tutorial activities guiding students through a problem; tutorials in which tutors facilitate discussion on the problem; post-tutorial quizzes testing understanding of ideas arising in the problem; and exam-oriented review exercises in the form of modelling exercises and example exercises. First, the article discusses three pedagogical theories: problem-based learning, blended learning and flipping the classroom. Secondly, the article outlines the Learning Modules programme. Thirdly, the article uses student survey data from 2020 and 2021 to evaluate the extent to which the programme represents problem-based learning, blended learning and flipping the classroom. In doing so, the article considers how closely a programme needs to conform with the classical characteristics of each theory to be considered to represent them. The article concludes that the programme can be said to represent each of the three theories, depending on how they are formulated.


Introduction
In 2020, the author introduced the Learning Modules programme to the compulsory 400-student full-year (two-semester) Land Law course in the undergraduate LLB degree at the University of Auckland, Faculty of Law. 1 The programme consists of several 1 To be admitted and enrolled as a barrister and solicitor in New Zealand, persons require a legal qualification approved by the New Zealand Council of Legal Education (NZCLE).The Bachelor of Laws (LLB) is currently the only approved legal qualification available at New Zealand law schools.Property Law is a required course for the LLB degree and is usually taught as separate Land Law and Equity courses, in which case each course is required.Overseas legal qualifications can be recognised for admission, but persons with an overseas legal qualification need to demonstrate competence in the required courses for the LLB degree and may be required to sit a New Zealand Consequently, PBL develops crucial communication and teamwork skills while prompting students to develop an explorative approach to problem-solving. 6Students must collectively identify the necessary issues to address.They must be innovative in doing so, as they lack substantive knowledge of the subject matter.Thus, in line with broader constructivist theory, PBL learners are actively involved in constructing their own knowledge.PBL is a self-directed process of exploration, rather than a passive process of information transfer.
Despite the active role of students, the teacher is far from inactive in the PBL process.After an ample opportunity to explore the problem independently, the typical PBL model will gather the students for a teacher-led discussion on progress. 7Such discussions often take the form of a tutorial.The teacher will facilitate and stimulate discussion -encouraging learners to review their progress together, helping them to connect developing knowledge to new concepts, and resolving any misconceptions about the relevant law and how to apply it to the problem. 8The back and forth between studentled group analysis and teacher guidance ensures that learners genuinely understand the problem-solving process and steps necessary to reach an answer.
While PBL has its origins in medical science, it has gradually been adopted in a wider range of professional studies. 9Legal education is no exception, with law school-based advocates of PBL praising its ability to develop the communication and people skills required by practitioners but seldom instilled by traditional teaching methods. 10PBL can provoke a "deep learning approach" to law, allowing students to improve critical thinking skills within the legal context. 11Further, PBL has been found to increase student engagement by providing real-world experiences that lectures cannot. 12hus, PBL has the potential to yield great academic success and vocational preparedness for those pursuing a career in law.

What does it mean for learning to be student-centred?
Barrows' first characteristic, that PBL is student-centred, is somewhat ambiguous.It is also unclear how student-centred a learning practice needs to be to represent PBL.

Original conception.
An appropriate starting point is Barrows' own thoughts on the matter.Barrows and Tamblyn define student-centred learning as students learning to determine what they need to know. 13 a flexible environment in which self-directed students can develop learner independence and metacognitive skills -learning how to learn.This approach to student-centred learning should not be confused with students learning entirely independently from a teacher.Barrows and Tamblyn are clear that, in student-centred learning, just as in a traditional classroom, the teacher sets learning objectives, assigns resources and selects materials.The key difference is that, in a traditional learning context, the teacher dictates what knowledge and skills students are to learn, whereas in student-centred learning, the teacher and materials are only guides.The teacher may have considerably more control over the learning process at the beginning, providing scaffolding to ensure students have necessary knowledge.However, over time, teachers will shift to a more facilitative role.Thus, students become free to adapt the content to meet their own educational needs.
Further conceptions.Since PBL's original conception, there have been numerous suggested revisions, including to what it means for PBL to be student-centred.David Boud's oft-cited interpretation of PBL defines student-centredness as "acknowledging the experience of learners" and "placing emphasis on student responsibility and control of their own learning". 14This more clearly demonstrates the need for teachers to assess student needs and provide guidance in a way that facilitates student self-sufficiency.
Contemporary conceptions promote student control over the learning process, while leaving teachers to set the overall format of instruction and assessment.Heidi Maurer suggests that the ideal manifestation of student-centred learning in PBL is for students to collaboratively run tutorials themselves with the teacher providing insight only on request or when absolutely necessary. 15Maurer's view highlights the distinction between format and process.Students have significant control over how tutorials are run, but the instructional method of having tutorials and the final assessment is still prescribed by the teacher.Sofie Loyens, Paul Kirschner and Fred Paas explain that PBL is not as student-centred as pedagogies such as project-based learning, in which students are able to develop their own project, because PBL still prescribes a uniform problem which all students must engage with. 16Finally, Sanit Haruehansawasin and Paiboon Kiattikomol note the importance of scaffolding and active teacher facilitation, especially in lower-achieving classrooms. 17The authors found that hard scaffolding (providing pre-developed support materials) was more effective than soft scaffolding (discussing and clarifying in-class) at increasing student self-sufficiency. 18This finding reinforces that PBL does not anticipate the complete absence of a teacher. 19as necessary. 20Some law school-based approaches pursue Maurer's ideal manifestation by empowering students to lead discussions and restricting teachers to a passive role. 21he literature review did not find any case studies in which students were allowed to design how the course would be convened or assessed.

Student-centred PBL in legal education.
Anita Szabo notes that while many law courses assess by problems, that does not itself mean those courses use PBL. 22A traditional problem question simply requires students to draw on knowledge presented in lectures and casebook readings.The distinguishing feature of a PBL approach is that students are presented with a problem they are unfamiliar with, for which they are provided only the most fundamental knowledge. 23Students must then independently research, interpret and apply the law, seeking teacher guidance only as necessary.
In Yang Wong's study, in a criminal law course, students were presented with fundamental knowledge, then put in small groups in which they would be exposed to the problem and begin self-directed learning. 24Shortly afterwards, each group was asked to prepare questions for a private meeting with their teacher.In these meetings, the teacher did not provide any instruction, only answering students' questions and challenging their opinions to provoke critical thinking.The approach left enough of a knowledge gap so that self-directed investigation was necessary and facilitated self-discovery of the correct application of the law to the problem.

How a learning environment can be student-centred.
Based on this literature review, five steps will assist in creating a student-centred environment: (1) Teachers and resources should not dictate the learning process.Teachers may provide direction and scaffolding at the beginning of the process but must work towards student independence.(2) Teachers should encourage students to coordinate the learning process, ideally to run small group tutorials themselves.(3) Teachers should encourage collaboration between students as the primary mode of learning, through group work and discussions, with teacher guidance only available as necessary.(4) Teachers should make many diverse learning resources available so that students can curate their learning experience according to their own needs.(5) Students should be assessed not on their ability to recall information, but through demonstrating how they have researched, interpreted and applied knowledge.Wong (n 10).

Blended learning
Broadly speaking, the concept of blended learning simply refers to the merging of different pedagogies and educational tools that can be practically combined. 25 common understanding of blended learning is combining digital and in-person instruction. 26This understanding is broad -teachers can put it into practice in many different ways. 27t is generally accepted that blended learning must go beyond simply recording and uploading existing material. 28Blended learning includes strategic attempts to maximise the combined benefits of digital and in-person instruction.It is more than using technology to make education more efficient -it must be effective.
The benefits of blended learning are increasingly known.Blended learning strategically uses technology to create more creative and stimulating styles of content delivery which increase student engagement. 29Digital instruction can also support accessibility, removing barriers of space and time to education. 30Blended learning can also facilitate the development of profession-relevant skills that might not be learnt in a course that is exclusively in-person. 31t the same time, blended learning retains the engagement derived from strong student-teacher relationships. 32Even in remote learning environments, blended learning should retain the warmth and connectivity provided by faceto-face interactions with teachers, which are important to encourage student engagement. 33n the legal education context, blended learning allows course content to be more efficiently categorised and delivered, which can help to address the seemingly perennial growth of content in law syllabi. 34

Flipping the classroom
FC means inverting traditional classroom and non-classroom activities to the benefit of teachers and learners.In a FC model, students learn the relevant content before class, as opposed to having it delivered to them in class, as happens traditionally. 36In class, students collectively engage in activities -conventionally reserved as after-class activities -to consolidate, explore and test understanding. 37he role of the teacher shifts from delivering content to a combination of imparting knowledge and helping students to apply that knowledge to relevant tasks interactively. 38Going into class with the basics, students can take full advantage of the teacher's expertise, seeking nuanced clarification and developing mastery of the content. 39Ultimately, FC supports a more interactive, collaborative learning environment than conventional lectures.FC is perhaps more familiar than PBL in legal education, which gains unique benefits from it. 40The additional time that a flipped classroom lends to discussions and debates between students can allow students to develop valuable critical thinking and oral communication skills. 41Also, FC gives students opportunities to apply law to facts with the teacher present to check and clarify their process. 42

Learning Modules Programme
Part 3 outlines the Learning Modules programme.First, it summarises the programme.Secondly, it outlines the programme's structure.Finally, it explains how the programme is scheduled.

Overview
Land Law is a compulsory stage three law course, 43 which students normally take in their third year of a pure law degree or fourth year of a conjoint. 44It is a full-year (twosemester) course with two one-hour lectures per week and several two-week rounds of tutorials throughout the year.Students usually take Land Law alongside another compulsory stage three law course (Equity) and between two and four other courses.
Recurring feedback at the end of the year included that tutorials were not as helpful as they could be, and that students did not feel adequately prepared for the exam.
In 2019, university-wide policy changes required adjustments to the assessment.The new policy required that tests and exams could compose no more than 70% of assessment in a course, and that marks could no longer be awarded for attendance.Land Law needed to address the 85% allocation to the test and exam, and 5% allocation to tutorial attendance.
In addition, New Zealand law schools are required to comply with the New Zealand Council of Legal Education's qualification standards.At the time the programme was designed, the standards required at least 60% of a compulsory course to be assessed by exam. 45o fulfil these requirements, the Land Law assessment was revised to a 1500-word essay (15%) and two-hour final exam (70%), with the Learning Modules programme contributing the final 15%.
Land Law is structured as several course topics, 46 five of which are designed as Learning Modules.The university uses Canvas as its online learning management system.The Learning Modules are available on the Land Law Canvas site, which is administered by the Land Law lecturers who publish each component at the appropriate time.

Structure
Each Learning Module is comprised of five components: the pre-tutorial activity; the tutorial; the post-tutorial quiz; and the EOR exercises, comprising modelling exercises and example exercises.Two components are graded and three are optional.This section outlines each component in turn.

43
The LLB degree at the University of Auckland has four stages.In stage one, students study Law and Society, Legal Method and Legal Foundations, as well as electives from another degree programme.In stage two, students study Criminal Law, Public Law, the Law of Torts, and the Law of Contract, as well as Legal Research, Writing and Communication.In stage three, students study Land Law, Equity, and Jurisprudence, as well as law electives.In stage four, students study Legal Ethics, as well as Advanced Legal Research, Writing and Communication, and law electives.Each stage equates to about a year of full-time study for students studying an LLB degree only.For conjoint students, stage two is usually spread over two years to allow students to simultaneously progress their other degree.

44
A conjoint degree at the University of Auckland is when students study for two bachelors degrees at the same time, but do not need to take as many courses in each bachelors degree as students studying those bachelors degrees separately.The LLB degree on its own is a 480 points bachelors degree, which can be completed in four years.By contrast, BA and BCom bachelors degrees on their own are 360 points, which can be completed in three years.BA/LLB and BCom/LLB conjoint degrees are each 675 points, which can be each completed in five years.The entry requirements for a conjoint degree are higher than those for a bachelors degree.

45
The regulations have since been revised to require that at least 50% of a compulsory course is assessed by examination.

46
In semester one, title, interests and priorities; indefeasibility, exceptions and compensation; rights to airspace, subsoil, minerals, water, and fixtures and non-fixtures; and Māori land tenure and customary title, including title to beds of water bodies.In semester two, leases; concurrent interests in land; home units, townhouses, flats and offices; easements; covenants affecting freehold land; and mortgages.

Pre-tutorial activity
The pre-tutorial activity is a scaffolded problem question.The activity is available for two weeks.The problem question is available from the start of the two weeks and students are encouraged to read the question as early as possible so that they can connect the problem to their learning in lectures.While the activity can be completed at any point in the two weeks, students can attend all relevant lectures before submitting it.Each activity contributes 1% to the student's final mark for a total of 5% -a low stakes incentive to complete the activity.
Each activity scaffolds students through answering a problem.Each activity has around 10 questions.The question types vary -some are multiple-choice, some require a one-word answer to complete a sentence and others require a short paragraph.The question sequences step students through the IRAC structure to answering the problem. 47

Tutorial
The tutorial is a small group of around 20 students led by a tutor, usually a highachieving graduate, or postgraduate student. 48The tutorial is based on a problem in the pre-tutorial activity.This helps to address the issue of students coming to tutorials without preparing -having already completed the pre-tutorial activity for 1%, students do not need to invest additional work into preparing for the tutorial and are incentivised to attend to deepen their understanding of the topic.Students bring their answers and the tutor works with the class to develop a sound answer.The tutorial also helps students to prepare for the post-tutorial quiz.

Post-tutorial quiz
The post-tutorial quiz is scheduled after the tutorial round ends and is available to be taken for 72 hours.Each quiz is 20 minutes with eight multiple-choice questions, drawn from a bank of questions (to mitigate the risk of collusion between students), on ideas featuring in the pre-tutorial activity and discussed in the tutorial. 49Each quiz contributes 2% to the student's final mark for a total of 10%.The quiz is the final compulsory component of each Learning Module.

EOR exercises
Upon completion of the post-tutorial quiz, students gain access to EOR exercises to assist with exam preparation for that topic.The EOR exercises comprise two types of exercises.In modelling exercises, students are provided a past exam problem which they are encouraged to plan an answer for, and pre-recorded videos in which a lecturer works through the problem step by step with a top-down camera recording the lecturer handwriting the plan on paper.At the same time, the lecturer provides guidance as to what would make a good answer, and insights into their thought processes when setting and grading the problem.In example exercises, students are provided a problem from a past exam and dozens of past 47 IRAC stands for issue, relevant law (rules), application (applying law to facts) and conclusion.

48
Tutors attend general trainings at the start of the year, attend course-specific briefings before each tutorial round and are able to access continuing professional development.

49
The quiz format allows for automatic grading by Canvas, which reduces the human hours required for marking and helps to make resourcing the programme viable.
students' answers to that problem at each grade, which students are prompted to review to deduce what distinguishes answers at each grade.The EOR exercises are available through the exam period and indefinitely for future reference.

Scheduling
The Learning Modules programme is scheduled throughout the year to coincide with the course topics.The scheduling of tutorials is constrained by the Faculty of Law's standardised tutorial schedule, which requires each tutorial round to run over a twoweek period.With a two-week period for the pre-tutorial activity, a two-week period for the tutorial round and a subsequent 72-hour period for the post-tutorial quiz, the scheduled components for each Learning Module run for about a month. 50

Methodology
Part 4 outlines the study methodology.In 2020 and 2021, students were surveyed at the start of the course (SOC) and the end of the course but before exams (EOC). 51The SOC surveys investigated students' expectations, while the EOC surveys investigated students' reflections.The surveys included five-point Likert scales (strongly agree, agree, neither agree nor disagree, disagree, and strongly disagree), open-ended boxes for students to elaborate, and other open-ended questions.
Descriptive statistics were computed to explore frequencies and distributions for each component.After preliminary analyses, Friedman tests and post-hoc Wilcoxon signed-rank tests were used to compare students' responses across the four surveys.The Friedman test was used as a non-parametric alternative of the Repeated-Measures ANOVA.When using ordinal scales, where the intervals between adjacent options are not necessarily equal, data can only be interpreted in terms of their order.Therefore, a non-parametric approach was required.The Friedman test is a statistical test that can compare results of multiple tests or conditions without assuming normal distribution of data, and helps to identify which groups are significantly different from each other.Non-parametric tests are valid when the sample size is small, and when the data is not normally distributed.P-values lower than 0.05 are deemed to be statistically significant.Except where indicated, no further post-hoc test was conducted as the difference between the distributions was too small.50 Starting 2023, Land Law is scheduled outside the Faculty of Law's standardised tutorial schedule as weekly Monday and Wednesday lectures, and fortnightly Thursday and Friday tutorials, so that the tutorials for each round are condensed in two days rather than spread over two weeks.This means that the tutorials are more proximate to the relevant lectures and every student goes to their tutorial for a round before and after the same lectures.The author considers this to be an improvement to the programme. 51The study was approved by the University of Auckland Human Participants Ethics Committee on 17 March 2020 for three years (reference number 024318).

Results and discussion
Part 5 uses student survey data from 2020 and 2021 to evaluate the extent to which the Learning Modules programme outlined in Part 3 represents the theories discussed in Part 2: PBL, blended learning and FC.The survey data needs to be contextualised.The survey period involved considerable disruption caused by the COVID-19 pandemic.In March 2020, as New Zealand reported its first COVID-19 cases, the New Zealand Government decided to use nationwide -and, later, regional -lockdowns as part of its response.The first lockdown resulted in tertiary teaching moving online for most of the first semester and part of the second semester.In 2021, the disruption was perhaps more profound.The Auckland region entered the strictest level of lockdown in August 2021 and remained at some level of lockdown for over 100 days, with students sitting their exams in lockdown.It is to be expected that, at least to some extent, increasing levels of student stress, anxiety and frustration resulting from the recurring lockdown learning environments will be reflected in the survey results.

Problem-based learning
PBL features prominently in the programme, notably through the pre-tutorial activities and subsequent tutorials.However, certain aspects of the programme stray from the classical PBL theory.

The classical characteristics
Small group learning, tutors as guides and student-centredness.The small group learning and tutors as guides characteristics may be present in the tutorial component.Classical PBL would require that collaborative learning occurs during tutorials.If students are predominantly learning remotely -for example, through the pre-tutorial activity, post-tutorial quiz or EOR exercises -these characteristics are jeopardised.
Whether the tutors are acting as guides will depend on the extent to which they encourage learner groups to review their progress, help them to connect developing knowledge to new contexts, and assist them to clarify misconceptions. 52Tutors cannot dictate the tutorials but must allow students substantial control over the process of solving the problem. 53The tutorials are taken by around five to six different tutors.While many students considered the tutorials to be collaborative, some students at EOC 2021 reported that their tutor simply explained the problem and provided answers.This practice would not reflect a learning environment in which students take the lead and tutors are guides only.
Some results raise flags about student control in some tutorials.In both years, respondents considered pre-tutorial activities to be more helpful than tutorials for developing practical skills.Also, at EOC 2020, many students responded that the pretutorial activities helped them learn how to apply course concepts independently of the tutorials.While these are positive results for the pre-tutorial activity, they indicate that some tutors did not seize the opportunity afforded by the pre-tutorial activity to assume student understanding going into the tutorial and hand more control over to students.
It is difficult to ascertain whether the student-centred PBL characteristic is present.There is insufficient data to determine whether the tutors fulfil their brief to be the guide on the side (rather than the sage on the stage) 54 and regularly respond to their students' learning needs in a way that affirms student agency over the learning process.Ultimately, the signature pedagogies of legal education can make it challenging for legal PBL to be entirely student-centred. 55This is not to say legal PBL cannot exist.If student feedback is actively used to review and enhance student-centredness, and tutors are properly trained to respond to learner needs, then it likely can be defined as student-centred for the purposes of PBL.
The results indicate that the pre-tutorial activities were most effective at increasing participation in tutorials -more so than merely increasing preparation for tutorials or increasing attendance.The Friedman test was used to analyse whether students considered the pre-tutorial activity more effective at preparing students for tutorials, encouraging student attendance at tutorials or encouraging student participation in tutorials.In all four surveys, effective at encouraging student participation in tutorials ranked either first or second (SOC 2020: Friedman chi-squared = 21.117,df = 2, p < 0.001; EOC 2020: chi-squared = 44.400,df = 2, p < 0.001; SOC 2021: chi-squared = 42.074,df = 2, p < 0.001; EOC 2021: chi-squared = 37.345, df = 2, p < 0.001).These results were statistically significant.majority of respondents strongly agreed or agreed that the pre-tutorial activities would help/helped them to prepare for tutorials.
A significant proportion of students agreed that the activities would help/helped them prepare for tutorials: in 2021, 61.2% of respondents strongly agreed at SOC; and 52% of respondents amid a larger sample size strongly agreed at EOC.In each year, about 90% of survey respondents at EOC strongly agreed or agreed that doing pretutorial activities helped them to prepare for in-person tutorials.These results suggest that the problem-based pre-tutorial activities, which students encountered early in their knowledge development, did present a novel learning experience.
Students likely had slightly different conceptions of helpfulness in mind when answering this question.In both years, the survey included open-ended questions that allowed students to provide additional comments.In 2020 at SOC, several respondents believed that the pre-tutorial activities would help them to understand how to approach a problem question, which is the question type most frequently encountered in the exam, and practise applying the law to the facts.In 2021 at SOC, another respondent commented that the pre-tutorial activities were most helpful when they required students to engage with content that they expected to be examinable.So, some students, when considering the helpfulness of the pre-tutorial activities for tutorials, were looking through the tutorial problems to the exam problems, which the tutorial problems emulate.
Table 1 might reflect that students believed the tutorials would be/were more helpful when based on a pre-tutorial activity.In 2020 at SOC, a student noted that the pre-tutorial activity meant they were able to actively participate in the tutorial instead of listening passively.Some responses in 2021 at SOC suggested that students would be more likely to engage -and have less anxiety about engaging -in tutorials having completed a pre-tutorial activity.
Relevance to the profession.PBL requires assigned problems to be professionrelevant.Legal education necessarily involves developing the skill of applying law to realistic factual situations, emulating the process of giving practical advice to real clients.Wong describes how PBL problems in legal education should be "illstructured" or "dynamic" so that they resemble actual practice -complete with irrelevant facts and overlapping areas of law. 59While the pre-tutorial activities are sufficiently practical to partially satisfy this characteristic, they are scaffolded rather than purely THE LAW TEACHER dynamic.Although Wong's suggested style of problem is not essential, it may allow for deeper learning from the PBL process. 60tudents were asked, for each Learning Module component, to indicate using Likert scales whether the component would help/helped them to develop the skill of applying law to facts. 61Students considered that all components would help/helped them to learn to apply the law to the facts (Figure 1).In 2020, for every component, the percentage of students who strongly agreed that the component was helpful increased from SOC to EOC (Figure 2), suggesting that students were able to see the full benefit of engaging in each component more clearly at EOC.Interestingly, the next cohort felt the opposite for the first three components, although students considered the EOR  exercises substantially more helpful for developing the skill of applying law to facts at EOC. 62 In contrast, students thought post-tutorial quizzes were the least helpful component for developing the skill of applying law to facts, which is likely because quizzes place greater emphasis on testing knowledge and interpretation.
Self-directed learning.Self-directed learning requires students to be in a learning environment in which they feel in charge of their own learning. 63The Learning Modules programme clearly demonstrates self-directed learning by giving students choice and accountability in their learning.Students have a large amount of freedom in determining how they engage with the various components of the programme.In each component, students are given some freedom to structure their learning and are rewarded for showing initiative.For example, while a student could let the problem slip from their mind after completing the pre-tutorial activity, they would be rewarded with a more positive tutorial experience if they are intentional about going to the tutorial with a good working knowledge of the problem.
Further, the EOR exercises are intended to reward self-directed students who choose to extend themselves.The optional EOR exercises are available following the posttutorial quiz, at a time when students could "shelve" the module to focus on the next module.Students need to choose to access them for less immediate exam preparation objectives.
Students were asked whether they believed the EOR exercises would be/were a good use of their time.Students were asked to indicate using Likert scales whether they expected doing each EOR exercise would be a good use of their time.Table 2 shows that the majority of students strongly agreed or agreed that it would be/was worth spending time doing modelling exercises.Similarly, Table 3 shows that the majority of students strongly agreed or agreed that it would be/was worth doing example exercises.The results indicate that students believed the EOR exercises would reward self-direction.The results also suggest that the programme inculcated a self-directed learning mindset, in that students were willing to do non-graded exercises despite competing demands on their time. 64These results perhaps reflect the stress, anxiety and frustration resulting from recurring lockdowns that year.In 2021 at EOC, the strongly agree responses were clearly greatest for the modelling exercises, which likely reflects the helpfulness of that component in developing the skill of applying law to facts in preparing for the exam, which would have been at the forefront of students' minds at the time.

64
If it is accepted that traditional teaching of law does not allow for much self-direction, then a comparison of student preference for traditional methods and the Learning Modules programme might reveal the extent to which students took agency over their learning as a result of the programme.See Shulman (n 55) 55.
An Ako New Zealand research report found that when asked what most affected their studies, 49.9% of respondents cited home or family issues and 32.6% cited work responsibilities. 65Many law students also undertake extracurricular activities or an otherwise demanding course load. 66Law students, therefore, frequently engage in prioritisation of their workload. 67Notwithstanding these factors, many students regarded the optional components as worthy of time, which indicates engagement with the components and may suggest student satisfaction with the programme as a whole.
The Learning Modules programme is also differentiated from traditional methods in that the optional content is presented digitally.In 2020 and 2021, students were asked whether they would prefer/preferred digital or in-class delivery of optional exam review exercises.Digital delivery is more accessible but also requires more self-directed learningthe student must decide to access the content rather than being taken through it in class.
Students were asked to indicate using Likert scales if they preferred: the lecturer modelling answers in pre-recorded videos or the lecturer modelling answers in lectures; the lecturer modelling answers in pre-recorded videos or the tutor modelling answers in tutorials; and the lecturer modelling answers in lecturers or the tutor modelling answers in tutorials.Table 4 compares student preferences for: modelling by the lecturer in pre-recorded videos versus modelling by the lecturer in lectures; modelling by the lecturer in pre-recorded videos versus modelling by the tutor in tutorials; and modelling by the lecturer in lectures versus modelling by the tutor in tutorials.
The preference for lecturers modelling in pre-recorded videos was strongest when compared with tutors modelling in tutorials (rather than lecturers modelling in lectures).Further, students preferred both modelling by the lecturer in pre-recorded videos and modelling by the lecturer in lectures over modelling by the tutor in tutorials, which suggests that students value the expect perspective.These results might be taken to indicate that students were sufficiently self-directing as they preferred modelling in asynchronous videos they had to choose to access, rather than in scheduled class time.
Self-directed learning may also be borne out in the relationship between the pretutorial activity and tutorial, where the pre-tutorial activity is designed to facilitate a tutorial environment in which students can lead discussion and take more control of the learning process.In other words, self-direction was required to be intentional about developing a good working knowledge of the problem (rather than completing the activity perfunctorily to get the 1% mark) to be able to engage and contribute effectively in the tutorial.
Students were asked whether doing a pre-tutorial activity would make/made them more likely to attend tutorials.Table 5 shows that most students strongly agreed or agreed at SOC and EOC.In both 2020 and 2021, the proportion of students who strongly agreed increased from SOC to EOC.Student attendance at tutorials typically declines in the second semester of this full-year (two-semester) course. 68It may be that the benefit of pre-tutorial activities is the most obvious at EOC after compulsory pretutorial activities have encouraged students to attend tutorials they may not otherwise have attended in the second semester.However, this is unclear and the proportion of students responding either strongly agree or agree did not increase in both years.

Does the Learning Modules programme represent PBL?
The programme involves work in small student groups, includes profession-relevant problem-solving simulations and allows new information to be acquired through selfdirected learning.However, the problems are not always authentic novel problems.third and sixth characteristics have express equivalents in Boud's variation, as well as other alternatives, suggesting these are near-essential. 73Many authors discuss these factors as if they are near-essential. 74owever, it is important to distinguish between features which are required and features which are important but not required.A distinction must be drawn between essential characteristics and characteristics which are near-essential but can be departed from with adequate justification.Barrows himself notes that PBL could be practised effectively in a teacher-centred way, indicating that at least his first and third characteristics were not intended to be absolute. 75Barrows further notes that teachers can draw on a "wide variety of options" when designing and interpreting PBL, suggesting that the only truly essential feature is student self-directed interaction with aproblem. 76While certain PBL characteristics may carry more weight than others, it seems that most can be missing or adapted without necessarily jeopardising a practice's claim to represent PBL.It is simply a matter of degree.
At the same time, teachers should be aware that removing or adapting a nearessential characteristic may make the PBL practice less effective overall. 77Usefully, some authors have sought to rank the importance of characteristics.Loyens, Kirschner and Paas, for instance, considered the most important characteristic to be problemcentredness, followed by tutors as guides, with other characteristics supporting PBL's goal to facilitate students to solve problems. 78urthermore, the application of PBL in contemporary studies can differ substantially from Barrows' original articulation.Some authors differentiate between small group PBL and self-directed PBL, placing more emphasis on either Barrows' second or sixth characteristics respectively. 79Some authors call for students to literally lead the class in place of the teacher, while others prefer that teachers support students with carefully designed scaffolding. 80Importantly, no authors who were canvassed ignored any PBL characteristics entirely.However, even if most classical characteristics were usually satisfied, there is a clear pattern of contextual variation in how teachers apply PBL.
Therefore, it appears that PBL is a spectrum of degrees rather than scripture.The spectrum's ends have limits.For example, a problem must always be central to the learning process.While characteristics such as student-centredness, tutor as a guide and self-directedness are not essential, the more a teaching practice removes or adapts these characteristics, the less that practice will represent PBL.Nevertheless, PBL is a theory designed to be applicable to teaching in a wide range of educational contexts and is flexible enough to adapt to the law school context by several degrees.

Blended learning
The Learning Modules programme involves a combination of digital and face-to-face instruction, satisfying the basic requirement for most definitions of blended learning.However, there are other conceptions.Charles Graham describes three categories of blended learning: enabling (making learning accessible), enhancing (supplementing one mode of instruction with the pedagogical advantages of another) and transforming (pursuing a radical pedagogical redesign which is only possible with technology). 81tefan Hrastinski further conceptualises quality-focused and quantity-focused blended learning, which aim to maximise the quality of learning and time doing quality learning respectively. 82he Learning Module programme reflects some of these ideas.The programme ensures that content is accessible, by providing online resources to students and incentivising students to engage with those resources.The digital components of the Learning Modules (pre-tutorial activities, post-tutorial quizzes and EOR exercises) help students to reinforce their knowledge of course content, which should reduce stressfor example, resulting from procrastination or intense last-minute preparation before exams. 83The digital components enable students to evenly distribute their study across the year rather than squeeze it in at the end.Students who cram before an exam may understand the law and get their notes in order, but may not allow enough time to practise the skill of applying law to facts.The digital components provide opportunities for students to develop this skill gradually throughout the year so that they do develop it during the course (in the land law context), even if not immediately before the exam.
Students were asked whether they believed doing pre-tutorial activities would help/ helped them to prepare for the exam.Table 6 shows that students at EOC in both years agreed -to different degrees -that the pre-tutorial activities did help them to prepare for the exam.
Similarly, students were asked whether doing post-tutorial quizzes helped them to prepare for the exam.Table 7 shows student responses to this statement at EOC 2020 and 2021.The results reflect a general perception among students that the quizzes were beneficial to their exam study.Interestingly, at EOC in 2021, this is despite most students believing that the post-tutorial quizzes were not helpful in developing the skill of applying law to facts (Figures 1 and 2).
There may be students who have a general preference against digital learningpotentially exacerbated by the COVID-19 lockdowns.Students were asked whether they preferred the lecturer modelling in pre-recorded videos or the lecturer modelling in lectures (Table 4).In 2021 at EOC, the digital mode of delivery was more polarisinga higher percentage of responses strongly agreed and agreed that they preferred the lecturer modelling in pre-recorded videos, a higher percentage disagreed and strongly disagreed, and there was less ambivalence.These results need to be contextualised by the fact that when students were surveyed in 2021 they had been in lockdown for several months, and were unable to access in-person lectures.Notwithstanding these circumstances, which could plausibly give rise to a preference for in-person learning, the results indicate that students preferred modelling in pre-recorded videos over each of lectures and tutorials.This preference remained constant in both years across SOC and EOC, suggesting that students perceive value in the integration of digital learning into the course.While the integration of technology into the course was not novel enough to be transformative, the programme would seem to make learning more accessible (enabling) and supplement traditional in-person instruction with the asynchronous scaffolding potential of digital instruction (enhancing).
Students were asked how each component would enhance/enhanced their understanding of course content.Figures 3 and 4 show that a clear majority of students strongly agreed or agreed that each component enhanced their understanding of course content.However, it is not clear which component students believed would be/was most effective at doing so.
Students generally considered the EOR exercises more helpful in enhancing their understanding of relevant content at EOC (compared with at SOC), when students were preparing for exams.Generally, students considered modelling videos to be the most helpful component for enhancing their understanding of relevant content. 84t is important to note that the extent to which the Learning Modules programme reflects blended learning will be a function of how broad one's conception of blended learning is.This article has largely followed blended learning in its ordinary conception (combining digital and face-to-face instruction).However, it was noted in Part 2 that blended learning can also refer more broadly to the merging of different pedagogies which can be practically combined.In this broader conception, the Learning Modules programme is arguably even more representative of blended learning.The components of the programme clearly combine different pedagogies.For example, the programme combines self-directed activities with facilitated tutorials and prescriptive post-tutorial quizzes.

Flipping the classroom
FC is not merely swapping homework and classroom activities, although that might be said to be the bare minimum.Class time should focus on "skill-developing" with a focus on "how to think, not what to think". 85Ideally, the flipped classroom will take full advantage of pre-class learning to ensure that classrooms are collaborative learning environments facilitating critical group discussions supported by nuanced teacher feedback. 86tudents were asked whether they believed attending in-person tutorials based on pre-tutorial activities would be/was more useful than attending tutorials not based on pre-tutorial activities.Table 8 shows that most students strongly agreed or agreed that the tutorials were more useful with a pre-tutorial activity, with at least half of 84 In 2021 at EOC, students considered post-tutorial quizzes to be by far the least helpful component in enhancing their understanding of relevant content; however, the result was quite different in 2020 at EOC, so it is not sensible to draw any conclusions.respondents in both cohorts strongly agreeing that attending in-person tutorials based on the pre-tutorial activities was more useful than attending traditional tutorials not based on the activities.These results indicate that the tutorials presented something novel in the flipped form.Given that students had already attempted an answer to the problem, students were better prepared to lead discussion.Students ultimately considered the flipped classroom to be beneficial to their learning.
In the Learning Modules programme, the pre-tutorial activities and tutorials in combination embody the FC model.Students are required to attempt a pre-tutorial activity for 1%, which seems to be enough to motivate law students to attempt an activity.As such, almost every tutorial attendee will be familiar with the problem, including the issues and relevant law, and would have attempted to apply the law to the facts on their own prior to the tutorial.Thus, the tutorial can focus on developing students' skills by facilitating students to work through the problem collaboratively with their peers, guided by the tutor.
Whether the programme truly represents FC will depend on the extent to which the tutorial facilitates collaborative engagement.In any case, while the pre-tutorial activities and tutorials in combination may represent FC, the pre-tutorial activities compose only 5% of assessment in the course and the number of hours students spend in tutorials during the year is about 10% of the number of hours students spend in lectures, which are generally still traditional, although the teachers are increasingly using a flipped classroom in lectures too.

Conclusion
The Learning Modules programme features aspects of all three theories: PBL, blended learning and FC.The programme clearly represents blended learning.Whether it can truly be considered to represent PBL and FC largely depends on what happens in tutorials.Both theories require student collaboration in tutorials.To mitigate the risk that a particular tutor is prescriptive rather than facilitative, lecturers need to invest in training their tutors, including inculcating the reasons for being more facilitative (as discussed in this article), and review the tutorials on an ongoing basis.Also, tutors, who are usually graduates and postgraduate students, need to be encouraged to persevere with facilitation despite concerns about student unpreparedness, which can cause teachers to get cold feet about being guides on the side. 87There are further potential issues for PBL in particular -for example, the problems may be too structured to be considered sufficiently profession-relevant problemsolving simulations, and students may be taught the relevant content in lectures prior to attempting each pre-tutorial activity, which raises questions about whether the problems are authentic novel problems.Loyens, Kirschner and Paas (n 3) 411.

THE LAW TEACHER
Ultimately, the programme is designed to accommodate all three theories, but the degree to which it represents PBL and FC largely depends on how the tutors conduct tutorials.If tutors are facilitative, the programme might be said to represent each theory.
Law and Practice Examination for one or more required courses.New Zealand Council of Legal Education, Professional Examinations in Law Regulations 2008 (2017); and New Zealand Council of Legal Education, New Zealand Law and Practice Examination: Information for the July 2023 Examinations (2023).See also Lawyers and Conveyancers Act 2006.Paul Maharg, "'Democracy Begins in Conversation': The Phenomenology of Problem-Based Learning and Legal Education" (2015) 24 Nott LJ 94, 95-96.See also Howard S Barrows and Robyn M Tamblyn, "Problem-Based Learning: Rationale and Definition" in Howard S Barrows and Robyn M Tamblyn (eds), Problem-Based Learning: An Approach to Medical Education (Springer 1980) 1.

Figure 1 .
Figure 1.Percentage of students responding strongly agree or agree that the component would be/was helpful in developing the skill of applying law to facts at SOC and EOC in 2020 and 2021.

Figure 2 .
Figure 2. Percentage of students responding strongly agree that the component would be/was helpful in developing the skill of applying law to facts at SOC and EOC in 2020 and 2021.

Figure 3 .
Figure 3. Percentage of students responding strongly agree or agree that the component would be/was helpful in enhancing their understanding of relevant content at SOC and EOC in 2020 and 2021.

Figure 4 .
Figure 4. Percentage of students responding strongly agree that the component would be/was helpful in enhancing their understanding of relevant content at SOC and EOC in 2020 and 2021.
Student-centred learning is best facilitated in 6 Heidi Maurer, "Best Practices in Problem-Based Learning" in John Ishiyama, William J Miller and Eszter Simon (eds), Handbook on Teaching and Learning in Political Science and International Relations (Edward Elgar Publishing 2015) 369, 372.
Most law school-based approaches to PBL manifest the student-centred characteristic of PBL in a process by which students identify the nature and scope of the problem, independently research and gather material, and collaborate to solve it, with the teacher providing scaffolded knowledge 14 Maharg (n 5) 96.See also David Boud (ed), Problem-Based Learning in Education for the Professions (Higher Education Research and Development Society of Australasia 1985).
Flipping the Law Classroom to Ease Student Anxiety" (EdTech Focus on Higher Education) <www.edtechmagazine.com/higher/article/2013/12/flipping-law-classroom-ease-student-anxiety>accessed 7 April 2023; Tenielle Fordyce-Ruff, "Teaching Active Listening: Flipping Roles in Client Interviewing Exercises" (2014) 22 Perspectives: Teaching Legal Research and Writing 131; and Debra D Burke, "Scale-Up!Classroom Design and Use Can Facilitate Learning" (2015) 49 The Law Teacher 189.See also Jane McNeil and others, "SCALE-UP Handbook" (Nottingham Trent University 2018); and Eimear Long, "Use of Flipped Learning in the Legal Skills Classroom to Improve Outcomes for Law Students" (Dublin Business School 2019) <https:// esource.dbs.ie/bitstream/handle/10788/3757/long_e_dbs-rday_2019.pdf>accessed 7 April 2023. 41Dan Berger and Charles Wild, "Refining the Traditional Flipped-Classroom Model to Optimise Student Performance on Undergraduate Degree Programmes" (2016) 11(1) JCLLE 57, 63.The online delivery of preclass content via an easily accessible platform assists exam revision, while providing more time for practical Lilia Gomez-Lanier, "Building Collaboration in the Flipped Classroom: A Case Study" (2018) 12(2) IJ SoTL (Article 7) 1, 1. 39 Erol Sahin (n 36) 114.40 Julie Tausend, "exercises during face-to-face sessions.Lutz-Christian Wolff and Jenny Chan, "Case Study: Flipped Classrooms for 'The Law of International Business Transactions II'" in Lutz-Christian Wolff and Jenny Chan (eds), Flipped Classroom for Legal Education (Springer 2016) 81, 102.42 See Dawn Jones and Lynn Ellison, "When Is a Word Not Just a Word?An Investigation into the Dissonance and Synergy between Intention and Understanding of the Language of Feedback in Legal Education" (2021) 55 The Law Teacher 155, 162 and 163.University of Auckland LLB students learn legal reasoning in terms of "applying law to facts".The author is confident that most respondents would commonly understand what "applying law to facts" requires of them in practice, even if they cannot articulate that as a general, abstract instruction.

Table 1 .
Student responses to "doing pre-tutorial activities will help/helped me to prepare for in-person tutorials" at SOC and EOC in 2020 and 2021.

Table 2 .
Student responses to "watching modelling videos will be/were a good use of my time" at SOC and EOC in 2020 and 2021.

Table 3 .
Student responses to "reading previous students' exam answers will be/were a good use of my time" at SOC and EOC in 2020 and 2021.

Table 4 .
Comparing student preferences for modelling by the lecturer in pre-recorded videos, the lecturer in lectures and the tutor in tutorials at EOC in 2020 and 2021.
65 Lynne Taylor and others, "The Making of Lawyers: Expectations and Experiences of First Year New Zealand Law Students" (Ako Aotearoa 2015) 22. 66 ibid 22. 67 Natalie Skead and others, "If You Record They Will Not Come -But Does It Really Matter?" (2020) 54 The Law Teacher 349, 353.

Table 5 .
Student responses to "doing a pre-tutorial activity will make/made me more likely to attend tutorials" at SOC and EOC in 2020 and 2021.
68The author has taught the course since 2018 and can attest to tutorial attendance data for the course in 2018 and 2019.

Table 6 .
Student responses to "doing pre-tutorial activities helped me to prepare for the exam" at EOC in 2020 and 2021.

Table 7 .
Student responses to "doing post-tutorial quizzes helped me to prepare for the exam" at EOC in 2020 and 2021.

Table 8 .
Student responses to "attending in-person tutorials based on pre-tutorial activities is more useful than attending tutorials not based on pre-tutorial activities" at EOC in 2020 and 2021.