Scaffolding for Success in Problem-Based Learning

Problem-Based Learning (PBL) is based on an alternative pedagogical model to the conventional, didactic one, and offers benefits to the quality of student learning. The approach has been adopted by many institutions. The focus of this paper is on the influence of the learning support structure in an environment (such as the typical PBL environment) that encourages student independence as one of its basic tenets. The immediate reaction might be to assume that it takes on a reduced importance in such circumstances. That assumption is challenged in this paper.

The concept of "scaffolding" may take many forms 1 ; here it is used in a wide context to refer to all forms of learning support, whether or not they be cognitively-based, logistical, etc.Thus, the term is used to broadly refer to the range of services provided to assist learning.This is of interest as Problem-Based Learning (PBL) approaches to education suggest a strong role for factors such as authenticity and student independence.With the shift of emphasis that moving towards a PBL approach implies, a re-examination of the nature of what it means to offer meaningful support to learners is important.In this paper the relationship of scaffolding to the success of PBL programs is examined.In this context, "success" is measured largely in terms of those benefits with which PBL is associated, principally in the encouragement of "deep" (or personally meaningful and potentially transformative) approaches to learning.Elsewhere, deep approaches to learning have been described in terms of higher levels of integration of knowledge and greater tolerance to complexity 2 .
The paper begins by briefly referring to the definitional dilemma that hinders attempts to compare PBL programs.A working definition is established.Desirable outcomes of PBL are then presented, with an emphasis on "deep" learning approaches.This is then countered by raising some concerns about the performance of existing PBL programs, while hinting that appropriate scaffolding may offer relief from some of these concerns.An account of the role of the tutor in PBL follows, as an obvious source of scaffolding.Scaffolding is then discussed in terms of improving the likelihood of successfully encouraging meaningful approaches to learning in PBL students.

Defining PBL
A review of the literature reveals an extensive coverage of PBL case studies, many based on differing concepts of PBL. 1 These concepts may directly affect the reported success or failure of the approach; for example, some authors refer to PBL as a mandate for "discovery learning" and then report that this introduces implementation problems in terms of required subject-matter coverage. 3Furthermore, as a reaction to this concern there is a risk of staff enforcing rigid course objectives to ensure subjectmatter coverage, with a return to what the authors refer to as "curriculopathy" (disease of the curriculum).
Although some variation in what is labeled as "PBL" may be aberrant, it is this author's observation that much of PBL's wide range of possible manifestations is to some extent inherent to the approach, as one which does not prescribe fixed and narrow pathways for learning.However, there are some essential criteria that constrain the definition of PBL.Importantly, PBL is identified as a constructivist pedagogy.Savery and Duffy 4 summarize some of the central tenets of constructivism: • Understanding is based on experiences with content, context, the learner's goals, etc., and these factors are inextricably woven together.
Thus, understanding is a construction that is unique to the individual.• Meaning is not transmitted, although it may be tested for compatibility with the meanings of others.From another perspective, cognition may be regarded as being distributed rather than individually localized.
• Puzzlement is the factor that motivates learning.
• Social negotiation and the ongoing testing of the viability of existing concepts in the face of personal experience are the principle forces involved in the evolution of knowledge.
The definition of PBL that is used here is that it is a subset of problem-centered learning methods which is easily identifiable by the use of typically illstructured problems which precede and motivate learning, and act as a vehicle for encouraging student ownership of the learning environment.Usually there is an emphasis on contextualization of the learning scenario, providing a basis for later transference, and the learning is accompanied by reflection as an important meta-cognitive aspect of PBL.Implementation of PBL programs usually occurs via group-based work, reflecting the constructivist focus on the value of negotiated meaning.A characteristic of PBL which follows from many of these attributes is that it is unconfined by discipline boundaries, encouraging an integrative approach to learning which is based on requirements of the problem as perceived by the learners themselves.Schmidt 5 summarizes PBL in terms of three essential principles: 1. Activation of prior-learning via the problem; 2. Encoding specificity such that the resemblance of the problem to intended application domains facilitates later transfer (leading to an emphasis on authentic learning environments); and 3. Elaboration of knowledge via discussion and reflection to consolidate learning experiences.

Desirable Outcomes of PBL: Deep Learning Approaches
Traditional approaches to preclinical medical education have been criticized as devoid of crossdisciplinary integration, having insufficient interface with clinical problems, and exhibiting insufficient retention of basic knowledge and students' inability to apply to actual cases, as well as concerns over accuracy of the knowledge base.

6
As an alternative to traditional methods, Robbs and Merideth 7 list a number of advantages that are associated with PBL modes of learning: • an increased retention of information; • the development of an integrated (rather than discipline-bound) knowledge base; • an encouragement towards lifelong learning; • a greater exposure to clinical experience and at an earlier stage in the curriculum; • an increased student-staff liaison; and • an increase in overall motivation.

Marton and Saljo
8 and the Gotteborg educational research group 9 have focussed on qualitative differences in individual approaches to learning, with the terms "surface" and "deep" used to reflect the degree of engagement and depth of understanding of the material.The word "approach" is emphasized here to indicate that this is not necessarily inherent to the learner, but may be variously (and possibly even strategically) applied by students in response to external factors.This establishes a position where the learning environment shares in the responsibility for encouraging more meaningful ("deep") approaches to learning.
Problem-Based Learning (PBL) has been claimed to encourage deep learning in students. 10,11.These studies indicate increased use of meaningful ("deep") approaches by PBL students in relating to the material, and decreased use of reproductive ("shallow") approaches.
PBL incorporates a number of factors that seek to encourage this meaning-orientation to learning approaches, such as high motivation, learnercenteredness, and group-based activity requiring peer negotiation. 12A study by Sobral 13 generally supports these propositions, finding that "the results ... reinforce the idea that problem-based learning, even in a single-course experience, may enhance the emotional well-being of the participants and the quality of the learning environment, as expressed by the learners themselves" (p.100).The current review relies heavily (although not exclusively) on student self-perception of the quality of learning, due to the availability of such information in the literature.It is reasonable to cast these student perceptions about quality of learning in the company of other perceptions in order to develop a feel for possible relationships that may be worth exploring.While this paper is primarily concerned with relationships between scaffolding and deep learning, it may not be possible to do so in complete isolation of other factors.
There is evidence in the literature for relationships between quality of learning and other factors.For example, Kember and Gow 14 summarize arguments which find correlations between reproductive learning approaches and such factors as high workload, assessment methods which reward recall, low levels of motivation in the course, and overlyconstrained learning environments.
PBL has a strong reputation for enhancing student motivation towards learning tasks and providing an unconstrained environment by means of student empowerment, thereby removing two of these possible indicators of shallow learning approaches.However, it is the author's opinion that PBL occasionally occurs in the presence of assessment schemes that are inappropriate to its orientation, and it is well documented that PBL may involve perceptions of increased workload.These areas will be explored here.

Ma
15 surveyed 3rd-year students of databasesystems using Biggs' Study Process Questionnaire 16 , and compared the results with the outcomes of PBL projects.He found a linear relationship between good student outcomes in the projects and indicators of deep or achieving learning processes, and conversely, shallow learning strategies were found to be negatively correlated with good marks.However, this relationship is not explored, and the report suggests that more evidence is needed before claims can be made for correlations between deep learners and better performance in PBL projects.
The increased motivation associated with PBL and evident in student surveys and staff observations of increased student attentiveness [17][18][19][20] is often regarded as an important outcome.Certainly, increased motivation is a desirable result.However, in this survey motivation is given a different emphasis.
First, the focus is on quality of learning.Second, motivation is regarded here as essential for the success of PBL in the face of some of its strong demands, and therefore is treated more as input rather than an output.This will be discussed further in the context of the cognitive demands made by constructivist approaches to pedagogy.

Concerns Raised in Response to PBL Cases Studies
The literature review conducted by Albanese and Mitchell 1 indicates a number of issues that may be of concern, as evidenced in the case-study literature.In the current review such issues are loosely classified in terms of suggested explanations for them: problems inherent to a process of change, problems emerging from traditional means of assessment, and problems relating to inherent difficulties with PBL as a mode of learning.

Issues Inherent to Change
Pereira, and colleagues 17  However, it is in just such broad discipline areas (most noticeably medicine) that PBL has made its greatest impact.The high value placed on experience may support a teacher-centered model of education in which the teacher embodies experience which is transmitted to students, or it may equally support a constructivist model which encourages the exposure to such experience by students.Much of this commentary on discipline-based inherent propensity for transmissive pedagogies is likely to be attributable to historical and political forces, and may more productively be associated with inherent difficulties brought about by the more general process of change itself.
O'Hanlon, Winefield, Hejka & Chur-Hansen 21 report on student evaluations of a first-year medical program run in parallel PBL and traditional modes, and note that although students were highly motivated by the PBL approach, overall the traditional approach was favored (found "more beneficial").Many students were undecided.Occurring as a firstyear program it is likely that one influencing factor on these results was student inability to acclimatize to the PBL philosophy, which is fundamentally different from the individualistic and competitive approaches typically rewarded during secondary schooling.This current paper identifies in this an opportunity for improving the situation with appropriate scaffolding to lessen the severity of the impact associated with any dramatic change.That is, change can be managed in order to reduce the sever-ity of its impact; this represents one aspect of the need for scaffolding within PBL.

Issues of Performance and Traditional Assessment Instruments
One of the focus questions of the literature review by Albanese and Mitchell 1 concerned the performance of PBL medical students in basic science exams such as the NBME I (the acronym refers to the National Board of Medical Examiners) and the USMLE I (the United States Medical Licensure Examination).They refer to a number of studies which suggest that PBL students may exhibit lower performance in such exams compared to students from conventional courses; it is important that they note that this is not a universal finding, and suggest that one explanation for the varying results may be due to different approaches to the delivery of PBL.
In a PBL course that integrated pharmacology into 1st and 2nd year medicine, 24 it was found that performance of PBL students in the traditional evaluation system was favorable, despite the concerns expressed elsewhere.Two of three groups studied performed in the NBME I with slightly lower means than the national average (for pharmacology) while the remaining group equaled the national average in the USMLE I.The latest three groups equaled and bettered the USMLE I national average.They note, however, that the USMLE I examination often integrates questions across discipline boundaries, and raise the question as to whether or not this offers an advantage to PBL students.McGregor and colleagues 19 were unable to register differences between didactic and PBL modes of learning in subject-based performance tests of medical students in their third-year clerkship.Lieux 25 resents the results of a study of PBL and conventional stream students in a food and nutrition course, and finds that they do equally well in exams, although PBL students exhibited a significantly higher attendance rate.

Inherent Demands Associated with Valued Learning
It is reasonable to suppose that more valued learning may involve greater demands on student learning processes.Lieux 25 notes that student surveys of parallel PBL versus conventional-stream modes of learning show that PBL students see themselves as working harder.Perkins broad ways in which constructivism makes strong demands on learners: 1. Cognitive complexity is a feature of constructivist learning environments, in which authenticity is sought.This is particularly the case where pre-existing knowledge may be naive (such as is common in science).Perturbation is a constructivist device that exposes these misconceptions, and forces the learner to confront them.It may be likened to an induced anxiety in which the testing of concepts against reality exposes significant (and unexpected) shortcomings in the existing concepts and intuitions.Thus, the resulting cognitive load may be high.However, if this is the cost of non-avoidance of misconceptions, then it is a cost that may be regarded as worthwhile.Perkins suggests that a middle ground exists (which he refers to as the "conflict deferred" approach), where learners may postpone dealing with misconceptions until some grounding in the new domain is achieved.This is essentially an argument for early scaffolding, although it is not presented as such.Clearly, regardless of cognitive cost concerns, the "conflict buried" approach which allows misconceptions to continue is not an option if meaningful learning is desired; such an approach may be permitted in non-constructivist environments in which assessment and presentation of content reward shallow, reproductive learning strategies.
2. Task management is increased in constructivist settings, as a necessary condition for students to exhibit learning independence and a function of the emphasis on student-centerdness.The danger is in not permitting time for gradual development of these essential managerial skills, something that would be magnified in the presence of cognitive complexity.This is where constructivist techniques such as "cognitive apprenticeship" may be useful in reducing the load.Savery and Duffy 4 identify cognitive apprenticeship as a natural device for assisting in metacognitive processes in PBL environments.That is, early scaffolding of skills essential to the constructivist process can benefit the functioning of those processes.Peterson 27 establishes a number of important skills that enhance PBL.Their ability to offer such enhancement suggests the importance of developing them, rather than simply expecting them to grow in response to need.
3. "Buying in" to the PBL approach is another area of possible difficulty.Students may not identify with the reasons given for not simply being told the information which teachers want them to know.That they are asked to adopt such a vastly different paradigm of learning to that in which they typically feel comfortable at the same time as they are asked to learn new material adds up to a potentially significant set of obstacles.These difficulties alter the way in which the motivational benefits associated with PBL are perceived.It is not apparent how scaffolding can bridge such perceptions, and it may be argued that the change required of the culture of student learning is so fundamentally different as to be incapable of full appreciation from within the instructivist, transmissive world-view.If such a proposition is valid, then the motivational benefits of PBL are not simply a desirable side effect, but essential to its success.If the importance of meaningful approaches to learning is obscured by preconceptions of education then high levels of motivation offered by PBL approaches may be an essential bridge towards new learning habits.
One may be moved to suggest that in this context motivation itself is a form of scaffolding that is fundamental to the success of PBL.

The Effect of Tutors on PBL Success
The last section presented some reasons for scaffolding.The tutor may be recognized as its most evident source.Zimitat and colleagues 18 report that student surveys revealed that 70% of students in a PBL course found the tutors' roles essential to the success of the method.
The desired role of the tutor in PBL must be quite different in nature to that of a tutor in a didactic system.Jones, Donnelly, Nash, Young and Schwartz 28 refer to two essential roles of the PBL tutor: facilitation of the learning process via prompting, and assisting in group processes to ensure that they maintain focus.They also mention that tutor training includes refraining from assuming the role of an authoritative source of knowledge.It is not apparent that the tutor's desirable relationship to subject content is well-understood; this is an issue which is relevant to a pedagogy such as PBL which values student-centered approaches to learning, thus bringing the role of the teacher's relationship to knowledge into question.Ambury subject-based expertise as a disadvantage to studentcentered learning and independence.However, Eagle, Harasym and Mandin 30 find that tutor expertise favors greater congruence between learning issues and case objectives, and stimulates greater numbers of learning issues to be explored.Sobral, 13 in comparing staff-tutored groups with peer-tutored groups in a PBL medicine course at the University of Brasilia, witnessed no significant difference between them in terms of problem solving or student selfevaluation of skills.However, scores for meaningfulness of learning and usefulness of group work were found to be significantly higher in peer-tutored groups.The level of tutor expertise is clearly one area that would benefit from continued research.
The importance of the tutor (without specification as to whether or not that role is filled by staff or advanced peer) is emphasized by Gijselaers and Schmidt. 31They found a causal relationship between tutor involvement in PBL and group processes, which in turn affects student motivation towards learning.The importance of such motivation to the success of PBL has already been discussed.
Other correlations have been proposed.It is widely recognized that the tutor role in PBL (or more widely, constructivist) learning environments should be one of facilitation rather than instruction. 14More widely, Kember and Gow 14 have found correlations between staff attitudes to teaching roles (facilitatory vs. instructional) and student approaches to learning (deep vs. surface): ...in departments where the predominant orientation is toward knowledge transmission, the students' use of deep approach is likely to decline through the period of the course of study.
On the other hand, departments with a propensity toward learning facilitation tend to discourage the use of surface approaches.The effect of an orientation toward learning facilitation is perhaps more likely to manifest itself in this way rather than as a positive advance in the use of a deep approach because of the difficulty of inducing students with a propensity toward a surface approach to consistently adopt a deep approach.(pp.67-8) Thus, there is a relationship between teaching role and quality of learning adopted by students.It may be suggested that the facilitatory role of the tutor in PBL is essentially one of providing scaffolding, and that the relationships that have been attributed to tutors may possibly be attributable to appropriate use of scaffolding in general, with the implication that this may extrapolate to other forms of scaffolding.This may be a valuable area of further research.
Clearly, if the effect of tutors play such an important role in the success of PBL, it is important to recognize that a mechanism for feedback regarding tutor performance is desirable.Dolmans, Wolfhagen and Snell-Balendong 32 developed an evaluation form to allow formative assessment of tutor performance by students.It focuses on three areas: guidance of students through the learning process (6 questions), content knowledge input (4 questions), and commitment to the group's learning (3 questions).These areas were validated by a confirmatory factor analysis.Questions were responded to simply by indicating on a three-point Likert scale whether the result was judged to be insufficient, neutral, or sufficient.
The experiences with such evaluations at the Medical School at the University of Limburg have been rated positive, and has resulted in a increase in teaching focus.

Attributes of PBL in Relation to Scaffolding Needs
It is clear that the effect of the tutor is an important one in PBL, and that the tutor is well placed to provide scaffolding to learners.Here, the discussion turns to consider scaffolding in terms of the main attributes of PBL.

Student Control
It is important to note that PBL's emphasis on student independence and learner ownership of problems does not preclude the use of scaffolding to assist in the development of these attributes and other skills required to enable meaningful learning.PBL lays the responsibility and, ultimately, the control of the learning process essentially at the feet of the student; however, this is not a process of abandonment.Koschmann and colleagues 6 state that "there is a well-defined curricular infrastructure under girding the process.Cases are developed from actual patient records and are presented in the way actual patients present for care: as an ill-structured problem that must be built by inquiring to gather necessary information..." (p.242).

Authenticity
Authenticity requires a level of "fidelity" to ensure that learning occurs in sufficiently real-world context, and also avoidance of the provision of information that a real-world context would not provide.6   Honebein, Duffy and Fishman 33 identify a number of elements, which lend authenticity to a task: • Learner ownership: This is supported by the argument that metacognition is essential to function well in complex environments and therefore they must be supported in developing a sense of responsibility for their management of problem-solving tasks, which suggests problems ownership.
• Project-based nature: This suggests a holistic representation of the task, with opportunities for authentic global (wider context) entities as well as more localized ones.
• Multiple perspectives: The empowerment of students to consider multiple perspectives when examining a problem domain is an important mechanism for developing expertise.One means for encouraging this is in the use of collaborative learning environments, as are typically used in PBL programs.
Problem authenticity is a feature of constructivist approaches to education, and is a readily recognized aspect of PBL, as shown by the above criteria.
Jayawickramarajah 34 examined the nature of six PBL problem documentation sets (via a process of content analysis) over three medical schools that revealed a number of factors influencing the construction of problem fidelity: • Presentation format may vary from patient simulations and video recordings of cases (high fidelity) to "written simulations" in which the case is described on paper (exhibiting low fidelity).• Manifestation may be ill structured and given as an individual variant on the range of possibilities (high fidelity) or may be a comprehensive list of the pool of manifestation possibilities (low fidelity, textbook-like).
• Associated problems may be included which complicate the scenario with possibly unrelated factors such as symptoms produced by medication or behavioral problems (high fidelity); alternatively, the problem may be filtered to present an isolated set of conditions (low fidelity).
• Context may resemble that of actual situations such as those that afford the opportunity to talk to family members of the patient (high fidelity), or may be more contrived (low fidelity).
• Objectives may be defined in terms of the health of the patient (high fidelity and integration across disciplines) or may be discipline-bound (low fidelity and artificial).
He found that low-fidelity problems such as those stylistically borrowed from textbook models of problems to be "counterproductive in terms of the objectives of PBL.In this context textbook derived health problems are not considered very appropriate" (p.

Group Processes
A study by Tipping, Freeman and Rachlis 35  Students later claimed that reasons for this difference between observed and reported behaviors included the intrusive effect of the video (although video was only one means of observation, and doesn't appear to be a mechanism that would permit some of the behavior more easily guarded against, such as sleeping!);lack of certainty as to expected roles (which is akin to suggesting that the group dynamics didn't work because they didn't work!); and lack of tutor guidance.It is suggested here that the last one is the only one with any plausibility, and emphasizes the importance of the role of the tutor in PBL.It is, most emphatically, a request for greater scaffolding.The authors suggest reasons of minimal prior exposure to group dynamics (especially as individual competitiveness is a characteristic of premedical curricula) and lack of formal training in this area; it was further suggested that students lacked a model of group effectiveness.Tutors, despite some training in the area, did not facilitate reflection on group processes, leading to the conclusion that even greater emphasis needs directing at this area in future staff training programs.Such failures in group processes may be interpreted as failures in the provision of appropriate scaffolding in both tutor facilitation of group processes and in establishing a successful mental model of interaction within the group.
While on the topic of perceptual incongruities, Lieux 25 mentions the results of student course evaluations that revealed that PBL students perceived themselves as learning less content than lecture-stream students, and, conversely, lecture-stream students regarded themselves as learning more; this is in spite of the equality of performance even in traditional assessment schemes.Certainly, despite some implementations of PBL leaning towards discovery learning, content is deliverable within a PBL approach . 36In addition to learning content, however, PBL students regarded themselves as learning problem-solving, communication skills, and developing a sense of "personal responsibility"; these were not found in the feedback from the lecture-based students.Similar results have occurred in the author's experiences.If the group is to work effectively, some effort must be directed towards it, and facets of group dynamics such as those described need to be given recognition within the course.Hebert and Bravo 40 have used these processes in developing an instrument for measuring all facets of PBL behavior.Tipping and colleagues 35 recognize a number of aspects of group dynamics in their Group Observation Guide: physical and emotional climate, individual involvement, interaction, cohesion, productivity and leadership.
Their study reveals that both students and staff were largely limited in their reporting of group processes to climate, interaction, productivity, and leadership, with varying levels of detail.Involvement and cohesion were unmentioned.

Discussion
The concerns regarding performance of PBL students in traditionally-valued assessment modes is matched, as we have seen, by results which indicate comparable performance.The variation in these results becomes as interesting as the initial question.

Robbs and Merideth
7 refer to a number of "myths" which have surrounded PBL.Their observations at the University of Southern Illinois School of Medicine are that PBL students do as well in the USMLE I exams as conventional students.They also refer to PBL as having a hidden structure; perhaps it is the case that this structure is absent in some PBL programs.These results reflect the advantages commonly associated with the PBL approach.Interestingly, (as this is a first-year course) the transition to group work does not seem to have been problematic.The favorable impression of group work that is consistently reported is interesting in light of Sobral's 13 findings that there is a statistically significant positive correlation between PBL scores for meaningfulness of learning and both group-work and selfevaluation.Similarly, motivation was shown to correlate positively with problem-solving scores.
There are essentially two factors here -time demands and the need for scaffolding.

"Do you have any comments on the general use of a problem-based approach to teaching medical knowledge?"
• Time consuming (29.4%); and • More guidance required (23.5%).
Once again the emphasis on scaffolding, and time is evident.Of course it is difficult to extract a sense of the extent to which this is an expression of the resistance to change (recalling that this is a first-year medical PBL program).
Cawley 42 produces a summary of results for PBL survey student evaluation in a mechanical engineering course (as a final-year option) on a 5-point Likert scale over 3 years, with sample sizes ranging from 21 to 37. The stable results (i.e., those which maintain their result over the years) indicate positive results for a number of factors, scoring well for interest (mean of approx.4.6), enjoyment (mean approx.4.3), relevance due to authenticity (approx.mean of 4.5), and an understanding of technical knowledge equal to that expected from a conventional approach (mean of approx.4.0).
Trends indicated through the results over the years are increasing feelings of quantity of knowledge being equal to conventional approaches as the course progresses (progressive means of 3.2, 3.2, and 3.8), and an increase in time required of PBL in comparison with lectures (means of 3.2, 4.2, and 4.4).
With respect to the time required, the first result of 3.2 (indicating that it takes about as much time as the conventional course) is discussed in terms of the time being greater than student estimations, since (p.183) "students could frequently be heard discussing the problems in the coffee room, an almost unknown occurrence on conventional courses, and this valuable peer learning may not have appeared in their estimates".The perceived rise in time demands of PBL in the 2nd and 3rd years of the course is proposed as possibly due to increased coursework requirements in other courses, and accompanied by staff perceptions of less time spent on PBL projects in those years.
Although it may not be immediately apparent, one means of support of the student in the presence of perceptions of increased workload may be via alterations to the assessment scheme, offering greater recognition of the work performed.Zimitat and colleagues 18 found that over half of students surveyed in a PBL course (n=231), and of whom 70% were motivated by the approach, expressed the opinion that the assessment weight for PBL cases should be increased due to the perceived extra workload in PBL.
Other mechanisms of support through the assessment system are possible.Ma 15 employed a negotiated assessment program in a course on database systems (of course, staff are essential elements in the negotiation!).
The change to a PBL curriculum may inherently suggest redesign of assessment.Sokas and colleagues 43 suggest that PBL may have benefits that are difficult to quantify, and note that in response to this supposition some PBL programs have opted for modification of testing towards reduced emphasis on recall, or have been satisfied with lower scores on traditional testing methods such that they are not significantly worse than conventional teaching approaches would generate.
It is important to cast PBL in terms of its original motivations, and note that it does not explicitly attempt to satisfy assessment schemes which reward shallow learning approaches.Additionally, PBL implies wide-ranging changes to the values of traditional education, and cannot be realistically applied "on top of" existing infrastructure.Ambury 29 refers to PBL in medicine as part of a larger move away from "basic sciences and organ systems as the organizing principle for the curriculum" (p.1).None of this, as has been seen, necessitates a disregard for factual information.Pereira and colleagues 17 studied student evaluations of PBL medical programs at the University of the West Indies, and found that only 4 percent were dissatisfied with their knowledge base obtained via PBL learning modes.DesMarchais and Vu 3 state that a move to emphasize analytic and problem-solving skills in areas of (medical) basic and clinical science using small groups increases a need for assessment.The issue of accountability becomes an important one.With respect to preclinical skills reproduction of facts is still valued and is measured by multiple-choice questions, and shortanswer questions with a content-based focus.The problem-solving and analytic skills are tested with newly-developed instruments in the form of problem-analysis questions and by oral exam.Smallgroup competencies (such as self-directed learning, etc) are tutor-evaluated by use of a form completed after each session.Initial (staff/student) concern over the subjectivity of the tutor-based evaluation instrument subsided once it was established that it was able to identify students with problems.It was found that tutor-based assessment shows most discrimination in the presence of long-term problems and when mean ratings are sufficiently high to encourage tutors to use low rating scores for weaker students.In the absence of a high mean all scores tend to be clustered rather than exhibit this (desirable) bimodality.
Sivam and colleagues 24  The factor analysis was unable to support their initial position that effective use of PBL and effective group communication were separately identifiable, and indicated existence within the one factor.They retrospectively identify this as a natural association, due to the group-based nature of PBL.Here, it strengthens the argument for emphasizing the group processes in PBL environments as a factor for increased successful usage of the PBL style (i.e., "getting with the program" requires feeling comfortable with small group dynamics).This inevitably carries through to the tutor, given that 1) group dynamics are important 2) the tutor has facilitation of group dynamics as one of the tasks associated with the job, and 3) the (previously indicated) importance that students give to the role of the tutor in success of the approach.

Des Marchais and Vu
3 state that: A student's learning evaluation system has so great an effect on the way students dedicate their energy in learning medicine that any institution must pay great attention to developing an evaluation congruent with its curriculum philosophy, its educational goals, and its students' learning needs.(p.282) They illustrate the development of such a scheme that has been found to successfully meet these demands.
O'Hanlon and colleagues 21 stress an important aspect of the evaluation of PBL programs that is under-represented in the literature, and that is the implications of PBL modes of learning on students with a non-English background and from a cultural perspective where the nature of learning is likely to consist of different values.They report that an optional supplementary tutorial to offer support for these learners.Those who attended rated the supplementary tutorial highly (76.7% found it helpful) and they indicated higher responses to the overall favoring of PBL on the general survey than those who chose not to attend.Furthermore, those higher responses were significant statistically for questions relating to "guidance in group work" and "spread of work".This is a good indication of the value of scaffolding in this area.Clearly, more research should be directed at ESL and cultural issues, given the cultural mix of students and the prevalence of PBL modes of learning.

Conclusion
In changing from a subject-based discipline to an integrated PBL mode, it is often difficult to anticipate the need for scaffolding in non-discipline areas (such as group dynamics, metacognition, etc.).The assumption of some unspecified, automatic development of these processes may be at the heart of some results, which we have seen where PBL improves over time, both academically and in terms of student perception.It may be a measure of the difficulty associated with transition from subject-based learning approaches to PBL, in fear that less content may be covered if too much time is devoted to group processes, etc.It may need to be accepted that PBL involves a slower start-up in terms of the disciplinebased content that is covered.This may be due to the development of important "hidden" skills, which will ultimately facilitate deep approaches to learning.This is an important time in which to ensure that adequate scaffolding exists to support this type of development.That such generic skills are important is widely recognized, as may be seen in the following quote by Koschmann and colleagues: 6 In summary, educating competent physicians is difficult.The amount of material that must be learned is vast, and is conceptually complex.Patient problems are ill-structured; information about the problem must be uncovered, dynamically changing the problem as it emerges, and decisions must be made and action taken in the absence of complete information.Finally, medical knowledge and techniques for acquiring patient information are changing at a phenomenal rate, but little is done in traditional medical education to develop the cognitive disposition and competence necessary for the student to become an effective lifelong learner.(p.240) One of the difficulties in measuring learning outcomes is that PBL advocates frequently claim that the differences between it and conventional, subjectbased learning approaches occur principally at deep learning levels, and many conventional evaluation processes measure this inadequately. 1It is not always clear that assessment is matched to PBL modes of learning, which may be a hidden source of anxiety for many students, who may recognize the assessment criteria as "shallow", but not feel sufficiently acquainted with them (through lack of practice).
Morrison and Murray, 44 in discussing the results of a student survey of PBL course which provided evidence of high motivation but low relevance, suggest that at least some of basis for the low scores in perceived relevance could be directly related to exam anxiety, stating that By the time the students enter the final rotation of their medical course they are already preoccupied with passing their final examinations.As their course so far has tended to encourage superficial learning likely to aid in passing examinations, the problembased learning, which was felt to be time-consuming although promoting deep learning .... was judged to be less relevant.(p.145) Support structures need to consider the wider context in which students find themselves.The benefits of PBL are often constrained by implementation factors, such as consideration for external (non-PBL) courses and student perceptions of time requirements, or by gradual introduction of the PBL environment (such as gradated fidelity), or by bounding the degree to which ideals are attempted to be implemented.This may mean temporary departure from some of the principles in order to introduce them in a way that is meaningful to the student.For example, excessive adherence to the belief in student-centerdness could lead to a position where students react against the learning environment as a whole.Such biases are not necessarily inherent to the PBL philosophy, but rather more likely to be an implementational over-correction in adopting PBL for the first time.Some studies indicate that student reaction to PBL becomes more favorable in later incarnations of the course, and this may possibly be a result of incorporating pragmatism into perhaps an initially overly-idealized curriculum.
Drinan 45 suggests that PBL demands a level of maturity that may not be applicable to all undergraduate scenarios, although it is appropriate to postgraduate expectations.Others report similar feelings with different aspects of the PBL environment, such as in group dynamics. 35Clearly, PBL requires certain skills in order to be optimally effective.These skills should not be assumed.They require a process of development in order to empower the student with the ability to realize the ideals of PBL.It does not weaken the case for PBL to actively address the learning of such skills, but rather is likely to strengthen the success of the approach.
Related to this is the importance of not assuming that "peripheral" aspects such as logistics, politics, and staff concepts about learning are secondary.In many ways they become crucial to the success of a PBL program, due to the ability for change to result in upheaval, and the fact that they are capable of being limiting factors despite their non-pedagogical position in the scheme of things.
Schilling, Ginn, Mickelson and Roth 46 discuss a need to change the nature of library services (and librarian training) in response to the different emphasis suggested by PBL programs.This reaction to PBL is a strong suggestion of the support structure that PBL may generate, and indirectly indicates the need for a broad approach to scaffolding in PBL.
Other research confirms that at least some of the excess time that students associate with PBL may be directed at information gathering, and therefore suggests that it is the nature of a problem that can be fixed 44 "in a packed medical curriculum the advantages of ease of retrieval of information from textbooks and the compact presentation of facts possible in lectures are all too readily appreciated" (p.144).
Finally, it is suggested that PBL may operate as its own support mechanism, by offering it in early years in a more structured format than its ideals suggest, so that the desired skills may be accommodated and enable a more mature approach to senior courses offered in PBL style.