This book is designed to show what happens when a university takes seriously the idea of the scholarship of teaching and learning and sets out to promote, develop and reward it. The aim of the book is to advance intellectual discussion and debate about teaching and learning improvement by showcasing research and scholarship on teaching and learning practice that has been carried out within the University of Sydney. A key concern is to demonstrate how such work has contributed to the improvement of teaching and student learning through transforming the ways in which teaching and curricula are understood.

In preparing this volume, we have been concerned to demonstrate what happens when one institution takes the development of the scholarship of teaching and learning seriously. The book aims to provide evidence of the effectiveness of research on teaching and learning for the transformation of university teaching and learning within one university and to demonstrate its impact by making the outcomes of some of this work publicly available. Contributors are key researchers in teaching and learning across the University of Sydney. Invitations were sent to academics who had hitherto carried out substantial internationally published research on aspects of their teaching asking if they would like to contribute a chapter either individually or in collaboration with colleagues. Contributors were asked to indicate the issues, questions or problems that led them to investigate the issue being discussed and to locate that within a relevant research literature and theory. They were asked to describe the investigation and summarise the results. Finally they were asked to indicate how they had used the research findings in improving teaching and learning.

The scholarship of teaching and learning

The idea of the scholarship of teaching and learning arose in the work of Ernest Boyer and colleagues at the Carnegie Foundation for the Advancement of University Teaching with the publication in 1990 of the seminal work ‘Scholarship Reconsidered’. This book appeared at a time when there was considerable concern about how academic work was rewarded, and a desire to bring the concept of scholarship up to date and make it more relevant to the modern university and to developments in the professions. Boyer’s intention was to bring research, scholarship and teaching together through a redefinition of four forms of scholarship: the scholarships of discovery, application (later referred to as the scholarship of engagement (Boyer 1996)), the scholarships of integration and of teaching. For Boyer, the scholarship of teaching was characterised by knowledge of the subject being taught, carefully planned and continuously evaluated teaching related to the subject matter, encouragement of active, life-long learning which develops students as critical, creative thinkers, and the recognition that teachers are also learners. Hutchings and Shulman (1999) subsequently suggested that before ideas of the scholarship of teaching were developed, teaching did not automatically renew itself. It was possible to teach for many years without any development of that teaching. However, what is now known as the scholarship of teaching and learning demands a kind of ‘going meta’ (Hutchings & Shulman 1999 p. 13) where academics frame 2questions that they systematically investigate in relation to their teaching and their students’ learning.

Initial formulations of the scholarship of teaching were helpful in suggesting a language with which to frame ongoing improvements in teaching and learning. By emphasising the scholarly nature of the teaching and learning process, it provided a framework for higher education teachers committed to improving teaching and students’ learning to think about their teaching as a scholarly process. Since the publication of Scholarship Reconsidered a number of scholars have explored the possibilities contained in the idea so that there are now many examples of practice in the literature, and a number of theoretical models which extend ideas of what it may encompass. Most scholars now agree that the scholarship of teaching and learning includes ongoing ‘learning about teaching and the demonstration of teaching knowledge’ (Kreber & Cranton 2000, p. 477-8). Indeed, there is now general agreement that the purpose of the scholarship of teaching is to infuse teaching with scholarly qualities in order to enhance learning (Hutchings, Babb & Bjork, 2002; Hutchings & Shulman, 1999; Kreber, 2002; Trigwell & Shale, 2004). These scholarly qualities emphasise systematic evaluation and critical reflection on teaching and student learning supported by peer review.

Different models of the scholarship of teaching and learning have developed in different contexts and different countries. Some have focused on the development of teaching portfolios for promotion, recognition or reward. Others have focused on the course portfolio as a way of integrating curricula in a specific discipline across a national system. Other models emphasise the development of critical reflective practice, while others have focused on the development of pedagogical research. Trigwell, Martin, Benjamin & Prosser (2000, p. 156) say the aim of scholarly teaching is to ‘make transparent how we have made learning possible.’ In order that this can happen, they argue, ‘teachers must be informed of the theoretical perspectives and literature of teaching and learning in their discipline, and be able to collect and present rigorous evidence of effectiveness.’ It is this view of the scholarship of teaching and learning that lies at the heart of the work presented in this book.

In order to frame the book, this chapter discusses the institutional strategies that have been implemented to encourage and support the scholarship of teaching and learning at the University of Sydney. The chapter then looks more generally at the relationship between the scholarship of teaching and learning and improvements in students’ learning experiences. It concludes with a brief overview of the organisation of the book.

The development of the scholarship of teaching and learning at the University of Sydney

The University of Sydney is a large research-intensive institution with approximately 31,000 undergraduate and 14,000 postgraduate students. As the oldest university in Australia, the University aims to be a leader both in disciplinary research and scholarship and in teaching and learning. The university has taken a systematic and scholarly approach to the improvement of teaching and learning since the year 2000. This includes a range of approaches to the management and evaluation of teaching and student learning driven by an emphasis on understanding and improving students’ learning experiences. As far as the development of the scholarship of teaching and learning is concerned four initiatives are particularly relevant: a teaching quality improvement performance-based funding system, strategic university-wide projects, for 3example, on research-led teaching and the scholarship of teaching and graduate attributes, the availability of training in carrying out research on university teaching and learning at graduate certificate level and the possibility of being promoted or gaining an award on the basis of outstanding teaching.

These initiatives indicate a commitment to achieving and rewarding quality teaching in a research intensive environment. At a time when the Australian federal government is about to introduce research assessment through its Research Quality Framework (RQF), these initiatives can be viewed as an important counterpoint to a preoccupation with disciplinary research.

Performance-based funding for teaching

A major part of the University of Sydney performance-based funding system for teaching is a ‘Teaching Dividend’ comprising the allocation of six per cent of operating grant money to faculties in proportion to their relative teaching quality as measured by a series of teaching performance indicators (Ramsden 2001):

• Student Progress Rate (SPR)
• First to Second Year Retention
• SCEQ Good Teaching
• SCEQ Generic Skills
• SCEQ Overall Satisfaction
• CEQ Good Teaching
• CEQ Generic Skills
• CEQ Overall Satisfaction
• Full-Time Employment
• Full-Time Further Study

The SCEQ (Student Course Experience Questionnaire) and the CEQ (Course Experience Questionnaire) include series of questions designed to measure students’ experiences of a range of aspects of the teaching and learning environment. The CEQ is used nationally to measure students overall course experiences, so the CEQ scores used in each discipline are benchmarked with the average score for the same discipline in other universities in Australia in the Group of Eight (research-intensive) universities. The teaching quality funding system also provides resources to enable faculties to address areas for improvement bid for on a competitive basis. The university’s improvement agenda has also included rewarding departments for a defined and weighted set of scholarly accomplishments in relation to teaching and learning via what is known as the Scholarship Index.

The purpose of the Scholarship Index is to provide financial rewards to departments whose staff members contribute to teaching quality through the scholarship of university teaching and learning. These are measured on a defined and weighted set of criteria. The Scholarship Index is sourced from 0.5% of operating grant money and a contribution of 0.5% of the previous year’s international student fee income. Claims are made annually and evidence for each claim is required. The criteria and their weightings are presented in Table 1.1. 4

The overall levels of achievement of faculties in the Scholarship Index are presented in Figure 1.1 This shows the variation in the extent to which faculties have actively engaged with it. Some faculties have taken it extremely seriously demonstrated by substantial achievements.

5The variation shown in Figure 1.1 indicates two major trends. First, it would appear that across faculties over the three years there have been substantial gains in points allocated. This indicates increasing levels of scholarly work being undertaken across the university as a whole. Second, the results show considerable differences between faculties in the levels of scholarly work undertaken with some faculties showing quite marked gains over the three years.

Strategic projects

The development of the scholarship of teaching and learning has been part of a university-wide project that was established in 2000 to increasingly employ undergraduate teaching and learning strategies which enhance the links between research and teaching and utilise scholarly inquiry as an organising principle in departmental organisation, and curriculum development; and to encourage and reward the scholarship of teaching and learning.

A large forum has been held every two years since 2000, each attended by approximately 200 academics and featuring many presentations of research on teaching by University of Sydney staff as well as internationally renowned keynote speakers. These events have been important in raising awareness and sharing good practice. A number of similar events have subsequently been held within faculties. In 2005, the University hosted the Annual International Higher Education Research and Development Society of Australasia (HERDSA) conference with 460 delegates. 120 University of Sydney staff presented at this event. Further strategies to encourage the scholarship of teaching and learning have included the establishment of a strategic working group with representatives from each faculty nominated by deans. The working group has established a set of performance indicators for research-led teaching and the scholarship of teaching and carried out an audit. It has established clear guidelines for dealing with ethical procedures when carrying out research on teaching and has been responsible for drafting policy and for a number of initiatives designed to share good practice. Other project strategies have included: the development of a web site with resources to encourage and support academics in developing the scholarship of teaching and learning, revision of the criteria for the Vice-Chancellor’s award schemes for outstanding teaching to strengthen the emphasis on demonstrating scholarship in teaching, and carrying out investigative work regarding best practice in research-led teaching in research-intensive institutions with which the University of Sydney has benchmarking relationships.

In 2001 the University of Sydney’s Academic Board, its main academic decision-making body, initiated a series of reviews in which questions were asked in each faculty about the development of research-led teaching and the scholarship of teaching. Each faculty was required to address the recommendations that were made. In addition, a Graduate Certificate in Higher Education unit of study focused on the Scholarship of University Teaching and Learning was established to teach academics the skills of scholarly inquiry related to teaching and learning (see Chapter 20). To date, over 250 academics have completed the graduate certificate.

Faculties have, in turn, adopted a series of strategies to develop the scholarship of teaching and learning. These vary from faculty to faculty but include: making changes to faculty policies; seminars and discussions of research on teaching and learning; research on teaching and learning websites to encourage development; research on teaching competitive grant schemes; making the University’s graduate certificate in 6higher education compulsory for all new staff; using scholarship index money to fund teaching awards; rewarding achievements in scholarship of teaching in teaching awards; attendance at higher education teaching and learning or research on teaching conferences. Evidence suggests that faculties that have put in place explicit strategies to increase performance on the Scholarship Index have indeed been successful. The extent to which these achievements have resulted in enhanced student learning experiences is examined below.

As a result of all of these initiatives, in the light of discussions at other universities in the UK and Australia, and taking account of the international research literature, the Research-Enhanced Learning and Teaching Working Group drafted a policy which has now been accepted by Academic Board. The policy includes the following:

Coexistent with these developments has been a related project to specify the attributes that the university considers its graduates develop. As a consequence of this project a set of generic attributes of graduate of the University of Sydney which embody the university’s scholarly values as a research intensive university has been developed. Resources to support staff in ensuring students develop the graduate attributes, a strategic working group to support the project and the ongoing dissemination and implementation of university graduate attributes policy within faculties as well as a benchmarking process has been developed. The graduate attributes strategic project has fed into curriculum reviews in many faculties. It underscores the university’s commitment to scholarly inquiry and evidence-based practice in relation both to student learning and academic work.

Does engaging in the scholarship of teaching result in better teaching?

In preparing this book we have been mindful of the need to link research on teaching and learning to improvements in students’ learning. In 2000 Healey reported that there was very little research evidence that engaging in the scholarship of teaching and learning enhanced learning (Healey 2000). There were many anecdotal examples of 7teachers improving aspects of their practice as a consequence of engaging in inquiries into their students’ learning. There was anecdotal evidence at the University of Sydney that teachers initiated into the practice of scholarship of teaching and learning were becoming leaders in teaching developments in their faculties. A number of these individuals are represented in this volume. There is some research evidence that engaging in training in university teaching leads to increased student satisfaction and an increase in the use of student-focused approaches to teaching (Gibbs & Coffey, 2004; Lueddeke, 2003). However, an Australian study of tertiary teaching award programs (Dearn, Fraser & Ryan, 2002) found that such courses were most likely to be focused on the development of teaching skills or the development of a specific teaching practice, for example, flexible and online teaching, assessment of student learning, postgraduate supervision and internationalisation, not on developing scholarly approaches to teaching.

There is evidence that when university teachers say they reflect on their teaching they do so at an instrumental or technical level focused on improving actions in the classroom, rather than in understanding the reasons why particular methods are chosen, why students respond as they do, or reflecting in ways that question their basic teaching assumptions (Kreber, 2004, McAlpine & Weston, 2002; Trigwell et al., 2000). In Chapter 20 we shall see that a key contribution of engaging in the scholarship of teaching and learning is its capacity to provide a means whereby teachers are enabled to develop a reflexive critique of their teaching enabling them to questions the values and assumptions that drive them to teach the ways they do.

However, given the efforts that have been made to develop the scholarship of teaching at the University of Sydney, it is pertinent to ask what its impact is on the experiences of students. In order to address this issue, my colleague Paul Ginns and I investigated whether faculty differences in performance on the Scholarship Index were associated with faculty differences in changes in undergraduate responses on the Student Course Experience Questionnaire (SCEQ) scales. SCEQ data has been collected from undergraduates since 1999, while Faculties have lodged Scholarship Index claims each year since 2002 (data for 2005 was lodged in the middle of 2006 and audited early in 2007). Our analysis therefore aimed to investigate the possible link between these two institutional initiatives by investigating the association between a faculty’s three year performance (2002-2004) on the Scholarship Index, and the change in the faculty’s SCEQ score between 2001 and 2005.

We calculated 2 results for each faculty. The first was the sum across 2002 to 2004 of the Scholarship Index performances for each faculty, weighted according to the number of full-time equivalent teaching staff in that faculty. The second was the change in SCEQ scores between the 2001 survey of undergraduates, and the 2005 survey. We investigated the association between these 2 variables using regression analysis, specifying the Scholarship Index sum variable as the independent variable, and the change in SCEQ scores as the dependent variable (Brew & Ginns, 2006). What we found was that this relationship was statistically significant for three of the SCEQ scales – Good Teaching (p=.036), Appropriate Assessment (p=.021), and Generic Skills (p=.020) suggesting that performance on the Scholarship Index is related to students’ perceptions of their assessment, how and whether their generic skills have been developed and their perceptions of the quality of the teaching (Brew & Ginns, 2006). In particular, we found that differences in faculty performances over three years (2003-2004) 8on the Scholarship Index were reliably associated with changes in student perceptions between 2001 and 2005.

These results provide support for the introduction of the Scholarship Index as a means for improving student learning experiences. They provide tangible support for Hutchings and Shulman’s (1999) suggestion that the scholarship of teaching and learning is how the profession of teaching advances. However, it is pertinent to ask why developing the scholarship of teaching has the effects that are seen here on measures of students’ experiences. Developing the scholarship of teaching ultimately has an effect on the ways in which students’ experience their courses. Curriculum development within such a context is no longer based on ad hoc assumptions or reactions to teaching methods experienced as a student. Instead, decision-making comes to be based on evidence of what is effective as demonstrated in the scholarly research literature and as evidenced in the specific context. As can be seen in this volume, engaging in the scholarship of teaching and learning means that teachers become capable of articulating their theories of teaching and of understanding the epistemological framework that drives their investigations. They become aware of the role that educational research and theory plays in their discipline. In short, they develop a reflexive critique of practice (see Chapter 20). There is also evidence to suggest that the scholarship of teaching and learning, by engaging teachers in the process of inquiring into their teaching, leads teachers to articulate a pedagogical framework or philosophy of teaching (see for example Brew & Peseta, 2004; 2001) in which specific approaches to teaching are viewed as instances of a broader theoretical approach.

Further research is needed to examine more systematically what faculties that are performing well are doing. We also need more information about the contributions that the different criteria on the Scholarship Index make in explaining performance differences. For example, highly successful faculties may be marked by the emphasis they place on encouraging staff to obtain teaching qualifications or write textbooks which are weighted highly on the scale. Another avenue of institutional research might be to continue to refine the composition of the Scholarship Index to increase its capacity to effect change. Examining the variation between faculties in how Scholarship Index funds are dispersed and the purposes to which these funds are put is also a subject for future research.

Organisation of the book

The chapters in this book represent a wide spectrum of disciplinary areas of the University of Sydney and address a considerable variety of questions in regard to teaching and learning using a considerable range of methodologies and theoretical approaches. There are five broad areas around which we have chosen to organise the book. We begin in Part 1 by presenting research which has been carried out in order to understand better the experiences and understandings of students. The focus of attention in these chapters is on addressing challenges presented within particular curricula: for example, concepts that students typically find difficult in a course as in the chapter by Erica Sainsbury and Richard Walker, the challenge of learning within service courses as in the chapter by Laura Minasian-Batmanian and Jennifer Lingard, the challenge of students’ attitudes to material presented as in the chapters by Ann Alias and Kathryn Marsh. Each of these chapters in their different ways focuses on inducting students into ways of knowing and thinking in specific disciplines. This theme is taken up in the chapter by Peter Goodyear and Robert Ellis whose specific focus is on 9understanding the ways in which online collaborative learning activities are and are not used to develop understanding of the way knowledge operates in their particular disciplinary area.

Part II presents work which has focused on developing a greater understanding of student assessment. Fiona White, Hilary Lloyd and Jerry Goldfried examine students’ attitudes towards collaborative group work and group assessment, while Ian Sefton and Manjula Sharma compare the findings of a phenomenographic study of students’ conceptions with students’ examination scores. This raises some interesting questions about the relationship between examination marks and students’ understanding

A number of studies that have been carried out in a wide range of contexts have sought to understand and respond to students’ preparedness for university study. These are the focus of Part III. The contexts for which students require preparation are varied. So in Chapter 9 Patricia Lyon discusses research which led to medical students being better prepared for learning in the operating theatre, while Susan Page, Sally Farrington and Kristie Daniel DiGregorio in Chapter 10 discuss work which has focused on Indigenous students’ preparedness for university study. Writing and numeracy are integral to university study, and in the chapters by Charlotte Taylor and Helen Drury and by Sandra Britton and colleagues, students’ writing and mathematical skills are the focus. In Chapter 13, Nerida Jarkey discusses a program of research and development designed to prepare first year Arts students for university study. This chapter focuses on an iterative process of research informing practice and vice versa. As such it forms a bridge to Part IV which contains a number of further chapters where the authors have engaged in ongoing cycles of research and curriculum change. Mark Freeman, Henriikka Clarkeburn and Lesley Treleaven discuss the ways that research on academic honesty has been successively integrated into strategies at the faculty level. They show how a more sophisticated understanding of the problems of plagiarism and cheating resulted from this. The chapters by Helen Wozniak and colleagues and Rafael Calvo and colleagues each focus on interactive processes of research and development in relation to eLearning but from very different perspectives; one on understanding how learners engage in online discussions, the other, understanding how to develop software that will engage students in deep approaches to learning. Anna Rubbo provides an insight into a global research and educational intervention in the teaching of architects, while the chapters by Barbara Adamson and colleagues and by Tania Gerzina recount a range of research on teaching projects that have been carried out over a long period of time, leading to successive changes in teaching and learning in the Faculties of Health Sciences and Dentistry respectively.

Finally, in Part V we reflect in different ways on the challenges and the successes of engaging in the scholarship of teaching and learning. In Chapter 20, Tai Peseta and academic development colleagues from the Institute for Teaching and Learning reflect on the challenges for disciplinary academics in engaging in the scholarship of teaching and learning and on some of the dilemmas associated with performing a role as change agents in an institution where the scholarship of teaching and learning is strongly encouraged. Rosanne Taylor then looks from a faculty perspective to highlight and celebrate the achievements of a faculty that has fully embraced not only the scholarship of teaching and learning, but the scholarship of academic practice more broadly. Finally, in conclusion, Judyth Sachs offers some reflections and implications for teaching and learning in the future. 10

Conclusion

Within Australia, as this book is in production, the introduction of a Research Quality Framework which measures impact and quality of disciplinary research is on the near horizon. Such a framework threatens to supplant efforts to improve teaching through the scholarship of teaching and learning. Through the initiatives discussed in this book, we believe that the university has made substantial progress in developing understanding of the nature of the scholarship of teaching at different levels of the University and that scholarly work in relation to teaching has demonstrably been used to enhance practice. The strategic initiatives discussed in this chapter have provided a context for scholarly work in relation to teaching and learning, but the research and developments detailed in this book could not have been achieved without the hard work and determination of individuals and groups of academics who with dedication and commitment to students’ learning have shown creativity and courage in advancing research on teaching and learning in the university.

More generally, as seen in Figure 1.1, the scholarship of teaching and learning is being energetically pursued across the university and its effects are clear and widespread. Progress has been made in moving thinking away from a teacher focused view to focus more on the student experiences. There is still much to learn about what it is that a research-intensive university can offer students that is unavailable in other higher education contexts. There is a long way to go in transforming a university, but it is already evident that the initiatives such as are detailed in this book are taking us beyond perfunctory notions of quality assurance towards sustained quality enhancement.

The process of transforming the teaching and learning processes and practices within a large and diverse institution is a long term project. It is an ongoing process that cannot ever be complete. We hope this book will provide inspiration to other institutions thinking about utilising the scholarship of teaching and learning to effect curriculum transformation and that it will encourage academics in other universities who are thinking about researching their teaching to take up the challenges it offers.

Acknowledgements

I am grateful for the contribution of my colleague Dr Paul Ginns, Institute for Teaching and Learning, to the statistical analysis discussed in this chapter. Some of the ideas in this chapter were first considered in my book Research and Teaching: Beyond the Divide published by PalgraveMacmillan in June 2006.