This chapter describes and discusses the approach taken by a group of researchers in the Faculty of Health Sciences to improving the learning outcomes of health science graduates. In so doing, we highlight a process and a model that could be used by other groups of researchers wishing to develop and strengthen graduate skills and attributes during university education. A key feature of the model is participation by key stakeholders: employers of new graduates, graduates and academics. Whilst consulting relevant stakeholders has been the cornerstone of improvements to teaching and learning (Foskett, 2005) our approach with this consultation process is unique. We discuss why we believe our approach is unique and the developments in research and curricula spanning over a decade (1993-2004). The original piece of work consisted of a follow-up of graduates to determine what skills and attributes are necessary in order to be well-equipped for working in a rapidly changing health care environment. The subsequent research and developments in teaching and learning were guided by this research and consist of in depth investigations into organisational and time management skills necessary for effective functioning in the workplace. Whilst this research has been published as a sequence of studies, no previous publication has discussed the complete series of studies and the many and varied applications that have resulted. Throughout the chapter, the reader will be referred to more detailed presentations of our research and subsequent applications of the findings.

Meeting workplace requirements: Challenges in higher education

In 1993 a group of researchers in the Faculty of Health Sciences led by Barbara Adamson (from the Discipline of Psychology), Lynne Harris (from the Discipline of Psychology), Adrienne Hunt (from the Discipline of Physiotherapy) and Robert Heard (Discipline of Psychology) was charged with the responsibility of assessing the adequacy of undergraduate preparation in health information management, occupational therapy, orthoptics, physiotherapy and speech pathology in meeting the needs of a changing health care environment. It should be noted that all allied health students complete practical components known as clinical placements or fieldwork during their undergraduate or graduate degrees as well as academic components. Clinical education or fieldwork may involve spending short (a few days) or relatively long periods (6-10 weeks) working in health or education settings with clients or patients. Typically these placements are supervised by an experienced practitioner (‘employer’) in the student’s discipline. This practical aspect is similar to the requirements of other health science programs (e.g., dentistry, medicine, nursing, pharmacy). Additionally, graduates are expected to be ‘workplace ready’ at the point of graduation. Hence health science curricula have traditionally been influenced by their respective professional bodies and workplace demands.

At the time of designing our research, very few studies had been conducted in allied health to assess graduates’ perceptions of their university education. This situation contrasted with the many studies conducted in the disciplines of medicine and nursing 198which focused on the impact of structural course changes to programs (Bottorff, 1986; Geffen, Saunders & Sefton, 1994) and the relationship with graduate preparation for the workforce. Furthermore, available research tended to address single disciplines or particular workplace settings (Huebler, 1994).

Evaluating quality in higher education: Follow-up of graduates

While there may be dispute over how quality in higher education should be defined, it is difficult to argue against the inclusion of a process of defining quality based on current and future workplace requirements. Hence a workplace driven approach to curriculum evaluation was selected. This approach is top-down since one group of stakeholders, that is, employers are given the opportunity to determine the direction of the curriculum development process. This contrasts to a bottom-up approach (Oye-Adeniran, Adewole, Iwere & Mahmoud, 2004) where academic teaching staff (at the ground level or ‘bottom’) in the first instance are given the opportunity to frame the direction of the process.

The three sections below describe the original piece of research consisting of a follow-up of graduates and the research process involved in our workplace driven methodology including consultation with key stakeholders. A model of the process is depicted and summarised in Figure 17.1. In addition, Figure 17.1 displays the subsequent developments in research stemming from the original study.

Survey of employers

In order to establish the attributes and skills that are necessary and desirable to equip graduates for the challenges of the workplace, a survey of employers (experienced practitioners who work closely with new health science graduates in the 5 disciplines 199mentioned above) was conducted. Employers (n=67) endorsed the need for the skills and attributes specified by the Academic Board (1993) of the University of Sydney, which included knowledge skills, thinking skills, personal skills, personal attributes (e.g., strive for tolerance and integrity, ethical practice) and practical skills. In addition, they identified a suite of skills that they perceived necessary in order for new graduates to function effectively in the current and future health care environment. These included: knowledge of the health care system, organisational management skills (e.g., budgeting, time management), and client management (e.g., knowledge of when to discontinue client treatment).

It is important to note the design characteristics of the survey we used with employers. Whilst closed ended questions asked them to make ratings of the importance of skills and attributes specified by the University of Sydney (a finite set) it was the open ended questions that allowed employers the flexibility to nominate other skills and attributes they perceived necessary in a rapidly changing health care environment. Moreover, the commonality of the suite of skills recommended as desirable for new graduates by employers across five distinct health science disciplines was noteworthy. This was a significant finding that hitherto had not been available in the literature since as discussed above, previous research had conducted single discipline studies (e.g., nursing or medicine). It is noteworthy that a current literature search of studies in the area indicates that this trend is still evident (Spratt & Walls, 2003) with few exceptions (Humphreys & Davenport, 2005).

Survey of graduates

The commonality of the employers’ responses across the five professional groups allowed us to devise a questionnaire for use by graduates from these five discipline groups. That is, the graduate survey was devised to reflect the skills and attributes perceived by employers to be necessary for new graduates to meet the demands of the workplace. The survey consisted of 52 items that were written in the form of questions asking graduates whether their undergraduate course had prepared or equipped them with the skills necessary to perform the tasks specified by employers. A seven-point rating scale was used to obtain this information. An example of a survey item is ‘Did the course equip you to produce written records and reports?’

A total of 527 health science graduates completed the questionnaire. Using a statistical technique called factor analysis eleven workplace dimensions (factors) were identified. We named these workplace dimensions by identifying the type of items that loaded on each factor. The eleven workplace dimensions were: communication with clients, communication with health professionals and the general public, knowledge of the health industry, confidence in clinical role, realistic expectation of workplace role, pursuit and application of knowledge, workplace management, applying an evaluative approach, essential tasks and ethical practice. By examining the means of each workplace dimension we were able to determine the strengths and weaknesses of current undergraduate courses in terms of their adequacy in meeting the demands of the workplace (Adamson, Harris, Heard & Hunt, 1996; Adamson, Harris & Hunt, 1997). 200

Survey of academics and employers

In order to assess the validity of the findings related to graduates, a very similar version of the graduate questionnaire was mailed to employers and academics who taught in the designated health professional disciplines. The results pertaining to the perceptions of these two latter groups of stakeholders (n=84 and n=41 respectively) were consistent with each other and with the perceptions of health science graduates, thereby substantiating the validity of the graduates’ findings.

A rigorous methodological approach to evaluating quality in higher education

The concurrence of perceptions between these three groups, that is employers, academics and graduates allowed us to feel confident regarding curriculum development to better match the skills and attributes that graduates acquire as a result of their university education, with the skills and attributes required of them in the workplace. The results of the survey of graduates, employers and academics indicated that on several workplace dimensions, undergraduate courses were not equipping graduates adequately. Specifically, graduates perceived themselves to be ill-equipped on dimensions concerned with workplace management, knowledge of the health industry and coping in the workplace. On other dimensions such as essential tasks (e.g., writing written records and reports, effective communication with clients), confidence in clinical role and ethical practice, graduates perceived themselves to be better equipped. These results and the rigorous statistical procedures adhered to in the design and data analysis of our research are discussed elsewhere (Adamson, Harris, Heard & Hunt, 1996; Adamson, Harris & Hunt, 1997). Subsequent to the research the challenge was now for educators to consider how best to accommodate the requisite attributes and skills identified by employers.

As discussed above, we believe our research has distinguishing and unique features not always present in the research conducted to assess the adequacy of undergraduate education (Harris, Adamson & Hunt, 1998). However, it is important to note two of these features since we believe it is these features that have contributed to a sustainable series of applications in researching and developing graduate attributes and skills.

First, the inclusion of open ended questions in the initial survey to employers proved valuable since without these questions the suite of skills and attributes they identified may not have been uncovered. Whilst this methodological approach was novel at the time our study was conducted current literature searches indicate that this is still the case (Foskett, 2005). Our approach contrasts to studies in the area where typically the focus is on employers and other key stakeholders commenting on the strengths and weaknesses of existing course content and structure. Hence the qualitative aspect of our research design with employers made possible the exposure to skills and attributes perceived to be necessary for effective functioning in the current and future health care environment. It is only at the next stage of the research process when graduates, academics and employers were asked to rate whether the current undergraduate course covered the skills and attributes specified by the initial group of employers that curricula issues became relevant.

Second, the employers who responded to our survey were from a range of workplace settings (e.g., hospitals, community settings) and five distinct allied health science disciplines. Despite the differences in demographic characteristics the commonality of the suite of skills and attributes that they identified was striking. This gave us 201confidence in the robustness of our data. Additionally, the graduates who responded to our survey were themselves working in diverse settings (e.g., public/private hospitals, community and educational settings). A statistical analysis indicated that their perceptions of the adequacy of their courses did not differ markedly across workplace settings (Adamson, Harris, Heard & Hunt, 1996).

The impact of our follow-up of graduates research on teaching and learning programs

The findings of this large scale study had a significant impact both within the Faculty of Health Sciences, the broader University of Sydney community, and beyond. In the first instance, the commonality of the findings across the five disciplines had a major influence on curriculum planning in the faculty, in that course content specifically designed for a single discipline rather than groups of disciplines became hard to sustain particularly in an era of shrinking budgets in the tertiary sector. This resulted in curriculum development involving interdisciplinary teams of teachers, teaching to cross-disciplinary groups of students (Harris & Viney, 2003). The findings of our study were also incorporated in undergraduate course reviews and consultations with relevant professional bodies and in graduate entry programs being developed at the time (e.g., Masters of Occupational Therapy) particularly in terms of addressing the skills and attributes specified by employers. The details of these developments have been published elsewhere (Adamson, Hunt, Harris & Reid, 1999; Adamson, Hunt, Harris & Hummel, 1998; Harris, Adamson, Reed & Hunt, 1998; Hunt, Adamson & Harris, 1998; Hunt, Adamson, Higgs & Harris, 1998).

In addition, the questionnaires developed in our study have been used and adapted by other academics and researchers across different contexts. For example, the Faculty of Dentistry at the University of Sydney used our original employer questionnaire to determine what skills are necessary for dental practice in a changing global market (Gonsalkorale, Dean & Sefton, 2003). The graduate questionnaire has been used to collect data for course reviews and the accreditation process for the Singapore Nursing Board through the Singapore Institute of Management (SIM) (Personal communication with the Director, Dr K O’Loughlin in the Faculty of Health Sciences). After publication of our research there were numerous requests from colleagues both within Australia and overseas to use the questionnaires developed from our research. Hence the impact of our large scale study whilst difficult to quantify is clearly far-reaching.

Targeting specific areas for change

Workplace management

Workplace management and knowledge of the health industry were two of the main dimensions that key stakeholders perceived to be covered inadequately in undergraduate curricula. Whilst it may be debatable as to whether these two dimensions should be covered in undergraduate degree programs in allied health, it was considered pertinent to further explore how one of these dimensions could be enhanced and strengthened.

In order to obtain an in-depth understanding of what underlying knowledge and skills to foster in relation to workplace management in undergraduate and postgraduate students, as with our previous research, we adhered to an evidence-based approach and surveyed experienced practitioners in allied health fields. Five hundred and three 202experienced practitioners with managerial duties from medical radiation sciences, occupational therapy, physiotherapy and speech pathology participated in our study. We used a survey with both open ended and closed ended questions to elucidate what managerial skills and tasks they perceived as necessary both for managers in the health care environment and for new graduates.

It is beyond the scope of this chapter to discuss the findings of this research in detail and the implications for fostering workplace management skills in both undergraduate and graduate courses. Nonetheless, one specific aim with this research was to build these skills and knowledge base into an existing postgraduate unit of study called Organisational Management. This unit is available to graduate students across the faculty. Eight managerial dimensions were uncovered using factor analysis (staff relations management, management of future planning, prioritising work tasks, quality assurance, implementation and change, career path management, legislative knowledge and running a department) and these were used to inform curricula development with this postgraduate unit of study (Organisational Management). Furthermore, to ground this unit in a health care industry managerial context, 10 experienced practitioners who agreed to be contacted were asked to write a case scenario from their everyday work experiences. These case scenarios represented the underlying managerial dimensions identified above and required critical thinking and problem solving skills necessary for effective functioning in management. The students’ responses to the case studies formed the assessment for the unit of study.

Although our intention with this study was to further our understanding of the skills and attributes underlying workplace management as with the follow-up of graduate research this research also had far reaching implications and applications (Adamson, Ateyo & Cant 2000; Adamson, Cant & Hummel, 2001; Adamson, Lincoln & Cant 2000; Atyeo, Adamson & Cant, 2001; Lincoln, Adamson & Cant, 2001). Apart from informing curricula development there were other positive outcomes. For example, Lincoln, Adamson and Cant (2001) conducted a separate analysis of the responses from speech pathology participants. This information was used by the Speech Pathology Association of Australia during a review of their Competency Based Occupational Standards for Speech Pathologists (CBOS, 2001) to change expected competencies in the area of management. CBOS defines the expected competencies of new graduates nationally and is an integral tool of the accreditation process for university programs. Hence the results of this piece of research were incorporated into Australia-wide competency based standards in speech pathology.

Time and organisational management skills

Another relevant finding from the research into managerial skills was the importance placed by experienced practitioners on new graduates possessing sound time management skills upon entering the workforce (Adamson, Ateyo & Cant, 2000). This finding echoed the importance attached to this skill in the follow-up of graduates’ research. Twenty-five percent of graduates felt inadequately trained in time management skills upon entry to the workplace (Adamson, Harris, Heard & Hunt, 1996). Consistent with these findings is the importance that students themselves place on time management skills (Humphreys & Davenport, 2005).

Poor time management has been reported to result in stress for both health science students and graduates (DiGiacomo & Adamson, 2001) and has been identified as contributing to less than optimal outcomes in therapy (Adamson, Hand, Heard & 203Nordholm, 1999). While the link between time management and academic performance appears to be more tenuous (Mace & Tira, 1999), the inconsistency in findings may be due to inadequate measurements of time management abilities and skills. Notwithstanding, the role of time management has been identified as one particularly significant point of weakness in students who are academically struggling (Proctor, Prevatt, Adams, Reaser & Petscher, 2006).

In order to further develop graduate attributes, specifically in relation to time organisational management skills we embarked on three research studies. Two of these studies were carried out in an academic environment with undergraduate students and the third in a clinical setting.

Time management studies conducted in an academic environment. The first study consisted of 154 final year undergraduate students from the disciplines of medical radiation sciences, speech pathology and physiotherapy (Covic, Adamson, Lincoln & Kench, 2003). The second study was conducted with 478 first year students and consisted of a brief time management intervention program with a five week follow-up (Adamson, Covic & Lincoln, 2004). In both of these studies we utilised a time management scale specifically developed for the Australian student population. The Australian Time Organisation and Management Scale (ATOMS) was developed by colleagues in the School of Psychology, the University of Sydney (Roberts, Krause & Suk-Lee, 1999) and it consists of 62 items that measure six dimensions of time-management behaviours and attitudes, namely: sense of purpose, meeting deadlines, mechanics of time management, effective organisation, propensity to plan and coping with temporal flow.

In our study of final year students we found that students had better skills in some dimensions of time and organisational management skills, such as a strong sense of purpose, a high level of focus and goal-setting, but not so in the area that specifically reflects time management, that is, the mechanics of time management. Interestingly, the mechanics of time management is quite amenable to improvement as it requires simple behavioural strategies such as the use of diaries and to-do lists. The majority of students we identified as at risk were deficient on only one dimension. These findings motivated us to develop an intervention program to target specific components of the time and organisation management skills construct.

Given that we identified a need to improve time management skills in final year undergraduate students, our intervention study targeted first year students in order to enhance their time and organisational management skills early in their academic careers. Initially, in week 3 of an academic semester, we surveyed 478 first year students’ time management skills using the ATOMS scale as in Covic et al’s study (2003). Then, in week 7 we provided students with their scores on each of the 6 dimensions and an intervention manual which reflected the six time and organisation management skills in Roberts, Krause and Suk-Lee’s (1999) scale. The students were instructed to carry out the prescribed exercises and to focus on areas of identified weakness. Students then had 5 weeks to work through the intervention manual and were then surveyed again in week 12.

The time and organisational management skills of the first year students were similar to the final year students surveyed in our previous study. However, while the majority of students found the intervention manual useful (63%) only a small minority of students completed all the exercises (3%) or focused on their weakest areas (29%). Subsequently, 59% did not feel that their time management skills had improved. Our 204findings suggest that a time management intervention with students needs to be considered in relation to the following issues: time management skills may be influenced by personality traits and therefore resistant to change; intervention delivery needs to be framed within the principles of adult learning; compliance with intervention requires attention; and simple intervention procedures may not have an impact on time and organisational management skills. It may be that a gradual but progressive inclusion of time and organisational management skills within academic programs could be a more effective intervention approach (Adamson, Covic & Lincoln, 2004).

Time management studies conducted in a clinical placement environment. Clinical placements are perceived by educators and students as ‘high stakes’ learning activities. Patient or clients need to be provided with high quality services that are delivered efficiently, professionally and ethically by students even though they are learning and inexperienced. Students perceive placements as high stakes because they occur ‘in view’ of their future colleagues and potential employers. Clinical placements are the most obvious and salient place that students learn about how to function in their future workplaces. Hence we moved our research on to focus on student performance on clinical placements.

Our previous work identified that time management skills was an issue for new graduates in the workplace, so we hypothesised that this may also be the case for students when on clinical placement. We also thought that clinical placements were perhaps a key learning experience that could be used to improve time management abilities before graduation. In order to explore this further Lincoln, Adamson and Covic (2004) studied the time management abilities of speech pathology students on clinical placements. This study found that students reported improved ability in the area of the mechanics of time management e.g., using a diary, making lists, timetables when placed full-time in health care settings compared to their peers who continued to attend university classes and simultaneously have clinical experiences in an on-campus clinic. Students also reported improvement on this dimension over a one year period. Interestingly, students’ abilities on other dimensions of time management such as sense of purpose, meeting deadlines, effective organisation, propensity to plan and coping with temporal flow did not change (Roberts, Krause & Suk-Lee, 1999). Consistent with other research findings, these time management factors may relate more closely to personality and hence be more resistant to change.

As with our previous work we sought to validate these findings with the other stakeholders in clinical placements, the clinical supervisors. In our study we asked clinical supervisors to also rate students’ time management abilities. Clinical supervisors agreed closely with students in their ratings of time management abilities on three out of the six time management dimensions. Clinical supervisors perceived students to be better at meeting deadlines than the students perceived themselves to be and poorer at propensity to plan and coping with temporal flow than students. In general clinical supervisors rated the speech pathology students as having good time management skills.

As a direct result of this work, objectives and learning activities were modified or included in the speech pathology curriculum. A developmental approach to the acquisition of time management skills in the clinical setting was developed. Beginning students are now expected to manage time well in clinical sessions with clients, intermediate students are expected to manage their time and responsibilities across a week and advanced students are expected to manage their clinical responsibilities 205across a semester or a block clinical placement. Professional development units of study in the academic stream in the final year also address management of services and caseloads, prioritorisation and health care service delivery models. Clinical supervisors receive training in how to facilitate the development of time management skills in students and also the importance of modelling good time management.

This work has also been influential in our work with students who exhibit chronic difficulties with time management, particularly in clinical settings. The knowledge that some aspects of time management can readily be taught and changed, e.g., the mechanics of time management and others are more resistant to change e.g., propensity to plan, has helped in understanding the individual strengths and weaknesses of students. Identifying the students’ strengths and weaknesses allows us to tailor learning support and intervention more effectively than previously. It remains now for us to evaluate whether these changes have had a positive effect on our students’ time management abilities in clinical placements and eventually as new graduates in the workforce.

Reflections on the process

The research and teaching and learning developments described above resulted from cross disciplinary collaboration. This collaboration has continued and broadened to include other workplace issues such as retention of the allied health workforce. The collaboration has also fostered and mentored academics in research and teaching and learning. Table 17.1 summarises the outcomes from the research process for organisations, the university and individual staff.

Additionally, the inclusion of stakeholders, particularly employers and clinical educators has improved reciprocal relations between stakeholders. Stakeholders are aware that faculty programs are striving to match graduate knowledge, skills and attributes to a rapidly changing workplace. A productive dialogue continues regularly between groups. Students have also benefited by having their curricula more closely aligned with the needs of the current health care system. Importantly, professional associations have also noted the research findings. As discussed above The Speech Pathology Association of Australia responded to our work on management competencies. This research has also opened the door to the successful submission of collaborative grants with professional associations. The collaborative grants to date 206have focused on related issues such as retention of health professionals in the workplace and their professions.

The current body of knowledge demonstrates that research into teaching and learning in health sciences can be evaluated in the traditional metrics used to measure academic research outputs and impacts. This work was supported by several competitive grants and resulted in international peer reviewed publications and conference papers. The work also established track records for the individual researchers in this field which subsequently assisted them to win external competitive grants and to attract doctoral students. The development and application of a rigorous research method also in turn developed the research skills of team members.

The dissemination of the results showcase a scholarly, evidence-based approach to curriculum development and have enhanced the faculty’s reputation as a leading education provider in the health sciences. Finally, the process highlighted the need for regular and sustained programs of research that continually evaluate the match between graduate skills, knowledge and attributes and workplace requirements. The implementation of an iterative approach will ensure that health sciences courses remain evidence-based and relevant into the future.

Where to from here?

The follow-up of graduates study commenced in 1993. It is now time to repeat the process because health care services, workplaces and allied health curricula have changed. Some of the significant changes in health care services relate to a focus on prevention in health, an increased use of technology, and implementation of evidence based practice. Changing directions in delivery of health care services have been accompanied by managing an increased need for services, larger case loads, increased legal, security and safety requirements and interprofessional collaboration between health professionals. Whilst some of the skills and attributes required to meet the new challenges faced by health professionals today were identified and anticipated for the future by employers in our follow-up of graduates study, the mix, the emphasis and changing demands within the health sector warrants further research.

Additionally, current changes within the Faculty in terms of health science education are substantial. For example, masters level professional entry programs which will be the norm for allied health professional education at the University of Sydney, open up a new era in terms of the challenge of building onto the existing skills and attributes of graduate entry students. How best to match the skills and attributes of this new cohort of students with the changing health care sector is a challenge now facing educators. Given the diverse nature of the education, work and personal experiences of graduate students it will be important for future research to determine how these contribute to the competent professional as well as their university academic and clinical experiences. It will also be necessary to investigate the strengths and weaknesses of the different pedagogies used in graduate education, for example problem based learning and case based learning and determine whether these approaches assist graduates to meet the needs of the current workplace. Another important question to be addressed is what clinical experiences best promote the skills and attributes needed for the current workplace. Graduate students generally spend less time in workplace clinical placements so it is imperative that their learning is optimised during their placements. Finally, what skills and attributes will be most highly valued in the next decade and 207how best to enhance them in a changing educational sector remains to be determined. If we were to look in our crystal ball we would predict that future graduates will need skills in managing support staff such as therapy aides, high level technology skills, research skills and project management skills. This in turn will require curriculum revision and the commencement of the next research cycle.208