|ORIGINAL RESEARCH PAPER
|Year : 2010 | Volume
| Issue : 1 | Page : 351
Outcomes from the Trial Implementation of a Multidisciplinary Online Learning Program in Rural Mental Health Emergency Care
DJ Hills1, T Robinson1, B Kelly2, S Heathcote3
1 NSW Centre for Rural and Remote Mental Health, The University of Newcastle, Orange, New South Wales, Australia
2 Centre for Brain and Mental Health Research, School of Medicine and Public Health, The University of Newcastle, Callaghan, New South Wales, Australia
3 Dubbo-Plains Division of General Practice, New South Wales, Australia
|Date of Submission||29-Apr-2009|
|Date of Acceptance||18-Mar-2010|
|Date of Web Publication||23-Apr-2010|
D J Hills
Locked Bag 6005, Orange NSW 2800
Source of Support: None, Conflict of Interest: None
Introduction: Emergency Departments (EDs) are often the first point of contact for people with acute mental health problems. The impact of the Mental Health Emergency Care (MHEC) online learning program on the knowledge and skills development of clinicians and support staff was evaluated, and options for sustainable implementation of the program were proposed.
Methods: Participants were recruited from the four rural health services and the rural areas of one metropolitan health service in New South Wales, Australia. The MHEC course was conducted online over 24 weeks and comprised four sequential modules, each based on a clinical scenario that demonstrated a typical, acute mental health presentation to a general hospital ED. The course was designed to model collaborative practice in mental health emergency care.
Results: Participants were surveyed before and after completing the MHEC course. Statistically significant improvements were detected in participant confidence in managing key mental health problems, perceived self-efficacy in dealing with challenging, aggressive behaviours and confidence in key online learning skills. Participants also reported very positive experiences in relation to the course content and structure, instructor support, multidisciplinary discussion and feedback, and improvements in their knowledge and skills about emergency mental healthcare.
Conclusions: This study provides evidence for the effectiveness and acceptability of an online educational program in developing the capacity of front-line staff to respond effectively to people who present to EDs with acute mental health problems. The positive outcomes and evaluations of the MHEC course provide support for the deployment of online learning programs that address mental health emergency care.
Keywords: Online learning, interprofessional, mental health, emergency departments, rural
|How to cite this article:|
Hills D J, Robinson T, Kelly B, Heathcote S. Outcomes from the Trial Implementation of a Multidisciplinary Online Learning Program in Rural Mental Health Emergency Care. Educ Health 2010;23:351
|How to cite this URL:|
Hills D J, Robinson T, Kelly B, Heathcote S. Outcomes from the Trial Implementation of a Multidisciplinary Online Learning Program in Rural Mental Health Emergency Care. Educ Health [serial online] 2010 [cited 2021 Sep 23];23:351. Available from: https://www.educationforhealth.net/text.asp?2010/23/1/351/101503
Emergency Departments (EDs) are often the first point of contact in New South Wales (NSW), Australia for people with acute mental health problems. ED staff have increasingly become the ‘gatekeepers’ to the mental health system, but they are often under-prepared to respond to people presenting with acute behavioural disturbance or mental health problems1-3. This report provides an overview of an education program that was implemented across NSW in 2007 with an interdisciplinary cohort of clinicians and support staff providing mental health emergency care to rural and remote Australian populations.
The NSW Department of Health provides an extensive range of health services to the NSW population, which, in 2006, represented one-third of the total resident Australian population of 21 million people4. State government funded health services represent the majority of specialist mental health services, all public hospitals and the majority of community health services. These are structured into eight Area Health Services (AHSs), each managed by a chief executive reporting to the Director-General of the NSW Department of Health. The four Metropolitan AHSs include, within their boundaries, parts of metropolitan and outer metropolitan Sydney, the State capital of NSW, while the four Rural AHSs provide health services to populations in the remainder of the State4. The Rural AHSs cover more than 90% of NSW.
‘Rurality’ in Australia is defined in terms of the Australian Standard Geographical Classification (ASGC) system, which classifies local government areas as being ‘major city’, ‘inner regional’, ‘outer regional’, ‘remote’ or ‘very remote’, based on estimates of an index of population size and accessibility5,6. Consequently, the Metropolitan AHS boundaries can encompass some ‘inner regional’ areas in addition to the ‘major city’ areas, whereas the Rural AHS boundaries primarily encompass ‘inner regional’ or ‘outer regional’ areas. While there are some remote and very remote locations in NSW, there are no local government areas in NSW classified as 100% ‘very remote’ and only one classified as 100% ‘remote’7. Outside of areas classified as ‘major city’, which comprise urban areas with populations greater than 100,0006, the term ‘rural and remote’ is generally used7.
Approximately 12% of the NSW population reported high or very high levels of psychological distress in 2007, with more than 50,000 mental disease hospitalisations during the twelve month period to June 2007, including 10,000 hospitalisations for intentional self-harm8. There were more than 2000 beds in over 450 specialised public sector mental health services in NSW in 20069. In addition, EDs located in the more than 200 public hospitals in NSW10 may provide services to persons presenting with acute mental health problems. A number of State plans and policies have identified the need to improve responses to mental health emergencies in NSW11-13. Nonetheless, the profound rural-urban differences in access, infrastructure and cultural issues and the need to recruit and support a clinical workforce to respond effectively to mental health emergencies within the context of the specific geographic and social characteristics of rural areas are often overlooked14-16.
To date, most of the published workplace education programs that address emergency mental health education have relied on face-to-face delivery. A training course for ED staff in Adelaide, Australia, reported improvements in participants’ knowledge and skills in responding to people with mental health and drug and alcohol issues17. In addition, the Association for Australian Rural Nurses and Midwives implemented an education program that addressed mental health emergencies for rural ED staff. The program resulted in improvements in the confidence and skills of participants18. There is, however, a paucity of literature that pertains to online mental health emergency care education programs.
While the internet may be viewed as the ideal vehicle for providing continuing professional development programs, particularly for learners who are located in more isolated areas, the loss of opportunity for face-to-face contact in a physical campus seems to underpin some concerns about the transition of programs to online modes of delivery19. Despite these concerns, online learning is pedagogically sound and widens access to learning programs where participation may be restricted due to geographical factors, such as rural or remote location20. Online learning can also reduce the financial and emotional burden of travelling long distances to attend face-to-face programs21. Additionally, online learning provides opportunities for standardising content, delivery and course management, while encouraging coherence between learners’ needs, instructors’ perspectives, the learning environment and the organisational culture22.
The framework for the Mental Health Emergency Care (MHEC) online learning program was initially developed in collaboration with mental health service consumers, clinicians in mental health and emergency care, Police and Ambulance personnel, and health educators for the Rural Mental Health Critical Care pilot (RMHCCp), which was implemented in the Northern NSW region in Australia in 200523. The RMHCCp piloted a model of care that was developed in response to the increasing acuity of mental health presentations to NSW hospital EDs, as well as government concerns about the management of mental health consumers in NSW EDs and during transportation to psychiatric units23. The NSW Department of Health then funded the NSW Centre for Rural and Remote Mental Health to redevelop and deliver the education program to a sample of employees in the rural regions of NSW, including in one Metropolitan AHS with rural areas within its boundaries.
The MHEC online learning program was designed to build on the existing skills of health workers to optimise the quality and safety of mental healthcare for people with acute mental health problems presenting to general hospitals in NSW. Critical learning areas included: key legislation, policy and guidelines influencing the provision of mental health services; assessing, identifying and managing risk and safety; therapeutic communication and de-escalation skills; and effectively managing transitions in care.
The objectives of the trial implementation of the MHEC online learning program were to test the viability of the course in reaching a multidisciplinary cohort of clinicians in NSW health services, evaluate the impact of the program on the knowledge and skill development of participants and propose options for sustainable implementation of the program.
The MHEC course was delivered during 2007 and implemented over 24 weeks. There were four sequential modules, each of approximately five weeks duration. A program introduction in Module 1 provided intensive orientation to the online learning environment. Each module was based on a clinical scenario that demonstrated a typical acute mental health presentation to a general hospital ED, enabling the application of key legislation, policy and guidelines to ‘real life’ situations that are commonly encountered. Each participant received an interactive CD containing course content and problem-based learning activities. The online component was hosted on the University of Newcastle’s Blackboard learning management system, which enabled participants to access specific readings, use discussion boards and group forums, submit periodic assessments and review grades. Online discussions were integrated into the learning and assessment structure to establish collaborative decision-making, problem-solving and critical reflection, both as a fundamental component of learning and as a model for collaborative practice in mental health emergency care.
The target population was ED and Mental Health Service clinicians, as well as Health and Security Assistants (non-clinical hospital assistants) working within rurally-based clinical services. Recruitment to the trial implementation of the MHEC course was undertaken by convenience sampling. Information about the course was distributed through health service email networks via the Directors and senior educators of Mental Health Services in the four Rural AHSs in NSW. In the Metropolitan AHS included in the study, information was provided to teams servicing rural regions of that AHS. Posters about the course included the contact details of project coordinators for interested persons to obtain further information and an ‘expression of interest form’. Copies of the posters and the ‘expression of interest form’ were provided to the Directors and senior educators for distribution in their Mental Health Services and in general hospital EDs. Participants voluntarily applied to enroll in the course using the ‘expression of interest form’, with the primary conditions of enrolment being that they provided mental health emergency care services in one of the targeted public sector AHSs and were willing to participate in the course.
A survey of participants enrolled in the course was undertaken at both the commencement and completion of the course. Customised questionnaires were developed and administered for voluntary completion by course participants. Questionnaires were used to gather participant profile data, including their experience in managing people with mental health problems and aggressive behaviour. Participants were requested to provide responses to items relating to ‘computer and internet skills’, ‘confidence in mental health and related issues’ and ‘understanding the role of different services’. Perceived self-efficacy (PSE) in dealing with aggressive behaviours was measured using a validated scale that has been applied in a number of settings24-26. To establish face validity, the questionnaires were reviewed by an expert panel27, composed of two academic colleagues and a senior mental health clinician not directly engaged in the implementation of the MHEC course.
The non-parametric, independent sample Mann-Whitney U-test was employed to determine the significance of differences in the distribution of pre-course and post-course scores for questionnaire items in each learning or skill area. This statistical test was utilised because the pre-course and post-course results were unmatched, there was a considerable difference between the pre-course and post-course sample sizes and distributions were skewed.
The post-course questionnaire also included items relating to course quality issues, such as acceptability and accessibility of the course structure and content, and online learning management systems, as well as facilitator support and feedback. Qualitative data were collected from comments provided by participants in the post-course survey questionnaires. In addition, comments were elicited and transcribed in a videoconference of course participants and instructors, conducted across five sites in NSW during the final week of the program. Comments were aggregated to search for possible themes in the responses.
Institutional ethics clearance was not sought for the trial implementation of the MHEC course as the NSW Department of Health directly funded the NSW Centre for Rural and Remote Mental Health to deliver and evaluate the trial. In addition, participation was completely voluntary, there was no randomisation of participants and pre-course and post-course quantitative evaluation components were not able to be directly linked to individual respondents or the facilities in which they worked, nor were they matched. Qualitative data were either drawn from non-identifiable questionnaires or immediately de-identified on transcription from group interview responses.
Participant profiles: From the initial base of 71 participants who enrolled in the trial, 26 (36.6%) enrolees exited the course prior to completing Module 1, with most leaving before actually completing any course activities. Of the 45 remaining participants, 10 people (22.2%) exited at other stages of the program, resulting in 35 (77.8%) completing all course requirements. The main reasons reported by enrolees for not commencing or for leaving the course included personal/family reasons (22.4%), being unable to commit to the program requirements (14.1%), health reasons (11.8%), committed to another course (9.4%), problems with information technology (IT) access (9.4%) and limited IT literacy (9.4%).
The mean age of the 71 enrolees at course commencement was 42.9 (median 44), with a range from 24 to 58 years. This increased slightly to 43.7 years (median 44) by the end of the program. Participation rates by gender changed slightly from 18.3% male and 81.7% female at enrolment to 22.9% male and 77.1% female at completion. Participation rates by professional group remained relatively stable during the implementation of the MHEC course, with a slight drift away from Enrolled Nurses and Assistants in Nursing to the Registered Nurse and Health Security Assistant categories (Table 1). Participation rates by health delivery area were also relatively stable during the program, with just a slight shift away from the ED/Small Rural Hospital category toward other categories (Table 2). In addition, participation rates by rural classification remained stable over the conduct of the course, with more than 90% of course participants based in centres classified as ‘inner regional’ or ‘outer regional’ (Table 3). This is not surprising in the NSW context, since most mental health and general hospital facilities are based in major towns within rural areas. Overall, the key nursing, allied health and clinical support disciplines from a range of health service areas providing mental health emergency care in rural areas of NSW were represented in the sample.
Table 1: Study Participation Rates by Profession at Commencement, Withdrawal and Completion
Table 2: Study Participation Rates by Health Delivery Area at Commencement, Withdrawal and Completion
Table 3: Study Participation Rates by Australian Standard Geographical Classification (ASGC)
Survey responses: The response rate for the pre-course survey was 77.5% (55) and for the post-course survey was 73.3% (33). Age, gender and facility in which respondents worked were not requested on the pre-course and post-course questionnaires, primarily to help preserve anonymity for this small sample of health service employees. In regard to location by AHS, 89.1% (49) of respondents were employed in Rural AHSs. Overall, the profiles of respondents returning questionnaires in the pre-course and post-course surveys, in terms of profession and health service area, were very similar to participant profiles at the commencement and completion of the program. This suggested that questionnaire responses were broadly representative of the views of all course participants, including those who exited prior to completion.
In general, mental health presentations are clearly common in EDs, with 90% of respondents to the pre-course survey indicating that they frequently encountered people with a mental health problem and 80% of respondents indicating that they frequently encountered people with challenging or difficult behaviours. A similar outcome was found in the post-course survey.
Confidence in IT and online learning: Participant level of confidence in relation to five ‘computer and internet related’ items was measured using seven point Likert-type scales (range 1-7), with ‘not confident at all’ = 1 and ‘very confident’ = 7. As presented in Table 4, confidence increased, with statistically significant differences detected in the distribution of pre-course and post-course scores for three key areas of participation in online learning: using email communication; searching the internet; and participating in online discussions.
Table 4: Confidence in Information Technology and Online Learning
Perceived self-efficacy and aggressive behaviour: Respondents were asked to indicate their feelings in dealing with challenging, aggressive behaviours of clients directed toward them, in relation to five items. With this single factor scale, item scores were summed to provide an overall level of perceived self-efficacy (PSE), with a possible range of 5 to 35. A significant improvement in PSE was achieved (z=3.835, p=0.0001), with the pre-course median and mean increasing from 21 and 21.5 (sd: 6.0), respectively, to post-course scores of 26 and 26.3 (sd: 3.9), respectively.
Confidence in mental health and related issues: Participant level of confidence in relation to eight ‘mental health issue’ items was measured using seven point Likert-type scales (range 1-7). As presented in Table 5, confidence increased, with statistically significant differences detected in the distribution of pre-course and post-course scores on all items except ‘talking to elderly people about their mental health issues’.
Table 5: Confidence in Mental Health and Related Issues
Understanding the role of different services: Participants’ level of understanding in relation to eight ‘role of different services in mental health emergencies’ items was also assessed using the seven point Likert-type scales. The ability to work in partnership with other services, including understanding the roles of General Practitioners (GPs), Ambulance, Police, Sexual Assault and Child Protection services in mental health emergency care, is a fundamental skill for all rural practitioners. For all items, however, there were no significant differences in the distribution of pre-course and post-course scores.
Qualitative data: Additional information was collected from the comments sections of the post-course questionnaires and from a videoconference held at the conclusion of the course. Comments were provided by 28 (84.8%) post-course survey respondents. In addition, 16 (48.5%) participants attended the post-course videoconference to provide further feedback on their experiences in the course. Individual comments recorded in questionnaires and transcribed from the videoconference (n=222) were aggregated and categorised. Most of the individual comments related to three main themes: course content and learning outcomes (n=107; 48.2%); computer-based and online learning (n=64; 28.8%); managing the opportunities and challenges of tertiary studies whilst working in demanding jobs (n=18; 8.1%); and other miscellaneous comments (n=32; 14.4%).
A number of course participants reported on their enjoyment of the course and the support provided (n=13; 5.9%). As one participant stated:
“I have had so much fun doing this course, and have gained an incredible amount of knowledge (not only mental health stuff!!). I have completed several college courses and have not achieved anywhere near the amount of satisfaction that I have come away with this time. The support was amazing.”
Other participants commented on their increased confidence and skills in working with and managing people presenting to health services for mental health emergency care (n=16; 7.2%). A typical comment was:
“Since completing the MHEC course my knowledge has greatly increased as has my confidence. I have a greater understanding of the roles of all involved in various presentations to the ED when there are mental health issues.”
Some course participants originally found computer-based and online learning a challenge but, nevertheless, their participation resulted in improved computer and online skills (n=8; 3.6%). For example, one participant indicated:
“Initially this was very difficult to the not so savvy computer user. With support from course coordinators and other personal supports this became very easy.”
There were also numerous general comments about engaging in the course, some reflecting the challenges that were faced (n=11; 5.0%). One participant captured some of the conflicting feelings experienced when working full-time and studying:
“Due to my busy schedule with work/shifts changing etc, getting work in was a priority (assessments and activities). Sadly, I rarely felt like discussing issues with the other students because it was too time consuming and I was always trying to squeeze it in.”
Overall, the feedback provided in the questionnaires and videoconference supported the findings of the quantitative data analyses in relation to improvements in knowledge, skills and confidence related to key learning areas in mental health emergency care. There was generally strong support for the online mode of course delivery. Some participants suggested that more face-to-face elements could be useful in supporting engagement and a greater sense of connectedness with other course participants. Additionally, there were a number of participants who requested assistance in relation to difficulties experienced in using workplace computers and computer systems, primarily due to an absence of basic software, such as Adobe ‘Acrobat’ and ‘Flash’ software, slow connection and download speeds and ‘time-outs’ whilst working online. A number of participants, including professional staff, were unable to secure email or computer access at work, with a number of people opting to purchase computers and broadband access to complete the course at home.
The evaluation of the MHEC online learning program demonstrated both a positive impact on participant knowledge and skills, as well as a positive educational experience for an interdisciplinary cohort of clinicians and support staff. In addition, the analysis of reasons for withdrawal provided by participants exiting early from the course suggests that family, health and lifestyle issues were the key reasons for not completing the course, with less than 20% of withdrawals explicitly relating to IT access or literacy issues.
The significant improvements in confidence in managing a range of mental health and related issues, self-efficacy in dealing with aggressive behaviours and confidence in computer and IT skills suggest that participation in this type of program can contribute to improvements in the safety and quality of mental health emergency care. This is an important contribution to understanding the acceptability and effectiveness of online learning programs for rural clinicians. It is also clear, however, that key learning areas require further support within the program, such as working with elderly people and understanding the roles of different services in mental health emergencies.
A particular challenge in the delivery of this online program was the diversity of IT systems throughout NSW public sector health services and the absence, in some facilities, of the basic requirements for interactive online learning, including email and internet access. Nonetheless, the majority of participants who remained in the course were motivated and able to work around any deficiencies in the work environment. For some, this necessitated purchasing a computer and broadband access for home use. Recent research into the barriers facing Australian nurses’ and other professionals’ use of IT in the workplace28,29, confirms that access to IT services can be a significant issue that impacts on both professional practice and professional development.
Key limitations in the current study relate to the utilisation of a small, convenience sample of health service employees, and to being unable to match pre-course and post-course survey data. Consequently, it is not possible to generalise the findings to a broader population of personnel providing mental health emergency care services. While the absence of paired responses created a potential for sample (self-selection) bias to affect the validity of inferences drawn from the data analysis, key respondent profile indicators were similar from enrolment to completion, and across the pre-course and post-course surveys. This suggests that there was minimal or no impact on the results due to sample bias. A further limitation of the study is that a subsequent evaluation, such as a six or 12 month post-course follow-up, could not be conducted, so that the stability of the impact of the course could not be assessed.
Despite these limitations, the current study provides important information for subsequent course development and evaluation in mental healthcare. In particular, the study has demonstrated that a systematic introduction to the online learning environment, backed up by individualised problem-solving and emotional support, can assist novices to rapidly master online learning, thus developing a set of skills that have much broader application, both personally and professionally. In spite of an early dropout rate of 37%, participants reported high levels of satisfaction with the program and any concerns about retention rates in this course would be consistent with the broader concerns of Australian educators about student retention rates in all courses30. In terms of strategies for improvement, there is a need to ensure a more rigorous research design for future course evaluations, including matching pre-course and post-course responses, adding a control group for comparative purposes and undertaking post-course follow-up evaluation to assess the stability of impact measures over time.
There is significant interest and need for an educational program that addresses mental health emergency care. This is a very challenging issue for all practitioners, but particularly in rural and remote locations where emergency response times may be slower due to geographical distances. Such a program needs to be offered broadly as a workplace development strategy for personnel involved in mental health emergency care. Integrating this course into postgraduate programs could be an important incentive for recruiting and sustaining participation. Considering some of the difficulties experienced by health workers in committing to tertiary study and balancing this against work and family commitments, wider dissemination may also rely on offering a shorter, non-accredited version of the program as an alternative mode of knowledge and skill development.
The MHEC online learning program was funded by the NSW Department of Health. The authors thank Dr. Richard Hastings of Bangor University, Wales, for his generosity in providing and granting permission to use the Efficacy in Dealing with Challenging Behaviours Scale. In addition, the authors would like to acknowledge the Directors of Area Mental Health Services and health managers who generously supported the staff who participated in the program. Finally, thanks must also go to the senior mental health clinicians and educators, and the course participants who demonstrated their commitment to improving the delivery of mental health emergency care in NSW.
The authors declare that they have no competing interests.
1. Happell B, Summers M, Pinikahana J. Measuring the effectiveness of the national mental health triage scale in an emergency department. International Journal of Mental Health Nursing. 2003; 12:288-292.
2. McDonough S, Wynaden D, Finn M, McGowan S, Chapman R, Gray S. Emergency department mental health triage and consultancy service: An advanced practice role for mental health nurses. Contemporary Nurse. 2003; 14(2):138-144.
3. Webster S, Harrison L. The multidisciplinary approach to mental health crisis management: An Australian example. Journal of Psychiatric and Mental Health Nursing. 2008; 11:21-29.
4. Population Health Division. The health of the people of New South Wales – Report of the chief health officer 2008 [Internet]. Sydney: NSW Health Department. 2008 [cited 2009 Dec 23]. Available from: http://www.health.nsw.gov.au/publichealth/chorep/index.asp
5. Australian Bureau of Statistics. ASGC remoteness classification: Purpose and use census paper no. 03/01 [Internet]. Canberra: Commonwealth of Australia. 2003 [cited 2009 Dec 23]. Available from: http://www.abs.gov.au/websitedbs/D3110122.nsf/4a255eef008309e44a255eef00061e57/f9c96fb635cce780ca256d420005dc02/$FILE/Remoteness_Paper_text_final.pdf
6. Australian Institute of Health and Welfare (AIHW). Rural, regional and remote health: A guide to remoteness classifications. Rural health series no. 4, cat. no. PHE 53 [Internet]. Canberra: AIHW. 2004 [cited 2009 Dec 23]. Available from: http://www.aihw.gov.au/publications/phe/rrrh-gtrc/rrrh-gtrc.pdf
7. Population Health Division. The health of the people of New South Wales – Report of the chief health officer 2008, data book – Rural and remote populations [Internet]. Sydney: NSW Health Department. 2008 [cited 2009 Dec 23]. Available from: http://www.health.nsw.gov.au/publichealth/chorep/rur/rur_databook.pdf
8. Population Health Division. The health of the people of New South Wales – Report of the chief health officer 2008, data book – mental health [Internet]. Sydney: NSW Health Department. 2008 [cited 2009 Dec 23]. Available from: http://www.health.nsw.gov.au/publichealth/chorep/databooks/men_databook.asp
9. Australian Institute of Health and Welfare (AIHW). Mental health services in Australia 2005–06. Mental health series no. 10, cat. no. HSE 562008 [Internet]. Canberra: AIHW. 2008 [cited 2009 Dec 23]. Available from: http://www.aihw.gov.au/publications/index.cfm/title/10594
10. NSW Health. Hospitals [Internet]. North Sydney: NSW Department of Health. 2009 [cited 2009 Dec 23]. Available from: http://www.health.nsw.gov.au/hospitals/index.asp
11. NSW Health. NSW rural health plan: NSW government response to the report of the rural health implementation coordination group. North Sydney: NSW Health Department; 2002.
12. Mental Health and Drug and Alcohol Office. The rural mental health emergency and critical care access plan. North Sydney: NSW Health Department; 2006.
13. NSW Health. A new direction for NSW: state health plan – Towards 2010. North Sydney: NSW Health Department; 2007.
14. O’Connor-Fleming ML, Parker E. Health promotion: Principles and practice in the Australian context. 2nd ed. Crows Nest: Allen & Unwin; 2001.
15. Fuller J, Edwards J, Procter N, Ross J. Mental health in rural and remote Australia. In: D. Wilkinson D, Blue I, editors. The new rural health. Oxford: Oxford University Press; 2002. p. 171-185.
16. Taylor J, Edwards J, Guest M. General practice and hospital mental healthcare integration: Issues in rural and remote South Australia: summary of findings. Canberra: Australian Rural Health Education Network; 2005.
17. King DL, Kalucy RS, de Crespigny CF, Stuhlmiller CM, Thomas LJ. Mental health and alcohol and other drug training for emergency department workers: One solution to help manage increasing demand. Emergency Medicine Australasia. 2004; 60:155-160.
18. Philip T. Mental health emergencies project: Evaluation report. Deakin: Australian Rural Nurses and Midwives Inc.; 2006.
19. Rahman M. Faculty recruitment strategies for online programs. Online Journal of Distance Learning Administrators [Internet]. 2001 [cited 2009 Dec 23]; 4(4). Available from: http://www.westga.edu/%7Edistance/ojdla/winter44/rahman44.html
20. Curran C. The phenomenon of on-line learning. European Journal of Education. 2001; 36(2):113-132.
21. Grootjans J, Hunt H, Cresswell S, Robinson T. Exploration of self-identified needs of alcohol and other drug workers. Australian Journal of Rural Health. 2006; 14(2):62-65.
22. Newton D, Ellis A. Development of an e-learning culture in the Australian Army. International Journal on E-Learning. 2007; 6(4):543-563.
23. Wolfenden K. Report on the evaluation of the pilot project, rural mental health critical care pilot. North Sydney: NSW Department of Health; 2005.
24. Hastings RP, Brown T. Behavioural knowledge, causal beliefs and self-efficacy as predictors of special educators' emotional reactions to challenging behaviours. Journal of Intellectual Disability Research. 2002; 46(2):144-150.
25. Lee F. Violence in A&E: The role of training and self-efficacy. Nursing Standard. 2001; 15(46):33-38.
26. Hills D. Relationships between aggression management training, perceived self-efficacy and rural general hospital nurses’ experiences of patient aggression. Contemporary Nurse. 2008; 31(1):20-31.
27. de Vaus DA. Surveys in social research. 5th ed. Crows Nest: Allen & Unwin; 2002.
28. Hegney D, Buikstra E, Eley R, Fallon T, Gilmore V, Soar J. Nurses and information technology: Final report. Canberra: Commonwealth of Australia; 2007.
29. Herrington A, Herrington J. Effective use of the internet – Keeping professionals working in rural Australia. Canberra: Rural Research and Development Corporation; 2006.
30. Duff A, Quinn D. Retention in online courses: Using a motivational framework and online peers to enlighten potential learners about learning online. Journal of Learning Design. 2006; 1(3):29-40.