Education for Health

ORIGINAL RESEARCH ARTICLE
Year
: 2016  |  Volume : 29  |  Issue : 3  |  Page : 195--202

Remote-online case-based learning: A comparison of remote-online and face-to-face, case-based learning - a randomized controlled trial


Peter Nicklen, Jenny L Keating, Sophie Paynter, Michael Storr, Stephen Maloney 
 Department of Physiotherapy, Monash University, Peninsula Campus, Frankston, Victoria, Australia

Correspondence Address:
Peter Nicklen
P. O. Box 527, Frankston, Victoria 3199
Australia

Abstract

Background: Case-based learning (CBL) is an educational approach where students work in small, collaborative groups to solve problems. Computer assisted learning (CAL) is the implementation of computer technology in education. The purpose of this study was to compare the effects of a remote-online CBL (RO-CBL) with traditional face-to-face CBL on learning the outcomes of undergraduate physiotherapy students. Methods: Participants were randomized to either the control (face-to-face CBL) or to the CAL intervention (RO-CBL). The entire 3rd year physiotherapy cohort (n = 41) at Monash University, Victoria, Australia, were invited to participate in the randomized controlled trial. Outcomes included a postintervention multiple-choice test evaluating the knowledge gained from the CBL, a self-assessment of learning based on examinable learning objectives and student satisfaction with the CBL. In addition, a focus group was conducted investigating perceptions and responses to the online format. Results: Thirty-eight students (control n = 19, intervention n = 19) participated in two CBL sessions and completed the outcome assessments. CBL median scores for the postintervention multiple-choice test were comparable (Wilcoxon rank sum P = 0.61) (median/10 [range] intervention group: 9 [8–10] control group: 10 [7–10]). Of the 15 examinable learning objectives, eight were significantly in favor of the control group, suggesting a greater perceived depth of learning. Eighty-four percent of students (16/19) disagreed with the statement “I enjoyed the method of CBL delivery.” Key themes identified from the focus group included risks associated with the implementation of, challenges of communicating in, and flexibility offered, by web-based programs. Discussion: RO-CBL appears to provide students with a comparable learning experience to traditional CBL. Procedural and infrastructure factors need to be addressed in future studies to counter student dissatisfaction and decreased perceived depth of learning.



How to cite this article:
Nicklen P, Keating JL, Paynter S, Storr M, Maloney S. Remote-online case-based learning: A comparison of remote-online and face-to-face, case-based learning - a randomized controlled trial.Educ Health 2016;29:195-202


How to cite this URL:
Nicklen P, Keating JL, Paynter S, Storr M, Maloney S. Remote-online case-based learning: A comparison of remote-online and face-to-face, case-based learning - a randomized controlled trial. Educ Health [serial online] 2016 [cited 2021 Oct 23 ];29:195-202
Available from: https://www.educationforhealth.net/text.asp?2016/29/3/195/204213


Full Text

 Background



Case-based learning (CBL) is an educational approach wherein students work in small collaborative groups to solve a series of problems that are presented in a context students are likely to encounter in practice.[1] A proposed benefit of CBL is the development of lifelong learning skills in problem solving, self-directed learning, and collaboration, while building an extensive and relevant knowledge base. A recent systematic review [2] found that students perceived that CBL enhanced learning; however, empirical data were inconclusive when CBL was compared to other learning activities. In CBL, students are responsible for identifying knowledge deficits relating to the case, encouraging them to develop and manage their own learning goals and strategies needed for lifelong learners.[3] CBL typically involves face-to-face interaction in small collaborative groups, with a focus on self-directed study.[4]

Computer assisted learning (CAL) is the implementation of computer technology which, when properly designed, can create a rich environment for active learning.[5] A key benefit of e-learning is flexibility.[6] CAL has the potential to link people into learning communities, an important goal of higher education.[5] Courses supported by CAL may require fewer face-to-face lectures, placing fewer geographical constraints on staff and students.[5],[6] Students can learn and progress at their own pace, with computer-based modules accessible at times convenient to the learner.[7] These benefits facilitate active, self-directed learning, potentially enhancing student knowledge and understanding.[8]

A number of reviews [5],[8],[9] have collated relevant trials investigating the effect of CAL on learning outcomes and concluded that CAL is neither superior nor inferior to traditional “face-to-face” delivery. However, these reviews have primarily focused on modes of delivery such as lectures, which require minimal integration or synthesis of learned knowledge or skills. Crawford [1] concluded the following review of six qualitative investigations that CAL could be beneficial within the context of CBL. The proposed benefits of CAL, improved flexibility and increased student autonomy, could complement the current CBL format without compromising student learning.[1] Crawford [1] also commented on the significant technical and time issues faced with web-based CBL with one study [10] reporting that computer-mediated communication resulted in significantly longer time-on-task compared to face-to-face groups. No data were provided on the impact of CAL on learning outcomes within this review.[1]

Two studies [11],[12] compared online problem-based learning (PBL) with face-to-face PBL on student learning outcomes and both reported a small nonsignificant effect in favor of the online methods. Two other studies [10],[13] found a statistically significant effect in favor of the CAL methods. Interventions contained CAL methods of both synchronous (real-time updates) and asynchronous (accessible at any time) communication. Most programs allowed users to communicate in online chat rooms and via discussion boards; others enabled document transfer and interactive exercises. None of the four studies utilized video conferencing within CBL.

Few studies [14],[15],[16] have evaluated the use of web-conferencing within education. These suggest that web-conferencing, when compared to face-to-face teaching, can provide comparable learning outcomes [14] and may enhance the learning experience.[15] Mavridis et al.[16] found that web-conferencing programs are capable of supporting collaborative learning activates based on two case studies. To the best of our knowledge, no randomized controlled studies exist evaluating web-conferencing learning within CBL, making this a unique study.

The aim of this study was to compare the effect of remote-online CBL (RO-CBL) with traditional face-to-face methods on learning outcomes of undergraduate physiotherapy students. The secondary aims were to explore student satisfaction and response, and perceived depth of learning with RO-CBL. Within the Physiotherapy course of Monash University in Melbourne, Australia, CBL is used in every year level to supplement lectures and practical sessions and to consolidate learning. Although CBL does not directly influence students' unit mark (grade), students are evaluated based on the five themes of physiotherapy: Personal and professional development; population, society and health addresses broader society, and population health issues; fundamental knowledge of health sciences; applied practice develops clinical competencies integral to physiotherapy practice; and research and evidence-based practice. Based on the results of earlier work, it was hypothesized that both methods of delivery would produce comparable learning outcomes. It is also hypothesized that student satisfaction and perceived depth of learning would be comparable between the two modes of learning.

 Methods



Design

This study was a randomized controlled trial (RCT) with concealed allocation. The RCT design was used as it is the gold standard for evaluating the effect of an intervention such as CAL,[7] and particularly appropriate given the complex factors that contribute to student learning. The intervention was the only manipulated variable in the students' education, allowing the comparison of RO-CBL to the control, traditional CBL (T-CBL). A mixed method framework of outcome measures (qualitative and quantitative) was used to evaluate outcomes for each group. Ethics approval for the study was obtained through the Monash University Human Research Ethics Committee (Ethics CF12/0086-2012000019).

Participants

This study took place over a single week at Monash University in 2012. Participants were 3rd year undergraduate students, studying a Bachelor of Physiotherapy. The entire 3rd year cohort (n = 41) was invited to participate. Prior to this study, all students had 2 years of face-to-face CBL experience. An independent research assistant recruited participants through face-to-face delivery and distribution of an information package consisting of the explanatory statement and consent form. Consent was necessary for participation. All students consented (100%) to the study activities and were randomized to control or intervention conditions.

Students were stratified based on the preceding semesters, written examination results, and randomized using permuted blocks of two. A random number generator determined whether the first student in the pair was in the control or intervention group. The CBLs were completed in small groups. Students in the control condition were randomly assigned to one of the four control groups, whereas students in the intervention condition were randomly assigned to four intervention groups. An independent research assistant operationalized the predetermined group assignment.

Case-based learning format

In the Monash Physiotherapy Program, the CBL “case” is presented to students at the beginning of the academic week (Part 1). This gives context to the classes held throughout the week, during which the students develop the skills and knowledge to complete the case by the end of the academic week (Part 2). Students are required to assign members of their CBL team the role of leader, scribe, and recorder. The leader has to keep the discussion on topic and monitor that the CBL is completed within the designated time frame. The scribe notes discussions on the whiteboard and the recorder copies what is on the whiteboard and distributes it to other students. Each group has an academic facilitator who monitors each group, encourages discovery and achievement of learning objectives, and provides guidance where appropriate.

The case begins with a trigger (brief scene setting), followed by the case details: The history, physical examination outcomes, further information, actions arising (treatments, tests), reassessment, and closure, to simulate a typical client interaction. Students are required to answer questions presented throughout the case and produce a problem list; these help to guide student discussion. Students are also supplied with recommended reading, to augment understanding of concepts introduced within the case, and a list of the examinable learning objectives. Students identify what they need to learn to understand the case (learning issues); these questions and topics need to be researched and completed by individuals or groups of students during the week and presented to the group in Part 2. Cases were developed to assist students to achieve examinable learning objectives and encourage self-directed learning.

Students participated in two CBL sessions, Part 1 and Part 2, and both the control and intervention groups ran simultaneously. Each of the two sessions ran for approximately 90 min. The CBL, titled “Rachel's pregnancy,” encouraged students to consider the role of the physiotherapist during the stages of pregnancy.

Control

T-CBL was delivered in small “face-to-face” learning groups of 4–6 students at the university. Once the leader, scribe, and recorder were appointed, students were presented with a paper-based case scenario. Each group had a whiteboard. Students also had access to a set of textbooks available in each classroom, as well as Internet access for information gathering. One facilitator, external to the study, monitored student activity.

Intervention

The intervention groups completed the same case via remote-online access, in the form of web-conferencing (RO-CBL) with participants physically isolated from one another. This was achieved by utilizing the computer software “WebEx”[17] manufactured by Cisco; however, the trial used only the generic features of the program, commonly available in all web-conferencing software. It allowed users to communicate through both written text and audio-visual (camera and microphone) means. Users were able to utilize the online workspace much like a white board, and use a common screen to share notes, as well as view, upload, and share documents.

Students were based on campus, which allowed participants to utilize the Internet provided by the university. This also meant that those students who faced technical difficulties could seek assistance. The students supplied their own computers, microphones, and video cameras.

Training

Training sessions occurred prior to the intervention period to familiarize students with the new online learning system to be utilized within the study. All participants were required to attend the first session, which ran for approximately 30 min and introduced the key features of interacting via a web-conference. Only those assigned to the intervention group were required to attend a second and third training session, both running for approximately 45 min, which provided students with the opportunity to trial the program in their small groups.

Outcome measures

The preceding semester grades were compared across the control and intervention groups to test for statistical difference at baseline. The Monash Physiotherapy Program has a fully integrated curriculum and the semester grade represented the outcomes of all written and practical examinations in the previous 12-week semester.

A postintervention survey was administered immediately after the second CBL session. This measured learning and self-assessed perception of learning, satisfaction, and participant demographics. Given the content covered in CBL is specific to the case, no known valid outcome exists; therefore, learning was measured using a purpose built postintervention examination consisting of ten multiple-choice questions. These were sampled from a range of examinable learning objectives that reflected all topics covered in that week's CBL, with input from the lecturers and CBL facilitators. Self-assessed perception of learning was measured for each examinable learning objective on a 3-point scale with the options of superficial, moderate, and in depth; satisfaction with the RO-CBL was measured on a 5-point scale (strongly disagree to strongly agree). All participants completed these. RO-CBL participants were also asked to provide feedback on positives and negatives of the RO-CBL. Data were collected in hard copy, under the supervision of the CBL facilitator.

A 30-min focus group was conducted approximately 2 weeks following the intervention period. Participants were invited to participate by an external research assistant who also facilitated the focus group. Themes such as overall reactions, positives and negatives, ways to improve the RO-CBL and applications to future learning were explored. An external transcription service was utilized to de-identify and produce the focus group transcript.

Data analysis

All quantitative statistical tests were performed using STATA 11[18] manufactured by StataCorp. Pearson's Chi-squared test was used to compare the gender balance in the intervention and control groups. Where t-tests were used, outcomes were checked for normality using a Shapiro–Wilk test. Parametric quantitative data was reported using means and standard deviations, and nonparametric summary statics reported using medians and interquartile ranges. For nonparametric data, a two-sample Wilcoxon rank-sum test was performed to compare groups. A Kruscal–Wallis rank test was used to compare groups for self-perceived learning, this was chosen as two independent variables were compared (the modes of learning) on ranked dependent variables (examinable learning objective). Student satisfaction and student learning are also dependent variables, with the independent variable again being mode of learning. Alpha was set at.05 for all tests.

Thematic analysis was employed to analyze the focus group transcript. This involved classifying and grouping segments of text to create and define themes that emerged from the data.[19] Two independent researchers completed this process. Once identified, both researchers came together to reach a consensus on the final themes. The positives and negatives were extracted from survey and used to supplement the focus group findings.

 Results



Participants

Thirty-eight students (control n = 19 and intervention n = 19) participated in both CBL sessions and completed assessments administered immediately following Part 2. Three students had unexplained absences on the day of the second session, excluding them from the study (attrition rate 7.3%). Eight students participated in the focus group. [Figure 1] is a summary of participation and data collection.{Figure 1}

Baseline data

Means scores for the previous semester's unit grade were comparable (intervention: 77 ± 7, control: 77 ± 5.8; P = 0.98). The male to female ratio of the intervention and control was 6:14 and 5:16, respectively, and were not significantly different (Chi-square[1] =0.2, P = 0.66) [Table 1].{Table 1}

Postintervention multiple-choice test

Median scores for the key outcome measure, a postintervention multiple-choice test, were comparable between groups (median [range] intervention: 9 [8–10], control: 10 [7–10]; P = 0.61) [Table 1].

Examinable learning objectives

Of the 15 examinable learning objectives, 8 were significantly different based on the Kruscal–Wallis rank test. All of these were in favor of the control [Table 1], however only two had different median values compared to the intervention group. These were “describe the anatomy of the pelvis and female pelvic contents” with the control reporting it was covered “moderately” and the intervention group reporting it was covered “superficially,” and “describe key components of physiotherapy assessment” with the control reporting it was covered “in depth” and the intervention group reporting it was covered “moderately.”

Student satisfaction

Satisfaction of students in the intervention group is summarized in [Table 2] and the most frequent response is presented in bold font. Sixteen participants (84%) disagreed with the statement “I enjoyed the method of CBL delivery,” with 12 (63%) disagreeing with the statement “I felt I was able to achieve all objectives given the method of CBL delivery.”{Table 2}

Qualitative outcomes

Eight students chose to participate in the focus group. Three key themes that are relevant to impact on learning emerged within the focus group: Risks to implementation, challenges to communication, and flexibility.

Theme 1: Risks to implementation

One significant risk for the implementation identified by participants is the unpredictable nature of technology. Students required equipment to participate in online CBL, potentially placing additional financial burden on students. Participants also identified the lack of social support within an online environment as a negative.

Four students identified that the CBL took longer than face-to-face, as more time was spent setting up the web-conference. One student considered that the leader would need more training than others to ensure the session ran smoothly.

Theme 2: Challenges to communication

Difficulties associated with communication were recognized by many of the participants, identified as primarily due to the lag time. This meant that participants had to repeat themselves and interrupt the flow of discussion. Another factor noted by students that impeded communication was the difficulty producing a flowchart in the online learning space. This flowchart is used to connect ideas and identify key themes for learning attention. Some students also felt that online learning would not provide them with adequate opportunity to develop communication skills required for future professional practice.

Theme 3: Flexibility

Participants in the focus group recognized the flexible nature of online learning as a benefit.

 Discussion



The T-CBL and remote-online CBL groups did not statistically differ on the postintervention multiple-choice test results. This nonsignificant effect corresponds with findings from Moeller et al.[11] and Raupach et al.[12] Others [10],[13] found that RO-CBL significantly improved learning outcomes when compared to traditional, face-to-face CBL. Students in our study produced comparable outcomes on the postintervention multiple-choice test. Power analysis for nonparametric tests has disputed validity.[20],[21],[22] Wang et al.[23] developed and modelled a method and applying these principles, we estimated that for the observed difference between groups on scores, a sample of 284 participants would be required to conclude that postintervention multiple-choice scores were significantly higher for the control group with power in the order of 80% and significance of 0.05. In addition, the control group perceived that the examinable learning objectives were covered in more depth than did intervention participants.

Sixty-three percent of those in the intervention group reported a perception that RO-CBL negatively impacted their learning. In addition, 63% of participates in RO-CBL did not enjoy the method of CBL delivery. Raupach et al.[12] also found that satisfaction with an online CBL module was low; however, participants in another study [24] found online PBL more engaging than paper-based CBL. This variation in student satisfaction may be a result of the different interventions. Raupach et al.[12] used CAL to replace face-to-face interactions, whereas Poulton et al.[24] utilized CAL to compliment the face-to-face environment. [Table 3] compares findings from this study with other similar studies.{Table 3}

There are several risks to consider when adopting any online component within education. Major service interruptions occurred during the second session, significantly impacting the internet connection speed and the number of times disconnection occurred. This could account for some of the negative responses regarding RO-CBL. Several other studies [10],[11],[25] have faced technical difficulties during the implementation of a web-based CBL. The slow internet connectivity and repeated disconnections experienced by students impeded the flow of discussion and discouraged students from using RO-CBL. These difficulties increased the duration of the CBL session, compared to T-CBL, a finding aligned with our own study. This could mean that more time might need to be assigned to complete CBLs online. Time would also need to be assigned to address adequate education of students in the use of programs if they are to adequately participate in online CBLs. This could have further added to student dissatisfaction.

Effective communication is a key component of CBL. Students in our study faced many difficulties associated with communication, primarily the lag time and feedback interference from other microphones. This corresponds with findings in other studies.[4],[11],[25] Participants had to repeat themselves and interrupt the flow of discussion, which may have negatively impacted on student learning and satisfaction. The lag time and loss of conversational practices, such as turn taking and reference to previous statements, creates challenges in online discussion.[25] Stromso et al.[4] found that the use of synchronous communication restricted communication, particularly in proposing and responding to problems and hypotheses. The fast moving and fragmented discussions of synchronous chat can also intimidate some students, which might encourage quieter students to withdraw from the CBL.[1] Valaitis et al.[25] found that there was a period of adaptation before students engaged in meaningful online PBL discussions. Given the intervention was novel and conducted over only one teaching week, students may not have had sufficient time to acclimatize to the online environment.

Students in the focus group proposed that working online made it easier for students to lose focus and become more distracted. The “invisibility” afforded by remote participation may invite distraction. Raupach et al.[12] found forty percent of students lost interest in the online module while it was running. This may be due to the engagement factor in the material covered and not due to the online platform. In contrast to this, students in one study [25] felt they responded to others in a more focused manner compared to face-to-face, adding depth to discussions. They also felt that they developed the required chat skills from experience with the online platform. A major point of difference between the two studies is that students in Valaitis et al.[25] completed the task in isolation, where as those in Raupach et al.[12] completed an online module face-to-face.

Students identified that flexibility could be an advantage of RO-CBL. The flexible nature of online learning gives students the freedom to determine when and where they participate in CBL.

Participants in Valaitis et al.[25] perceived more learning flexibility with online PBL. These participants valued the flexibility in time and location as well as the opportunity to learn at their own pace.[25] Increased temporal flexibility may enhance students' recruitment and retention of information.[1] The flexibility provided by RO-CBL offers new opportunity and could benefit from students working at a distance.[4],[25] This could allow the implementation of CBL in distant education, as well as, enhance interprofessional education.

Limitations

The main limitation of this study was the technically difficulty faced during the second CBL session. This could have caused a negative bias toward RO-CBL of unknown strength, affecting our results.[11] The small sample size is another limitation and all focus participants were from the intervention group, which can skew results and reduce the transferability of the findings. The intervention period took place over a single teaching week, affecting the amount of time students had to familiarize themselves with the program.

Students were instructed to complete the postintervention outcomes by themselves this included the multiple-choice test. It is possible that the knowledge tests were not sufficiently difficult to differentiate between groups, although the results of this nature are not uncommon in these high achieving students. The exact validity and reliability of the postintervention questionnaire is unknown. This needs to be considered when interpreting our findings, although we believe it has good face-validity. Validation studies are recommended to verify our findings.

 Conclusion



We report a RCT run in the course of typical curriculum delivery, and a model for research of this nature. In this study, the face-to-face T-CBL and RO-CBL CBL groups did not produce significantly different results on the postintervention multiple-choice test. This could indicate that RO-CBL provides students with a comparable learning experience to T-CBL. However, students in the RO-CBL group did not perceive that learning objectives were covered in as much depth compared to face-to-face CBL.

Technical difficulties faced during the intervention period might account for some of the dissatisfaction students had with RO-CBL. Students recognized that the flexibility provided by RO-CBL offers new opportunities and would benefit students working at a distance. This could enhance both distance education as well as interprofessional education.

Implications for educators

This unique study and subsequent findings support the claim that CAL can support the current CBL format by means of web-conferencing.

To successfully incorporate CAL into a CBL environment, technical difficulties need to be overcome with responsive technology, appropriate training, and support to staff and students, with the required infrastructure in place.

It appears that more research is required if CAL in CBL is to be as enjoyable as face to face CBL, and perhaps a balance needs to be struck between the social rewards of learning together and the flexibility afforded by on-line meetings. Procedural and infrastructure factors need to be addressed in future studies to counter the potential for student dissatisfaction and decreased perceived depth of learning.

Financial support and sponsorship

There was no financial support or sponsorship.

Conflicts of interest

There are no conflicts of interest.

References

1Crawford TR. Using problem-based learning in web-based components of nurse education. Nurse Educ Pract 2011;11:124-30.
2Thistlethwaite JE, Davies D, Ekeocha S, Kidd JM, MacDougall C, Matthews P, et al. The effectiveness of case-based learning in health professional education. A BEME systematic review: BEME Guide No 23. Med Teach 2012;34:e421-44.
3Hmelo-Silver CE. Problem-based learning: What and how do students learn? Educ Psychol Rev 2004;16:235-66.
4Stromso H, Grottum P, Lycke K. Content and processes in problem-based learning: A comparison of computer-mediated and face-to-face communication. J Comput Assist Learn 2007;23:271-82.
5Greenhalgh T. Computer assisted learning in undergraduate medical education. BMJ 2001;322:40-4.
6Childs S, Blenkinsopp E, Hall A, Walton G. Effective e-learning for health professionals and students – Barriers and their solutions. A systematic review of the literature – Findings from the HeXL project. Health Info Libr J 2005;22 Suppl 2:20-32.
7Rosenberg H, Grad HA, Matear DW. The effectiveness of computer-aided, self-instructional programs in dental education: A systematic review of the literature. J Dent Educ 2003;67:524-32.
8Lewis MJ, Davies R, Jenkins D, Tait MI. A review of evaluative studies of computer-based learning in nursing education. Nurse Educ Today 2001;21:26-37.
9Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM. Internet-based learning in the health professions: A meta-analysis. JAMA 2008;300:1181-96.
10Dennis JK. Problem-based learning in online vs. face-to-face environments. Educ Health (Abingdon) 2003;16:198-209.
11Moeller S, Spitzer K, Spreckelsen C. How to configure blended problem based learning-results of a randomized trial. Med Teach 2010;32:e328-46.
12Raupach T, Muenscher C, Anders S, Steinbach R, Pukrop T, Hege I, et al. Web-based collaborative training of clinical reasoning: A randomized trial. Med Teach 2009;31:e431-7.
13Taradi SK, Taradi M, Radic K, Pokrajac N. Blending problem-based learning with Web technology positively impacts student learning outcomes in acid-base physiology. Adv Physiol Educ 2005;29:35-9.
14Yeung SK. Synchronous Web Conferencing: Toward a pedagogical model for effective learning advances in SoTL. Scholarsh Teach Learn 2014;1:1-18.
15Long JM, Cavenett SW, Gordon E, Joordens M. Enhancing Learning for Distance Students in an Undergraduate Engineering Course through Real-time Web-Conferencing, in ASEE International Forum; 2014.
16Mavridis A, Tsiatsos T, Tegos S. Exploiting Web Conferencing to Support Collaborative Learning, in Panhellenic Conference on Informatics; 2011.
17Cisco. Cisco Webex; 2012. Available from: http://www.webex.com.au/. [Last accessed on 2014 Jun 14].
18Stata. Stata: Data Analysis and Statistical Software; 2012. Available from: http://www.stata.com/. [Last accessed on 2014 Jun 14].
19Fossey E, Harvey C, McDermott F, Davidson L. Understanding and evaluating qualitative research. Aust N Z J Psychiatry 2002;36:717-32.
20Fagerland MW, Sandvik L. The Wilcoxon-Mann-Whitney test under scrutiny. Stat Med 2009;28:1487-97.
21Rosner B, Glynn RJ. Power and sample size estimation for the Wilcoxon rank sum test with application to comparisons of C statistics from alternative prediction models. Biometrics 2009;65:188-97.
22Zhao YD, Rahardja D, Qu Y. Sample size calculation for the Wilcoxon-Mann-Whitney test adjusting for ties. Stat Med 2008;27:462-8.
23Wang H, Chen B, Chow SC. Sample size determination based on rank tests in clinical trials. J Biopharm Stat 2003;13:735-51.
24Poulton T, Conradi E, Kavia S, Round J, Hilton S. The replacement of 'paper' cases by interactive online virtual patients in problem-based learning. Med Teach 2009;31:752-8.
25Valaitis RK, Sword WA, Jones B, Hodges A. Problem-based learning online: Perceptions of health science students. Adv Health Sci Educ Theory Pract 2005;10:231-52.