Developing OSCE for Pharmacy Students in The Pandemic Era

The COVID-19 pandemic has affected many aspects of life, one of which is in the field of education. Transformation toward various e-learning process has been conducted in order to prevent transmission of the disease caused by massive gathering. Obstacles appear when trying to deliver and assess practical skills which are usually done face to face with the teachers or instructors. These obstacles are also seen in pharmacy schools where skills such as patient counselling, compounding and dispensing, etc. are important competencies for the students to achieve. In order to overcome these obstacles, we developed a method for conducting the objective structured clinical examination (OSCE) through online process. This study was aimed to evaluate the new online OSCE method for pharmacy students to be conducted in the pandemic era. The evaluation of online OSCE trial was conducted using a simple Google form-based questionnaire which evaluated the platform used, the process of online OSCE and the instruction understandings. Respondents consisted of 52 examinees from the students of Pharmacist Profession Program UMY and 15 examiners from the lecturers or assistants who participated in the online OSCE trial held in June 2020. Data was analyzed descriptively to give a comprehensive picture on respondents’ perception towards the new method. Jurnal Farmasi Indonesia Vol. 17 No. 02, bulan November tahun 2020 online: jfi.setiabudi.ac.id ISSN: 1693-8615 EISSN : 2302-4291


INTRODUCTION
Objective Structured Clinical Examination or familiarly abbreviated as OSCE is a structured examination used to assess clinical skills of health professional students. In Indonesia, OSCE was firstly used by the medical faculty in order to evaluate skill competencies of the medical students during their study. Evaluating clinical skills are necessary for health professional students to ensure they are capable in conducting specific skills of their profession as a healthcare provider. Inadequate skills of graduate health professional students could lead to insufficiency of healthcare services which could be fatal for individual patients as well as the whole healthcare system. Therefore, medical students, nurse students, pharmacy students and other health professional students are required for a minimum safety performance throughout their study [1] [2].
As the COVID-19 pandemic begin in 2020, various modification has been conducted in many aspects of life, not to mention in the field of education itself. With the government regulation to lockdown several community services, schools and universities has also been affected in the means of avoiding massive gathering. Therefore, e-learning process has been introduced to be the main learning and teaching method throughout the country during the pandemic. Internet connectivity was and has still been the main problem in implementing online courses to the students in many countries [3][4] [5]. The next problem was then the method for evaluating students learning process. Examinations were usually conducted offline where it is possible to supervise all students and avoiding any cheating. This is also the matter specifically for OSCE examination where there should be close observation on how the students perform each skill competency [1]. Therefore, development of a modified OSCE should be done to assess students' skills in the pandemic era where close interactions are prohibited.
The Pharmacist Professional Program is a program to graduate licensed pharmacist in Indonesia. The use of OSCE as one of the national examinations for obtaining license as a registered pharmacist is essential to ensure the fresh graduates are capable in conducting pharmacy services in the wide area of pharmacy fields such as drug formulation and production, drug distribution as well as patient care [6] [7]. With the pandemic occurring throughout the year, developing an online-based OSCE could be a breakthrough to overcome the need for skill assessment in pharmacy schools.

Jurnal Farmasi Indonesia
This study was aimed to evaluate a new method of online OSCE for assessing pharmacy practice skills during the pandemic. Evaluation of the online OSCE trial was done to ensure better performance in the following OSCE examinations during the pandemic era or similar situations in the future.

1. RESPONDENTS
Respondents recruited in this study consisted of the examiners and examinees of the OSCE trial. The 15 examiners were lecturers and lecturer's assistants from the Pharmacist Professional Program of UMY. Examinees recruited in this study were the fifth-year Pharmacy students (or first-year of Pharmacist Professional Program), Faculty of Medical and Health Sciences, UMY. These students have graduated from the 4-year Bachelor degree of Pharmacy and continued to the Pharmacist Professional Program in UMY. A total of 52 students participated in this study.

METHOD
The online OSCE trial was prepared by the department to be conducted for the second Block of the 1 st semester, which was the Clinical Pharmacy Block (Course code: PA.I.02). Competencies evaluated in this OSCE consisted of three clinical competencies such as medication reconciliation, monitoring of adverse drug reaction, and patient counselling. The preparation of the trial was held in May 2020.
As implemented in the non-pandemic situation, preparation of OSCE comprise of several aspects which are the case scenario and instruction, worksheets, scoring rubric and form, as well as other documents or references used for each competency. All documents were reviewed to make sure every aspect of each competency has been considered. Modification due to the online process was done in order to ease the examiners and examinees in reading the instructions and other references, using and filling worksheets and scoring form, while drawing up attempts to prevent any fraud during the exam. It was agreed that the platform used for the trial was Microsoft Teams. Before trial, all examiners were socialized on the OSCE flow in order to ensure each examiner understood how to perform during the trial.
The online OSCE trial for the Clinical Pharmacy Block was conducted in June 2020. For each competency, a number of 3 examiners was prepared to give scoring for the 52 examinees (17 to 18 examinees for each examiners). Evaluation of the OSCE trial was done using a Google form-based survey which was given to the respondents on the following day after the online OSCE was conducted. Two forms were prepared to obtain evaluation from the examiners and the examinees. Descriptive analysis was undertaken to describe the perception of the respondents on three areas such as the evaluation on the platform, evaluation on the OSCE process and evaluation on instruction understanding.

DEVELOPMENT OF ONLINE OSCE
Developing an online OSCE to evaluate skill competencies of pharmacy students is a huge challenge to overcome. OSCE has become the main process in evaluating skills for health professional students, not to mention pharmacist students [1]. With the COVID-19 pandemic beginning in the year 2020, the department of Pharmacist Professional Program UMY has been trying to develop various method to adapt with the pandemic. The online OSCE was introduced by the department as a solution to evaluate pharmacy skills of the students while avoiding massive gathering. The development process could be divided into three steps which was determining the platform used for the OSCE, determining the flow or process of the online OSCE, and preparing materials or documents for the OSCE.
In determining the platform used for the online OSCE, several consideration were taken such as ease in use, internet quota, and examiner or examinees familiarity with the platform. A quick survey was given to the students on which platform was the easiest to access or use based on their experience during online lectures in the previous courses from April to June 2020. Fig. 1 shows the preferences of the students from three platforms used in the online lectures.

Figure 1. Preference of Online Platform
Microsoft teams was agreed to be the most feasible platform to be used. UMY has subscribed Microsoft which allows every staff and students in the organization to easily and freely use all application featured in Microsoft. Teams is capable in providing various features and services that allow the members to have open chats, voice and video calls as well as content sharing and other learning features [8]. Schools and universities around the world have gain benefit from using this platform, especially in conducting online teaching and learning process [9][10].
The online OSCE was meant to mimic as much as possible with the standard OSCE usually conducted before the pandemic in order to maintain the same quality for pharmacists' skill evaluation. Therefore, the process was developed with the same steps as the normal OSCE. Important steps which were modified due to the online process were the way examinees search information from the reference and the way examiners give scoring for each examinee. Due to the pandemic, preparation for the materials and documents used in the OSCE which consist of case scenarios and instruction, worksheets, scoring rubric and other forms was modified. All of these documents were prepared as softfiles to be shared to the students during the exam. Therefore, the length of each file should be simplified in order to ease the examinees in reading the documents through Microsoft teams. This was also the matter for the references used to solve the cases. Students who were mostly at their hometown during the pandemic had difficulty in getting access to references that should be used in the OSCE. Therefore we also provided scanned documents or specific pages of references that were applicable for the skill competencies.
As for the scoring rubric and forms which were used and should be filled out by the examiners, all forms were also prepared as softfile and sent to the examiners before the OSCE. Scoring rubric is a set of rubric used to guide the examiners in giving score for each competency area completed by the students. The score achieved for each competency was then filled in the scoring form which was provided as an excel form.

IMPLEMENTATION OF ONLINE OSCE
The online OSCE trial was conducted in June 2020 for the second block of the first semester in Pharmacist Professional Program curricula which was the Clinical Pharmacy Block. Three skill competencies were evaluated for this trial, medication reconciliation process, monitoring of adverse drug reaction and patient counselling. These competencies are included in the clinical pharmacists' activities mentioned in the regulation of The Indonesian Ministry of Health regarding pharmacy service standard in the hospital and primary healthcare center [11].
Medication reconciliation process is a process of comparing previous routine or non-routine patients' medication with the newly prescribed medication by the physician in order to identify and analyze any discrepencies that can lead to adverse drug events [12] [13]. Evaluation of this competency included the skill to conduct information gathering, analyzing and giving recommendation for drug related problems (DRPs) identified during the medication reconciliation process, as well as their professional behaviour. Students were observed on how they communicate to patients in order to draw important informations regarding their routine or previous medications, how they identify discrepancies in the form of drug related problems, and also how they provide recommendations to overcome the DRPs.
The second competency tested in the online OSCE was monitoring of adverse drug reactions (ADR). ADR is harmful or unpleasant reactions that occur in the patients after or during the consumption of certain medication [14]. ADR can form as a simple and non-severe reactions such as mild urticaria or in some severe cases can be seen as hypersensitivity reactions such as anaphylaxis. The skill was divided into 3 competency area which were information gathering, problem determination and behavioural as well as professional aspects. Students were observed on how they could identify the potential drug to have caused the ADR, how they can use the Naranjo tool to give interpretation on the potential ADR, and how they can perform professionaly as a pharmacist during the process. The skill to conduct ADR monitoring is an essential skill that is beneficial in lowering the number of preventable ADR which can give negative impacts to the patients [15].
Patient counselling was the third competency tested in the OSCE. Patient counselling is an important skill as a pharmacist, especially those who give clinical and community services such as in the hospital, clinic, primary health care center, and drug stores. Patient counselling by the pharmacist has shown effect on patient medication management as well as medication error prevention [16] [17]. Student patient counselling was evaluated based on three competency areas which were effective communication, documentation skill, and professional behaviour as a pharmacist.
The online OSCE was performed with a total duration of around four hours. Fifty two students were divided into five groups. Each group consisted of three examiners who were responsible in evaluating one specific skill. The examiners and examinees were also connected through Whats app application to manage the sequence of the students entering Microsoft Team and also to give further instructions needed for any obstacles during the OSCE.

EVALUATION OF ONLINE OSCE
In order to evaluate the online OSCE trial, the department prepared a survey for both the examiners and also the examinees. The survey was arranged using Google form application which consisted of several questions including of evaluation of platform, evaluation of process and evaluation of instructions.
Microsoft Team was used as the main platform of the online OSCE trial. A specific team was arranged for this OSCE which was then divided into three channels for three groups of students. Fig 2 shows how the examiners and examinees feel about the platform used. Since then, various online lectures have been performed using this platform. This could be the reason why most examinees were already familier with the platform. Familiarity of students towards such technology could be drawn by several factors including ownership, accessibility and usage of digital tools, as well as accessibility to the internet and previous computer-based trainings [18] . Therefore it is essential to understand the familiarity degree of the students toward technology before implementing any new concept of digital learning or assessment.
The second evaluation conducted was regarding to the OSCE process which included the evaluation of the sequence of the exam process as well as sufficiency of the time allocated for each stations. The results were important to improve the process for the following online OSCE. Based on the result in Table 1, the overall sequence of the online OSCE process did not give much difficulty for the students nor the examiners to follow. It is necessary to prepare examiners with adequate familiarity in technology, such as lecturers who are used to handling e-learning process in their daily lectures. These type of examiners are usually quickly adaptive with various online methods and are able to manage simple troubleshooting that could occur in the online process. The main problem which could affect the flow of exam was the stability of the internet connection. It is therefore important to assure that the students are well prepared in the means of hardware and connectivity before joining the exam. Students should conduct trials in determining the best way or location with optimum internet capacity in their own places. The success of such e-learning process is influenced by several factors that should then be considered and prepared by the department, academic staff and also the students. Factors contributing to the success include the personal, social factors, cultural aspects, as well as technological, organizational and environmental factors [19]. As the pandemic being predicted to stay quite long in Indonesia, online process would probably still be the main learning method throughout the year. Hence, students must be able to build their own best strategy for their online learning process.
As for the time sufficiency given for the three stations, it could be seen that both examinees and examiners felt that students were mostly running out of time in accomplishing the stations. This finding is similar to several surveys conducted during the normal offline OSCE in pharmacy and other health professional schools where mostly students perceive an inadequate or unbalanced time allocation for OSCE stations [1] [20][21] [22]. The allocated ten minutes was mentioned inadequate mostly for the medication reconciliation station where examinees were instructed to gather a complete medication history from the patient as well as analyzing for drug related problems. Time allocation for entry-to-practice OSCE are usually in the range of five to ten minutes, whereas the time allocation for stations in the national OSCE examination in Indonesia are strictly determined for a maximum of ten minutes [7]. Therefore, evaluation and follow up should be conducted for the medication reconciliation process in the institutional OSCE. Modification could be done for the number of competency areas that should be achieved by the students in order to complete the station in the ten minute period. Time insufficiency was also mentioned as the effect of internet instability which made the examinees having to repeat certain tasks from the instruction.
The last evaluation for the online trial was evaluation of how the examinees and examiners understood the instruction given before the exam. Documents were prepared beforehand which consisted of instructions for preparing the exam, procedure during the exam as well as the scoring checklist used to assess students skills for each competency. Discussion was opened before the examination day to ensure that all students and examiners understood the instructions given. Table 2 shows the level of understanding towards the instruction given to the examinees and examiners. Based on the result, all examiners were clear enough with the instruction given to them. There was no difficulty in understanding the procedure and applying them during the trial. Several examinees showed less understanding towards the instruction given that may have affected their performance during the exam. This could therefore become a disadvantage for the particular students in obtaining maximum score for their exam. It is necessary that all instructions are understood by the whole component ot the OSCE exam before beginning. A simple questionnaire could be given to the examinees and examiner before the exam to picture the understanding towards instruction given. Follow up for those who still have difficulty must be conducted to ensure the same understanding by all components and avoid any non-essential matter due to the incomplete understanding towards the instruction given.

CONCLUSION
Overall, the online OSCE trial was applicable to be used in the future with several modifications based on the evaluation given by both examinees and examiners. Although offline method will still be the best way to conduct OSCE in pharmacy schools, the online OSCE could be a promising alternative to assess, particularly, clinical pharmacy skills of the students such as medication reconciliation, ADR monitoring, as well as patient counselling during the COVID-19 pandemic or other similar situation in the future that restrain physical interaction.

ACKNOWLEDGEMENT
The authors would like to thank all components of the trial who have participated in developing and evaluating the online OSCE together with the Pharmacist Professional Program of UMY.