Google Forms for the Online Assessment of Osces Stations Volume 5 - Issue 1

Carmen Botella Martinez¹, Matilde Moreno Cascales², Miguel A Fernandez Villacanas² and Joaquin Garcia Estan*³

• ¹Centro de Estudios en Educacion Medica, Hospital Universitario Virgen de la Arrixaca, Spain
• ²Facultad de Medicina, Universidad de Murcia, Spain
• ³Edif. Departamental, Universidad de Murcia, Spain

Received: May 23, 2018;   Published: June 06, 2018

*Corresponding author: Joaquin Garcia Estan, Edif. Departamental, 3, 32, LAIB, Campus de Ciencias de la Salud, c/ Buenavista s/n, 30120 El Palmar (Murcia), Spain

DOI: 10.26717/BJSTR.2018.05.001163

Abstract PDF

In this article we show a simple and free method to carry out the entire OSCE evaluation process allowing the almost immediate recording of the results obtained by the examiners. The process is completely online, eliminating, therefore, the need to manually fill out checklists in paper and the subsequent analysis of all these results. The method begins with the preparation of these checklists that, once finished, will allow them to be filled online from laptops or tablets, even from smartphones. All this can be made fast with the use of the Google Forms system. In relation to the commercial systems already present in the market, we emphasize that the system we present stands out for its simplicity, immediacy and, most of all, free of charge.

For some years, the OSCE (Objective and Structured Clinical Evaluation) tests have been generalized in practically all the Faculties of Medicine of Spain [1]. This test has been selected by the National Conference of Deans of Faculties of Medicine of Spain (CND-Medicina) as a method of reliable evaluation of the clinical competence of its medical students [2]. As described by Van der Vleuten [3], the OSCE is an adequate instrument to evaluate the third competency level of the Miller pyramid [4], where the student must “demonstrate” how certain clinical competences are used in a simulated environment that guarantees patient safety. In essence, the OSCE is nothing more than an evaluation with standardized clinical cases in which students must demonstrate their practical competence in areas such as history taking, physical examination, clinical management or communication skills, among others [5-8].

One of the most tedious aspects of OSCE is the process of obtaining the qualification of the students examined. To do this, it is necessary to analyze and, usually, to record in a spreadsheet the contents of the evaluation sheets or checklists that the evaluators of each station have filled for each of the student evaluated. Imagine 100 students and 20 stations; these are 2,000 evaluation sheets to fill up and analyze. In order to improve this aspect, a large number of software packages and electronic applications have emerged that solve this problem, although most, if not all, are quite expensive, closed and do not allow to include the specificities implemented in each of our Schools [9]. In this document, we show a simple and free method to carry out the whole process of OSCE evaluation for the almost immediate obtention of the results attained by the examinees. The method also allows the whole process to be done online, thus eliminating the need to fill in evaluative lists on paper and, what is more tedious, the subsequent transcription of all these results. Logically, all the necessary process for the preparation of the cases or stations cannot be automated and it is the mission of the authors responsible for each station writing the cases. For the Spanish users, we recommend to follow the standard dictated by the CND-Medicine for the writing of the cases [2].

The method begins with the preparation of evaluative lists or check lists that, once finished, will allow them to be filled online from laptops or tablets, even from the current mobile phones or smartphones. To do this, we will use the Google Forms system (https://www.google.com/intl/es/forms/about/). The only requirement is that the person preparing the online forms from the evaluative listing of the case must have a Google account, with its Drive activated. Google accounts are easy to get, free and allow you to use up to 15 GB of storage in the so-called Drive, the Google cloud (if you need to create a Google account, copy the following link in the browser (https://support.Google.com/ accounts/ answer/27441?hl=en). For Microsoft users, there is a similar tool called Microsoft Forms (https://forms.office.com/). Our method consists of three procedures:

a) Creation of the evaluative listing form for each of the OSCE stations.

b) Completion of the form and obtention of the students’ answers in each of the stations.

c) Evaluation of the answers to obtain the partial scores of each station and the overall score of the OSCE.

a) Log in to Google.

b) In the upper right corner, click on the Google Apps icon and then the Drive icon (Figure 1).

Figure 1: Entering Google Drive.

c) Once in Drive, in the upper left corner, click New, then More, and finally Google Forms.

d) A new window will open with the form’s home page (Figure 2). The first thing we will do is to give the form a name. Suppose we are going to prepare the evaluative list for station number 1. After clicking on “Untitled Form”, we will write for example “Evaluative List of Station 1” or “ListEvalSta1” or as we want to call it.

Figure 2: First page of our Form.

e) Below the title line, there is another space in which we can, if we want, assign a small description of the document. It is useful to write a warning that we want the person using the form to keep in mind. For example, “Do not forget that you cannot comment on the student”.

f) The next thing we should put on our form is a question that identifies the examinee. Since the OSCE is done anonymously, students must have anmindividual code to show the evaluator. In “Question without title” we will write “OSCE Code #” and to the right we will select that the type of question is “Short answer”. A little further down we will select that this question is obligatory (Figure 2), logically. This way we make sure that no survey will be left unidentified. If you do not respond, you cannot continue.

g) We will now begin the preparation of the evaluative list itself. The ideal is to make sections according to the areas of competence that we evaluate. In our example, our list has anamnesis, physical examination and clinical management. We will begin, therefore, with section 1 of Anamnesis. To do this, click on the icon on the right “Add section” and write the name of the section, “Anamnesis”, in this case.

h) Now we will add the items to be evaluated, one by one (Figure 3). To add an item, click Add. Next to the title of the question, you have to choose the type of question you want to add. In our case, since we only want the evaluator to write down when the item coincides with a student’s question, we will select “Multiple-choice Grid” and in “Column 1” we will write “Yes”. If we wanted the evaluator to always write a response (Yes or No) on all items, we would write “No” in “Column 2”. We think it more useful, especially for later correction, to fill out only if the student asks that question. If it does not, it does not fill anything, it is left blank. Obviously, we could put more columns, like Good, Fair or Fail.

i) Now, in the left part of the form box we will begin to fill in the items we want, one in each row. We will also have to fill in the title of the question, for example a sentence that serves all items, but a question mark or a space would suffice. As we see in Figure 3, we have included 3 items in the rows and only one column, Yes.

Figure 3: Filling up items on the evaluation check-list.

j) It’s time to check how the form is coming out. Click on the eye icon “Preview” at the top of the screen and a new screen will open with the form view so far.

k) Once back to the previous tab, we will continue editing the document. We will create a new Section, which we will call “Physical Examination” and create the items we wish, in the same way as we have seen before (Figure 4).

Figure 4: Another section and items on the evaluation list.

l) Finally, we have created another section called “Clinical Management”, and introduced the items. We have practically finished the design of the evaluative listing.

m) It is also important that the presentation and appearance of the form be enjoyable. To do this, we go to “Palette of colors” and choose the color that we like for the form. If you want to add a theme or one of our own photos, click on Insert image, dragging a photo to the window or clicking on “Select a photo from your computer”.

n) At this moment, we must already have the complete form, with all its sections and its questions. If we detect any error or want to add or remove something, just click with the mouse in the area of interest and edit it.

o) In all this process, we did not have to save the document even once, since Google automatically saves it. We can check it out by going to the tab of the Drive, where we will verify the existence of our form.

p) The file is complete. It is now a good idea to check that the file is working properly and log the responses appropriately. The system allows us to test it directly or assign it to an evaluator or someone of confidence to verify it.

Filling Procedure and Obtention of Answers

Once the form is assigned to a particular person, for example the evaluator in charge of the station, the station can start receiving students, so that the evaluator will fill up a form for each of them. To do this, the answers must be enabled in the form and you have to assign the spreadsheet document where all the answers will be stored.

In the form window, click on “Answers”, where we will see that there are none yet and that the system is “Waiting for answers”. At the top of this window click on “More” and then on “Select answer destination”. Now we have to choose between creating a new spreadsheet and using an existing spreadsheet. In our case, as it is the first time, we will choose “Create a spreadsheet”, which would bear the same name of the form, with the added “(answers)”. Then click on “CREATE”. Once this is done, we can find the form answers spreadsheet on the Google Drive screen. It is convenient to review the form and appearance of the form.

Now we must verify that the configuration of the form conforms to what we want.

a) For example, we could limit users to a single response by clicking on the wheel (top right) and checking the box next to “Limit 1 response”, and then SAVE. Logically, this characteristic would be for those survey type forms, where we will only allow one response per person; it is not practical for the OSCE.

b) We could also allow you to edit the responses to those that filled them out. This could be useful in the event that an evaluator makes a mistake and wants to rectify any response. To do this, click on the wheel and check the box next to “Edit after sending”. Again, we do not advise this possibility, but the user has it at his disposal.

c) Similarly, when a person has finished filling out a form, you can see a link to the results. To do this, check the box next to “View a summary of the answers”.

d) In our experience, the ideal for the OSCE is to keep marked only “Show link to send another response”, in the “Presentation” tab of the Configuration button (sprocket). In this way, the evaluator or the person who fills out the forms always has an empty form ready to respond and send.

Now is the time to test the form, add a few answers and analyze the worksheet with them. There are several possibilities for this. We will comment on the most useful ones:

a) Send a form by email: in the upper right of the form window, if we click on Send, we will add the email addresses of those who will have access to the form, along with the subject of the mail and the message. Then press Send.

b) Get a link to a form: it is very useful to simply share the form by sending a web link to those interested. In the upper right, click on Send and in the upper area of the window, click on Link and then on Copy. The link can then be emailed, placed on a web page, and so on. This is probably the best system, since it will allow us to always have the same link to the form and, at the same time, modify the form when necessary without having to resend it every time to our contacts.

Once we have submitted the forms or links, we will start receiving answers, which will be stored according to your order of arrival in the spreadsheet that we had selected before. If we open the spreadsheet, we will check how responses have been recorded seamlessly, in consecutive rows, according to the day and time the messages arrived. In the columns of the spreadsheet, we will see the answers marked with “Yes” and those that have been left blank. You can also see a summary of the answers in the form itself, selecting at the top “Answers”.

We have now completed the form. We can now, if we have been doing tests before, remove all the answers to leave the spreadsheet blank. At the top of this window we click on “More” and choose “Delete all the answers”. Another option would be to prepare a new spreadsheet, which will have to be linked to the form as explained in point 2 above.

After completing the spreadsheet with the data obtained from each of the evaluation sheets sent by the different evaluators, we will proceed to obtain the scores, for which it is necessary to follow a series of steps, summarized here:

a) Create a backup.

b) Assign a value to each of the items.

c) Transform the answers (yes / no / well / bad) into numbers.

d) Add the scores of each competition.

e) Transfer the data of each station to a new global OSCE spreadsheet.

I. Once we open our spreadsheet, the first thing we should do is create a working copy to always keep the original file intact. To do this, click on “File” and then on “Create a copy”. In this file “Copy of List Evaluative of Station 1” is where we will work.

II. First, we will assign the nominal value to each of the items on the form. To do this, placed in row 1 and pressing with the right mouse button select “insert 1 over”. Now in box A1 we will write “Scores”, just above the “Timestamp” box. In column B we should have the student code and from column C we can insert the score of each of the items. In our case, in the section of Anamnesis, we have 10 items and each one of them is worth 5 points, for a total of 50 points throughout the Anamnesis. Then, we will begin to fill in row 1, above each item its score. We will do the same for the rest of the items (Figure 4).

III. Now we have to convert each of the “Yes” numbers in our spreadsheet into numbers. Too easy For example, suppose we have 60 rows of answers. We will place the cursor, for example, in row 65, column C, just in the column of the first item and add the formula that will change the text “Yes” to a number: “= IF (C3 =” Yes “; C$2; Where C3 is the cell of the first item of the first response and C$2 is the cell that contains the score of that first item. This formula simply says “if cell C3 has a” Yes “, type the number at C$2 or otherwise write zero.” Now we will copy the formula to the rest of columns and rows and immediately we will have the points of all the forms.

IV. Now what is important is probably to add up all the scores in each competition. Nothing easier on the spreadsheet we place in the last column, after the last item, we leave a blank column and add a formula to add the scores of all the columns that contain items of “Anamnesis”,”= SUM (C65: L65)”; we will do the same for the other two competitions.

V. This data must go to another spreadsheet where the data of all the stations are gathered. But this is already beyond the scope of this article. Interested readers are sure to figure out how to do it.

We have presented a simple, free online system to prepare the evaluative lists of an OSCE test. We have not been able to compare this system with those already present in the market, since the purchase of each and every one of the available systems is not within reach of our economy, but we would like to highlight several aspects in which the system that we have presented stands out. The first is the possibility of adapting to the peculiarities of each of our different types of evaluative lists. By playing around with the different possibilities of Google Forms we will find that most of what our colleagues want to do when preparing the evaluative lists of their cases or stations, can be done in a simple way. We believe that it is very difficult, if not impossible, that the commercially available systems of electronic OSCEs conform exactly to what the author of the station intends to design, although they certainly work well in a standard preparation. The second feature is immediacy, which is shared with commercial systems. The fact that the application is online makes the whole process take place in a very short space of time, so that obtaining the scores of the evaluated is practically instantaneous. Another important feature of our forms system is that it is independent of the computer platform that we use. Whether we are Windows, Apple or Linux users, the system will work on any computer and more importantly, it will also work on phones and tablets with iOS or Android. The only requirement is the internet connection.

Finally, we emphasize that the system proposed here is completely free. We know that in today’s online world, nothing is totally free and that the price to pay is some control over our privacy. But, we can protect ourselves. Thus, it is not necessary to include the names of the examinees in the Google application, since coding systems that preserve anonymity can be used. As mentioned, the inclusion of the evaluated code makes the whole process perfectly acceptable and opaque for Google. Already in our computer and once offline, we will relate the list with names and surnames of the examined ones with their test codes.

1. Conferencia Nacional de Decanos de Facultades de Medicina de Espana. Prueba Nacional De Habilidades ECOE-CNDFME (7-4-2016). Consultadoen web 27-1-2017.
2. Garcia-Estan J (2013) Prueba Nacional de Evaluación de Competencias Clinicas de la Conferencia Nacional de Decanos de Facultades de Medicina de Espana. FEM 16 (Supl 3): S59-S62.
3. Van der Vleuten CPM (2000) A paradigm shift in education: how to proceed with assessment? 9th International Ottawa Conference on Medical Education. Cape Town 28.
4. Miller G (1990) The Assessment of clinical skills/competence/ performance. Acad Med 65(9): s63-s67.
5. (2014) Guia para la evaluacion de la practica clinica en las facultades de medicina. Instrumentos de evaluacion e indicaciones de uso. Ed. Millan J Pales J. Rigual R Union Editorial 2014, Madrid.
6. Khan KZ, Ramachandran S, Gaunt K, Pushkar P (2013) The Objective Structured Clinical Examination (OSCE): AMEE Guide No. 81. Part I: an historical and theoretical perspective. Med Teach 35(9): e1437-1446.
7. Khan KZ, Gaunt K, Ramachandran S, Pushkar P (2013) The Objective Structured Clinical Examination (OSCE): AMEE Guide No. 81. Part II: organisation & administration. Med Teach 35(9): e1447-1463.
8. Rubiano EK, Ricarte JI, Juncosa S, Martinez JM (2007) Evaluacion de la competencia clinica de las facultades de medicina de Cataluna, 1994- 2006: evolucion de los formatos de examen hasta la evaluacion clinica objetiva y estructurada (ECOE). Med Clin (Barc) 129(20): 777-784.
9. R Lilley P, Patricio M Churchill Livingstone (2015) The Definitive Guide to the OSCE. The Objective Structured Clinical Examination as a performance assessment. (1st edn).