Development of a New Innovative Teaching Method Using a Comprehensive Approach: A Survey of Students’ Opinion

Comprehensive teaching methods have been evaluated by many health professional colleges and universities for a number of years [1-6]. These methods include the flipped classroom, discussionbased learning, web-based learning, case-based learning, interactive team-based learning, direct lecturing, student presentations, daily quizzes and multiple homework assignments [1-6]. The techniques stated above involve both instructor-centered learning and studentcentered learning. More higher learning institutions are moving towards student centered learning and away from traditional lecture-based learning to help students improve learner retention, promotes critical thinking skills and encourages lifelong learning [7-15]. There is little information in literature that has assessed the combination of all the teaching methods that uses a comprehensive approach to teach drug information. Current pedagogical methods dictate that most teaching methods incorporate active learning. Nevertheless, this doesn’t take into account that the learning styles of different students could confer benefit from a traditional method of teaching. Therefore, it was imperative that we included both instructor-centered and learner-centered opportunities to engage all the learners taking this course. Although these are some of the teaching styles described in the literature, which method of teaching style is the best approach to deliver a drug information course for pharmacy students was not solidified. Different students learn differently and one teaching style does not fit all. As a result, the goal of this study was to develop comprehensive teaching methods that incorporated the various teaching tools that captured an assortment of learning styles and identify its effectiveness using student evaluations.


Introduction
Comprehensive teaching methods have been evaluated by many health professional colleges and universities for a number of years [1][2][3][4][5][6]. These methods include the flipped classroom, discussionbased learning, web-based learning, case-based learning, interactive team-based learning, direct lecturing, student presentations, daily quizzes and multiple homework assignments [1][2][3][4][5][6]. The techniques stated above involve both instructor-centered learning and studentcentered learning. More higher learning institutions are moving towards student centered learning and away from traditional lecture-based learning to help students improve learner retention, promotes critical thinking skills and encourages lifelong learning [7][8][9][10][11][12][13][14][15]. There is little information in literature that has assessed the combination of all the teaching methods that uses a comprehensive approach to teach drug information. Current pedagogical methods dictate that most teaching methods incorporate active learning. Nevertheless, this doesn't take into account that the learning styles of different students could confer benefit from a traditional method of teaching. Therefore, it was imperative that we included both instructor-centered and learner-centered opportunities to engage all the learners taking this course. Although these are some of the teaching styles described in the literature, which method of teaching style is the best approach to deliver a drug information course for pharmacy students was not solidified. Different students learn differently and one teaching style does not fit all. As a result, the goal of this study was to develop comprehensive teaching methods that incorporated the various teaching tools that captured an assortment of learning styles and identify its effectiveness using student evaluations.

Methods
Drug Informatics is a mandatory 3 credit hour course offered once a week to the first-year professional pharmacy students in our College. During the course, the student is expected to acquire the knowledge and skills related to responding to drug information inquiries, conducting medication use evaluations and participating in medication quality assurance programs, such as; monitoring adverse drug reactions, drug and herbal product interactions, and medication errors. Students are trained to develop the skills to obtain information from various literature and reference sources to answer drug information questions efficiently. Techniques for researching and evaluating drug literature are covered in depth. Emphasis is placed on systemic approaches to formulate responses utilizing both verbal and written communication skills.
The objectives of the course as stated in the course syllabus are as follows: a.
Describe the role of drug information in the profession of pharmacy b.
Perceive and interpret the medication information needs of patients, families/care providers, and healthcare professionals c.
Given a drug information request, determine the appropriate background information needed to assist in formulating precise and accurate answers d.
Use appropriate primary, secondary and tertiary reference sources in arriving at answers to information requests e.
Based on a systematic search, interpret, apply, and communicate the requested information to the requestor The course is coordinated by one faculty member and team taught. The course has four major components that introduce students to the various aspects of principles and practice of drug information. The first part deals with the most common types of drug and medical information resources and their search strategies. It includes, Google Scholar, PubMed, Ovid, Science Direct, Clinical Keys, Micro medex, Clinical Pharmacology, Medscape, and other similar search resources. The second portion of the course deals with literature evaluation and statistical methods. The third portion deals with principles and practices of patient counseling with a focus on the top 300 drugs. The fourth section deals primarily on writing assignments including drug information consult, monograph preparation, adverse event reports, and drug interaction reports.
The course uses comprehensive teaching methods described in the introduction section of this paper to deliver the course material. In this course, several software and online resources are incorporated. These include e-learning platforms namely the Blackboard Learning Software for course development [16]; ExamSoft® for assessment of the written exams [17] and Tegrity®, a lecture capturing software for all recorded presentations [18]. Students have access to a comprehensive pharmacy database of textbooks and references through the use of Access Pharmacy® [19]. Students are given an introductory training on how to access and use the software during the orientation in their first week of pharmacy school. Students also are given tutorials on the various teaching methods used in the course and the expectation of the course and course objectives. At the beginning of the first class, they were required to form a small group of 5-7 which stayed the same throughout the course for all team-based learning activities.
Most of the classes started with a quiz on the content taught the previous lecture. It is then followed by a 20-30 minute traditional lecture format that is interactive. Then, based on the topic of the day, several in class and outside class activities are given. These activities include participation in small group assignments, review case studies and individual and group assignments to work at home. As a flipped class, students are also required to read articles or view videos to develop search skills of the various database before coming to class. Overall, there are over 25 quizzes and other assorted activities given in and outside the classroom associated throughout the course. There was a myriad of teaching methods that were incorporated at various times throughout the course. The course evaluation outcome was extracted from the College's end of year overall course evaluation.

Evaluation Survey Development
An in-house course evaluation was developed by our assessment expert who is a full-time employee of the College. The course evaluation was constructed through the Qualtrics survey platform. Of the 27 questions posed to respondents: 26 used a sliding rating scales ranging between 0.0 and 5.0,one question was measured from 0-50, question were free text entry boxes. No logicbased coding was needed in the implementation of this evaluation. An inclusive questionnaire was used to elicit general opinions of the students on different variables to address the study's objectives. The questionnaire included different opinion variables that the authors believed had the most potential to reflect pharmacy students' perception on the adopted and preferred methods of instruction that were designed.
The survey consisted of six parts that were collectively composed of 26 questions. The first part (1 question) dealt with students' giving their overall impression of the course rated on a five-point scale (0.0-5.0). The second part of the questionnaire (12 questions) dealt with the methods of instruction separated by how each instructor performed regarding each measure. The third part (4 questions) was composed of questions regarding the delivery of the course and the student's impression of how effectively it was presented. The fourth part (6 questions) asked students to selfassess their own effort to critically think and assess their knowledge or skills they acquired throughout the course. The fifth part (2 questions) dealt with students' overall satisfaction of the course. The Sixth part has two questions and deals with attendance. The last question consisted of open-ended questions asking students to identify the strengths and weaknesses of the course as well as any advice they would give to an incoming student. The survey tool was tested for internal consistency. This is the first implementation of this survey and we have not piloted it before release.

Results
A total of 80 students completed the evaluation which represents a 92% response rate. There were 13 questions on the effectiveness of the faculty and/or the facilitators participated in the course. Students were asked as shown in (Table 1) on several aspects of the faculty performances. They were asked about their preparedness, showing up for lecture and class activities on time, effectiveness of the teaching materials or techniques they used, and their helpfulness in answering questions or explaining the course material. A Likert Scale is used to rate participants response (with 5 for "Strongly Agree" and 1 for "Strongly Disagree"). The students overall average score for the course was 4.28 (STD 0.66). Although all the questions were rated very high, the ones that scored relatively higher are question #3, #5, and #11. Question #3 refers to the preparedness and attendance of the instructor and starting the class on time. Question #5 refers to the instructor presenting content in an organized manner. Question #11 refers to the instructors 'ability to be critically engaged with the material. Questions #1 and #4 have the least variance comparing to the others. These are the questions that most of the students agreed the most in rating. Although the questions are answered overwhelming by major of students, question #9 and #10 had the least number of students responded.
As shown across all the questions, students are happier with the course delivery and the faculty members involved in teaching the course. The next set of questions was about students learning and what component of the course that helped them learn the best. The question that is scored the highest in this category is question #3 which refers to the course assignments, lectures, and class activities. The question with the least variance is question #3 which refers to the usefulness of the various activities in the course in complementing each other. Overall, students rated this category of questions also very high with the lowest score being greater than 4.3/5.0 (SD = 1.0). Category 3 of the survey questions is shown in (Tables 2 & 3). The participating students believed that this course helped them in their ability to think critically, synthesize fundamental knowledge, and obtained problem-solving skills which are key components in becoming a successful pharmacist. Question (Table 3) had the least variance and also scored the highest.  The students believed that this type of approach allowed them to synthesize fundamental knowledge and skills through literature search and literature evaluation. Based on these results, the teaching methods used in the delivery of this course seemed to help students achieve the overall stated objectives. Students, in the 4th category of questions, asked how satisfied they were in the overall course and its delivery system. The average score for this category of questions was 4.41 (STD = 0.73). When they asked how would they rate the course despite any limitations they encountered during the course, the average score was 4.44 (STD = 0.62) and it had the least variance which reflected an agreement among the majority of students.
As shown in (Tables 4 & 5), students on average attended almost 23 class sessions out of 25 sessions although each class is 3 hours long. This represents over 90 percent attendance rate. In an environment where all the lecture material and live lectures is recorded and available online, such a high attendance rate reflects the effectiveness of the active learning component of the course. When students get interested and they think they learn, they tend to show up for class and showed up for activities more often. In addition to attending class, students on average spent over 16 hours outside the classroom engaging in activities related to the course. Despite hours of work required to be completed outside the classroom, students gave the course over 4.0/5.0 when they asked how satisfied they were with the final grade they obtained.

Discussion
Based on the evaluation, students scored the course above 4.4 on a Likert scale where 5 represents "Strongly Agree" and 1 represents "Strongly Disagree". Although students were not asked specifically which specific type of teaching style they prefer, their overall rating shows their overall satisfaction. In this course, a comprehensive teaching method was used in combination of the various aspects of the course. The working Definitions of Terminology and Platforms Utilized in the Classroom are discussed below.

a.
Flipped Classroom: is a teaching technique where the customary teaching environment is switched around by delivering instructional material, frequently online, outside of the classroom and before the actual classroom lecture. Then, it shifts activities like homework into the classroom. The phrase "flipped classroom" was founded in 2007 and has been swiftly gaining popularity [1].

b.
Discussion-Based Learning: Discussion-based learning is an educational method that concentrates on students gaining knowledge and skills through conversation and dialogue rather than non-participative strategies that focus on lectures, reading, or observations. It is typically used with other tactics including case studies [2].

c.
Web-Based Learning or E-Learning: Web-based training (occasionally called e-learning) is instructions delivered anywhere and at anytime over the Internet or a corporate network to browser-equipped students. There are two main models of Web-based instruction: synchronous (instructorassisted) and asynchronous (self-directed, self-paced) [3].

d.
Case-Based Learning: Case-based learning is a teaching strategy that uses a case-based method that lets students participates in discussion of detailed scenarios that look like or typically are real-world examples. This technique is learnerfocused and can be done in individually, in a small group, or as a class. Nevertheless, in the majority of situations case-based learning is mostly done in a small group where individuals work together as a group to analyze a case to answer problem sets allocated related to the case. Case-based is more beneficial when it is done with a facilitator to assist the students through the case [4].

e.
Team-Based Learning (TBL): Team-based learning is an organized form of small-group learning that enhances student development out of class and implementation of knowledge in class. Students are structured strategically into different teams of 5-7 students that will work together throughout the class. They are given an assignment to aid discussion of the case study or additional aspects of content. This strategy requires prior preparation and content assimilation through the class by the educator [5].

f.
Direct lecturing: Refers to the oldest style of teaching where the facilitator provides information in a lecture format using other teaching tool such as PowerPoint presentations.
ExamSoft® is predominantly utilized to administer quizzes, tests, and examinations [17] It is an exclusively owned educational software company based in Dallas, Texas and Delray Beach, Florida. The software system is used at more than 1,300 academic establishments worldwide and used as a platform to administer examinations in a protected setting [6]. In this class, the majority of the tests, quizzes, and examinations are administered using this program. One of the essential elements of the program is that it stops the computer from accessing any other data on the participants' computer. Once the assessment is downloaded onto the computer, it also does not need an active internet connection while the examination is being taken. The finished examination will stay on the persons' computer until it gets an internet connection to upload the results.
Tegrity® software is utilized to record each and every presentation given by the lecturer in this course [18] Tegrity is a software that captures lectures and records the information shown on the instructor's computer screen, for instance, PowerPoint presentations alongside with the audio and (voluntary) video. Tegrity captures, saves and itemizes the content online so the students can access this information at their convenience. The saved material is linked to the course website on the Blackboard Learning System and becomes attainable for students to watch the lecture from start to finish during their spare time. An elected student from each class is delegated to coordinate and assist the professor to set up and record each lecture and class activities using this software. Blackboard Learning System: is an online learning domain and course management system invented by Blackboard Inc [16] This program is operated as a central web tool to provide content, resources and assignments. This software is also utilized in the class to post grades and any comments for all students related to their performance and progress.
Access Pharmacy® from McGraw-Hill Medical is operated as one of the main resources of e-books and virtual information that can be accessed in this course [19] It is an all-inclusive, content-rich online pharmacy support that allows users to review leading pharmacy references, look for curriculum subjects and read up on drugs and supplements. It also gives pharmacy students immediate access to videos and cutting edge pharmacy textbooks to get information on drugs, supplements, herbs, consult features and editorials relating to the boundless array of prevalent pharmacy publications is also included.
The information that students receive from this course in the first year is then reinforced during their second year in a pharmaceutical care skills course and integrated as an assessment during their objective structured clinical exam (OSCE) while in their third year. The fourth-year students are evaluated on their ability to comprehensively evaluate literature in a journal club format during their advanced pharmacy professional experiential (APPE) program. McCabe and colleagues compared flipped model to traditional classroom learning in a professional pharmacy school setting and they found a significant difference in knowledge retention with students in the traditional course scoring higher than students using flipped approach in the experiential course [20] MacDougall G. published the results of the research done to compare a new teaching method that was used in combination with active learning to teach antimicrobial spectrum activity course among 360 students. The research concluded that an active-learning approach lead to modestly higher scores on exams that tested the students' long-term retention of pharmacology knowledge and was well receivied [21].
Wu and colleagues compared mini lectures and active learning given online in delivering a drug literature and evaluation course to pharmacy students. The course was given to 90+ students in a flipped class format. The lecture focused on effective literature skills. The authors who are librarians by profession were encouraged by the outcome and supported active learning in their settings [22]. Other similar search was conducted and most of them reported the results of one method against another. However, we could not find a recent publication comparing the various teaching methods that was incorporated in a course. In that sense, our study is the first one to report the efficacy of a comprehensive approach as a teaching method that was utilized to help the facilitator teach the drug information course. The P1 students that took this course recently expressed gratitude and confidence towards their research abilities specifically because of their experience in this course. The array of search engines that were exposed during the class she felt optimistic that through learning about Micromedex and Clinical Pharmacology, they were able to do clinical research and other extra-curricular activities more effectively in the future in addition to getting ready for more strenuous classes that would require this invaluable knowledge.

Limitations
This is the first time that this mode of student evaluation has been used at the college. However, the teaching tool is not specifically designed for this course. The grading and evaluating system has also not been evaluated in comparison to other schools so we are not sure how reliable this method of evaluation is. There were a total of 27 questions in this survey and this could have been burdensome to students to complete therefore requiring an incentive to be provided. A side by side comparison of another well-established tool with the evaluation used in this study is recommended. An evaluation across the four-year program and for multiple years should be assessed to see whether students are continuing to develop critical thinking skills and retaining information for lifelong learning.

Conclusion
This study reviews the use of combined teaching methods and was found to have a high rating and appreciated by professional first year pharmacy students in our college. This combined technique is instituted in the Drug Information class which is a 3-credit hour mandatory course for first year students. The teaching methods that combined the traditional lecture format with a more advanced active learning including team work and flipped classes has a very high acceptance rate by students. Students learn differently and implementing various teaching methods, as shown in this study, had a high rating and long-lasting impact on their learning.