Where Do Medical Students Look for Information? A Study on Scientific Consultation Sources in Peru

1 School of Medicine, Faculties of Health Science, Universidad Privada Norbert Wiener, Lima, PERU 2 Department of Pathology, Hospital Nacional Docente Madre Niño San Bartolomé, Lima, PERU 3 Education Unit, NESH Hubbs, Lima, PERU 4 South America Center for Education and Research in Public Health, Universidad Norbert Wiener, Lima, PERU *Corresponding Author: jeel.moya@gmail.com **Corresponding Author: hans.contreras@uwiener.edu.pe


INTRODUCTION
Medical training requires a series of processes and procedures that are different from other professions and university students. From the selection of applicants to admissions in medical schools, the curricular development in the undergraduate, postgraduate and medical specialty, to their intrinsically human and interpersonal work activity. Medical education is based on scientific evidence, in the clinical-practical and ethics processes, which must be fundamental pieces in all these stages [1,2].
Medical students face a heavy workload, which exceeds the workload of other university majors. These jobs include the overwhelming load of knowledge, different learning styles, and the impact of knowledge on their milieu [3,4]. Nowadays, with the advent of the technological age and the imperishableness of the imago, the scientific search by medical professionals is carried out in computerized information systems that have improved the quality and efficiency of available information [5].
Virtual search provides several opportunities that enhance medical knowledge and allow it to be continuously updated by reading scientific articles, digital books, and video conferences, being a vehicle also for involvement in student research [6,7]. However, the massive search and improper use of high-quality computing resources often deviate from the goal of browsing the Internet.
In the student community, the source of consultation may vary and affect the quality of medical training and access information [8,9]. Knowing where to find high-quality information is crucial. In addition to the bibliographic resources that students can often get and use for medical education, the consulting resources visited are also relevant. It must be considered an essential axis in the digital framework in the current reform of continuing medical education [10].
In Peru, the medical school of the Universidad Nacional Mayor de San Marcos (UNMSM) was the first in medical education, followed by the Universidad Nacional Federico Villareal and the Universidad Peruana Cayetano Heredia. Currently, around 40 public and private universities have medical schools, 5 of them meet the educational quality guidelines according to the National Superintendence of Higher Education (SUNEDU) and have the license to operate [11]. One of the criteria for licensing is that within the academic proposal of each university, the curriculum provides medical skills and knowledge.
To meet these requirements, students must have solid training through a mosaic of clinical practice-theoretical training-research. To achieve these indicators in medical students, the use of reliable medical information is necessary. Since the most frequent users of electronic information are medical students [12] and the majority of them fully trust what they find [13], the use of more rigorous sources in academia should be encouraged. Like in other low-and-middle-income countries, in Peru, there are barriers to accessing high-quality information during medical training and poor knowledge of reliable and quality sources of information.
Regarding medical schools, Norbert Wiener University (UNW) is one of the newest in Peru, and its student profile is mixed. On the one hand, it has recent high school graduates and students with previous careers in the same programme. In addition, the curriculum is innovative and incorporates courses in social sciences, administrative sciences and public health alongside the specialised medical courses. On the other hand, pre-professional internships and access to patients start in the second year (IV cycle). Also, it has incorporated virtual educational environments for anatomical, physiological and emergency care simulations.
We aimed to determine the most frequently used sources of scientific consultation in Peruvian medical students of the UNW. Also, we highlighted the role of scientific search engines and their implication in the process of contemporary medical training and learning. The importance of delving into these access components and sources of medical information lies in disaggregating the preferences of the students each semester in order to know where they focus their searches and which search engine options are most frequent during their medical training.

Study Design and Setting
This was a survey-based observational study developed at the UNW School of Medicine. This medical school was created in 2015 and presents a seven-year program that includes basic and specialty courses. Furthermore, this program follows a German model that integrates the humanities, virtual reality and clinical theory-practice.

Participants and Inclusion Criteria
We conducted a randomized sampling at the School of Medicine. The inclusion criteria were: i) first-time career and second-career students, ii) Peruvian students, iii) students studying the III and VI cycles (2nd and 3rd medical year). Each cycle corresponds to one semester of study (two semesters per medical year). Students were divided into cycles because, according to the UNW curriculum, students move from basic courses to clinical practice (i.e., semiology and internal medicine courses) and speciality courses (i.e., microbiology, immunology, and epidemiology) during the 2nd and 3rd medical years [6,14]. At the same time, students need to improve the sources of consultation and the thoroughness of the information reviewed.
In this study, we also included working students. Although UNW has a programme for working adults, the medical school admits both new and continuing students to the same programme. For this purpose, it offers validation of previous careers (nursing, dentistry, and Medical Technology) and agreements with Peruvian institutions (such as firefighters and the national police).
All the students enrolled in this study have taken the course "Digital Strategies for Learning" in the first year. In this course, students will understand what the science engine is, how to research and how to read scientific materials [14]. The background of the use of the information source was academic, mainly in the preparation of coursework (70%), exams (7%), study materials (19%), and other (4%), such as problem-solving and guidance.

Data Sources and Measurement
The three-part questionnaire "Student Sources of Scientific Consultation -USSC19" was applied during the last weeks of the 2019-I semester. This questionnaire was developed in 2019 to assess the information access characteristics of students, as well as part of the first academic census of UNW students conducted in two simultaneous research projects [14]. The survey has 21 items (19 questions have been closed), divided into three parts: 1. Demographic data (10 items) 2. Scientific consultation sources -Scientific, audiovisual, and general search sources (7 items)

Information search sources (4 items)
Since this was the first study conducted at UNW's medical school, we initially deepened the understanding of the demographic characteristics of the students, which explained most of the questions on this topic in the first part of the USSC19. The second part included three components: i) scientific source articles, specialized scientific books, and Medscape; ii) Audiovisual sources for documentaries on the history of medicine and YouTube and iii) General search sources as newspapers and Wikipedia. The scientific source referred directly to the primary evidence available to students, while the audiovisual sources describe the platform where information is. Although the latter can communicate research results or products on the subject, they can also have arbitrary material, in-home videos, etc. Therefore, we do not regard it as strictly scientific material. USSC19 has good internal consistency (Cronbach's alpha=0.87), internally validated by UNW's Research Committee experts, and the response time of the instrument was ~10 minutes per participant.

Data Analysis
A code was assigned to each survey included in EPIDAT-info v2.0 (Xunta de Galicia, Spain). Two reviewers independently verified the tabulation, and data analysis was performed with descriptive statistics. Fisher's exact test and one-way ANOVA with Bonferroni post-hoc test were used to determine the differences between and within groups. A simple linear regression was performed to find correlations between the study variables. All tests were analyzed considering a p-value<.05 and a 95% confidence interval as significant [15]. Statistical analysis was performed in IBM SPSS v25.0 (Armonk, US) for Windows.

Ethical Aspects
This study was undertaken following the Declaration of Helsinki and has been approved by the University Ethics Committee (No. UNW-VRI-018-012). The students voluntarily agreed to participate in the study through informed consent.

Demographic Data
The average age of the 148 students was 22.1±5.4 years (95% CI 21.2 to 22.9), of which 157 (38.5%) students were less than 20 years old. The mean age of the 101 (68.2%) females (22.06) 5.4 years) was significantly different from the 47 (31.8%) males (22.2±5.38 years) (p=0.041). As a voluntary survey, some students did not answer some questions, and we considered a total of 138 for the global characteristics ( Table 1). Ninety-four percent (140/148) of students have reported good internet access, and others have experienced home connection problems at some point in the last six months. The time among secondary school studies, the beginning of medical enrollment and the start of university education was three years (58.7%) in most cases. In addition, 37 (26.8%) students worked, of which 51.4% (19 students) were in the health field and worked less than 20 hours per week.

Scientific Consultation Sources
The results of the second part of USSC19 are shown in Table 2. As scientific search sources, the scientific articles and specialist books were considered useful or very useful in 75.4% and 49.3%, respectively, while ~32.8% were unaware of Medscape. As for the audiovisual sources that included documentaries and YouTube, both were widely considered as useful (in 41% and 48%, respectively) or very useful (31% and 32.4%, respectively). Regarding general search sources, 6/10 (63.9%) and 7/10 (71%) students considered Wikipedia and newspapers as little or no useful, respectively. We found differences in the use of consultation sources (p=0.031), Medscape (p=0.001), documentaries (p=0.009), and YouTube (p=0.022) among medical years.

Information Search Engines
The frequency of use of search engines proves the heterogeneity among students (Figure 1). Students use Google very frequently, with an average of 58.3%. Google Scholar was the second-most frequently queried information source among students in the second medical year (III and IV cycle), and SciELO represented the second-most frequently queried information source among students in the third medical year (V and VI cycle). We found a correlation between the year of  Google Scholar showed a slight increase as a very frequent search engine from 31% to 43.9% among the second and third medical-year students, respectively. PubMed has little infrequent for 37% of third-cycle students (second medical year) and 53.7% of sixth-cycle students (third medical year) (p=0.009). Regression analysis also showed a significant correlation between gender and the use of Google Scholar (p=0.013) and SciELO (p=0.017).

DISCUSSION
Our results show that UNW medical students often use scientific articles and audiovisual resources (such as documentaries and YouTube) as scientific search sources. In addition, Google was the most widely used source of scientific advice for all medical students; and Scielo and Google Scholar were the second-largest sources of information used by thirdyear and second-year medical students, respectively. Previously, a cross-sectional study among Peruvian medical students from Piura, Cusco, Arequipa, and Lima showed that the most consulted sources of information were university libraries (33%), SciELO (26%), and PubMed (22%) [8]. While for the students of the UNMSM in Lima, the most used information engines were SciELO (72.3%), PubMed (71%), and Google Scholar (91%) [9]. On the other hand, among Paraguayan students, the most frequent used sources were Google (82%), SciELO (56%) and PubMed (31%) [16]. Our findings placed Google, SciELO, and Google Scholar as the principal search sources in all academic years, coinciding with these studies. However, the study results differ from previous studies [8,9,16,17] in the use of PubMed since, on average, 19% of students preferred it.
Our findings deeply showed the opposite, since the longer the PubMed career continued, less frequent it became to the point that 5/10 students considered it not at all or not very useful in the third year (sixth cycle). 30% of the students considered SciELO not very useful. Conversely, this search engine showed an increase between 2nd and 3rds medical years, being very useful on average for 39% of students. The reasons for this trend could be the habit of using a scientific search engine (which presents biases in the information), the custom of checking unreliable sources, and the use of massive social networks (i.e., WhatsApp), where little information is shared precisely. Also, the uncommon use of these scientific search engines could explain the poor performance of UNW medical students in reading scientific articles in digital and written formats [18].
These results highlight the heterogeneity between UNW students against national and international university communities regarding the link with scientific search engines [19]. These links to scientific engines may change as the career progresses, according to medical training goals. In Chinese medical students, an increase in learning demands is associated with a higher academic level of students (from 28.7% for undergraduate students to 73.5% for doctoral students) [20]. Also, it has been seen in postgraduate students [16] that the most frequent search engines were SciELO (36.4%), PubMed (34.4%), and HINARI (25.5%), medical professionals preferred PubMed, Cochrane Library, and UpToDate as serious scientific sources in ~ 60%, 14% and ~14% of cases, respectively [21,22]. These changes show that the information becomes more rigorous as medical professionalization occurs due to the demanding degree work that requires a more robust scientific search. Further prospective cohort studies are needed to understand to what extent they could change.
Contrasting previous studies in Peru [8,9,23] we determined, for the first time, a limited usage and knowledge about Medscape (Figure 1). Since several medical students have difficulty with English (~70%), the low affinity of this great tool may be related to language limitations [6,14,16,23]. Another aspect that may affect students' learning is the presentation format (type of scientific article) in Medscape, which is slight known and readable by students [18]. Further studies are required to clarify the causes and effects of professional medical platforms on medical students.
As in this study, in a cohort of medical students in Japan, research articles, e-books, online courses, and videos have been identified as the preferred sources of information [19]. Regarding audiovisual sources, we found that documentaries and YouTube were helpful in all medical years, with gradual increases according to the semester. Less than 1% were unaware of YouTube ( Table 1). YouTube's potential rests on its ability to provide multimedia education. Although a direct link with human medicine lessons is not established, we found that 27% of videos have been convenient for teaching anatomy [24], and 47% for teaching clinical examination of the nervous system [25]. This tool could complement traditional medical education and promote the students' approach towards complex specialties such as neurology or surgery, where is an urgent need to explain the living human being as an organized activity [26,27]. Further studies must evaluate the scope and limitations of this free access platform based on the quality of information (since several videos have errors in the content) and its usefulness in medical teaching.
On the other hand, other platforms such as Wikipedia, Facebook, Twitter, and blogs have potential medical communication content with extensive spread and free access [28]. Recently, a randomized trial in Canadian medical students demonstrated better performance in knowledge acquisition with Wikipedia compared to UpToDate and digital textbooks [29]. In this study, about a third of students considered Wikipedia to be of little or no use. It is necessary to carry out feedback reports from medical students to improve this platform since undergraduate medicine courses are widely consulted [30].
The advantages of technology and the use of digital tools, such as podcasting [31] and video tutorials in medical training, have demonstrated the establishment of the technological age in selected undergraduate topics [32]. In this context, medical schools must understand the reasons for virtualization, technocratization, and cybernetics in the medical field and daily medical practice. Establishing digital networks with health professionals from other countries should be taken into consideration in the collaborative environment [33] to improve student learning and their participation in Science and Technology research networks, as well as the development of innovation and patents (R+D+i).
The higher education reform by SUNEDU must be developed with a digital outlook. The evolution of medical education must be contemplated within a digital governance framework, as electronic medical information management tools allow for improved clinical and research activities [34]. These digital practices must dialogue with the medical curriculum, which, by changing the conventional paradigm centered on the body and the corpse, must enhance the person's understanding using digital means. To this end, during the 4.0 technological revolution, it is necessary to adapt the student's tools for learning and strengthen the sources of information they prefer. From improving the usage of mobile devices and learning apps [35] to incorporating tools that enhance computer skills [36], medical training universities must extend their quality practices to access quality, rigorous, open-access, multi-platform information.
This study had limitations: 1. At the discretion of the authors, they are considered the most frequent sources of scientific consultation, however, it is necessary to develop future studies considering the new free-access educational platforms, as well as the new forms of communication [37] among medical students; 2. consultation with teachers, classmates, blogs, and chat groups have not been considered as a source of information; 3. the results show the use of scientific search engines in the UNW medical school, however, it is necessary to carry out prospective multicenter evaluations to understand the national-wide trends during medical undergraduate; and 4. there may be differences between the assessed population and students from high-income countries, where there are a large number of tools, materials, and resources corresponding to the academy of each country.
In conclusion, our results suggest that for medical students, the most frequently used scientific search engines were Google and SciELO; the most frequented query sources were scientific articles, books, documentaries, and YouTube. In addition, we found that with the advancement of medical studies over the years, students less use rigorous scientific consulting resources.
Further comprehensive studies should be carried out among student populations to understand which search tools are consulted and what type of content they access. Undergraduate medicine programs can allocate elective courses for the correct learning of the most effective and accessible methods to achieve the right medical training. With technology, medicine can improve the comprehension of health and change the paradigm of medical care that focuses on attending and treating.