Development of the Health Literacy Scale for Protection Against COVID-19

Introduction: The aim of this cross-sectional methodological design research is to develop the Health Literacy Scale for Protect against COVID-19 and examine its validity and reliability. Methods: The population of the research consisted of participants aged above 18 on 15-30 January 2021 and the sample consisted of 547 individuals. As data collection tools, questions regarding sociodemographic characteristics, Turkey Health Literacy Scale (THLS) and Health Literacy Scale for Protect against COVID-19 to be developed by the researchers were used. The content, construct and criterion validity of the Health Literacy Scale for Protect against COVID-19 were examined. The reliability of the scale was evaluated using Cronbach’s alpha and item-total correlation. Results: Content validity was verified with Content Validity Index (CVI) .97. In the exploratory factor analysis (EFA) of the scale, a single-factor structure with an eigenvalue value greater than 1 and factor loads above 0.60 were found, which explains 70.639% of the total variance. In the confirmatory factor analysis of the scale, it was seen that the Goodness-of-Fit Indices values were at acceptable levels. Criterion validity was supported by a moderate positive correlation (r = .0569, p <0.001) between THLS and Health Literacy Scale for Protect against COVID-19 score. Based on Cronbach’s alpha and item-total correlation results, the item’s reliability was found to be at a satisfactory level. Conclusion: Health Literacy Scale for Protect against COVID-19 is a reliable and valid tool that can be used to evaluate the health literacy of society for protection from COVID-19.


INTRODUCTION
When the World Health Organization announced the coronavirus pandemic, it led to unexpected and drastic changes in daily life [1]. With the aim of educating people about the virus and how to prevent contracting or spreading it, COVID-19 related health communication has become widely accessible. The most useful material is delivered in a fairly comprehensible format that provides quick and plausible solutions, such as hand washing, maintaining physical distance, and knowing where to find the latest tips and advice. Unfortunately, there is also complex, conflicting, and inaccurate information [2]. So, in these challenging and unpredictable times, a good level of health literacy has never been of such vital significance [1].
There are many definitions of health literacy, one of which reads; "the degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions." Health literacy not only requires a complex group of skills including reading, listening, analytical thinking, and decision-making, but also the capacity to apply these to health conditions. Four dimensions of health literacy in the domain of healthcare are namely, accessing, understanding, evaluating and applying information [3].
Health literacy has a positive impact on one's health, and it's linked to better health outcomes. People who have a higher level of health literacy are usually better at managing their health than people who do not. Higher levels of health literacy within groups (whether a country or an organization) are linked to increased protection, better quality of life, fewer variations in health results, and a more stable and equal community. People who lack good health literacy are unable to distinguish between fact and fiction and are more likely to be influenced by unreliable facts [1]. It has been reported in both national and international studies that the health literacy of the society is inadequate [3][4][5][6][7][8][9][10][11]. It has also been reported that individuals with low health literacy are significantly more likely to postpone or quit care or report difficulty in finding a provider than individuals with adequate health literacy [12]. In a systematic review; low health literacy was consistently linked with more hospitalizations; greater use of emergency services; lower receipt of mammography screening and influenza vaccine; failure in following medications appropriately; failure in interpreting labels and health messages; and poorer overall health status and higher mortality rates among the elderly [13]. Low health literacy is also related to an increased risk of hospitalization and fatality in patients with heart failure [14].
In the world, COVID-19 outbreak has not only become a "pandemic" but also an "infodemic" [15,16], which could be defined as an epidemic of excessive and inaccurate information and news. Health literacy plays an important role in preventing and controlling diseases and pandemic [16][17][18]. Although vaccination is continuing with the purpose of controlling the pandemic, the health literacy of the society is expected to be at a good level for preventing COVID-19, since the most effective method for the time being is protection. However, due to the lack of a valid and reliable scale, the health literacy of the society regarding protection from COVID-19 is unknown.
When the scales used to assess health literacy in Turkey are examined, it was determined that the six scales used were eligible for assessing the general health literacy [8,[19][20][21][22][23][24] but not appropriate to evaluate the health literacy related to the pandemic. In international studies evaluating health literacy in the COVID-19 pandemic [25][26][27], it was observed that general health literacy scales were used, and that only one study developed a scale to evaluate the health literacy of the public regarding the COVID-19 pandemic [19]. However, no scale evaluating the health literacy for protection from COVID-19 could be reached. The aim of this research is to improve the health literacy scale for protection against COVID-19 and examine the validity and reliability of the scale.

Aim
The aim of this research is to develop the Health Literacy Scale for Protect against COVID-19 and to examine its validity and reliability.

Participants
The population of the research consisted of participants aged above 18 on 15-30 January 2021. In the literature, the sample size for factor analysis is regarded to be poor for 100 cases, fair for 200 cases, good for 300 cases, and very good for 500 cases [28] or it is suggested that the sample size be 5-10 times the number of items in the scale [29]. It was aimed to reach more than 500 participants and the sample of the study was finalized with 547 individuals who participated in the study via e-mail and were selected with the convenience sampling method.

Data Collection Tools
As data collection tools, questions regarding sociodemographic characteristics, Turkey Health Literacy Scale (THLS) and the Health Literacy Scale for Protect against COVID-19 developed by the researchers were used.

Methodology
The Health Literacy Scale for Protect against COVID-19 has been developed in three stages. In the first stage, scale items were created, in the second stage, the content, face and construct validity were examined within the validity study, and a reliability study was conducted in the third stage [30].

Generation of the Items of Health Literacy Scale for Protect against COVID-19
The conceptual structure of the scale is based on the definition of health literacy by the World Health Organization and Sorenson et al. Furthermore, the health literacy scales previously developed in Turkey or adapted to Turkish were examined [8,20,22,23,31]. Information on protection from COVID-19 has been obtained from official websites such as the World Health Organization and the Ministry of Health. In line with the conceptual structure of health literacy, the items were generated using the information obtained from the literature. A 22-item pool was generated covering the four processes of health literacy (accessing, understanding, evaluating and applying health related information) regarding social distance, wearing a mask, hand hygiene and physical health in protection from COVID-19 [3].

Validity
The validity of the scale was assessed with content validity, face validity, construct validity (exploratory factor analysis, confirmatory factor analysis) and criterion validity.

Content validity
The Davis technique was implemented to evaluate how well the items in the scale could represent the domains aimed to be measured [32]. A total of 12 experts who were academics in the field of internal medicine, fundamentals of nursing, and public health evaluated the content validity of the scale. Each expert was asked to evaluate the 22 items on the draft scale as to how appropriate they were in measuring health literacy for protection against COVID-19 using a four-point Likert-type scale (item is appropriate = 1, item should be slightly revised = 2, item should be substantially revised = 3, and item is not appropriate = 4). In this technique, the number of the experts who selected options "1" and "2" was divided by the total number of experts to obtain the content validity index (CVI) for the item. The CVI was expected to be greater than 0.80 in the study [32,33].

Face validity
In the literature, it is recommended that the draft scale, whose content validity was confirmed by obtaining expert opinions, be applied to a small sample (10-30 people) with similar characteristics and that the incomprehensible items be corrected [34].

Construct validity
It was evaluated by exploratory factor analysis (EFA) and confirmatory factor analysis (CFA).

Exploratory Factor Analysis (EFA):
It is recommended that EFA be conducted with Principal Components and Rotated Component Matrix technique in order to assess construct validity [28,29,[35][36][37][38]. Before performing EFA, it is recommended that the suitability of the data set for factor analysis be examined and that Kaiser-Meyer-Oklin (KMO) and Bartlett Sphericity tests be conducted. For the data set to be suitable for factor analysis, the KMO value should be higher than 0.50 and the Bartlett test of sphericity should be significant [29].
The conceptual structure of health literacy consists of four processes; accessing, understanding, evaluating, and applying health-related information [3,5,22,23]. In the EFA, it was expected that the Health Literacy Scale for Protect against COVID-19 would indicate a factor structure in accordance with health literacy processes and that the factor loadings of the items would be .40 and above [28].

Confirmatory Factor Analysis (CFA):
It is recommended that models revealed by exploratory factor analysis be verified with CFA. Fit indexes are used in the evaluation of CFA. Fit indices test how well the designed model corresponds to reality, thus revealing the construct validity of the model [39]. Several goodness-of-fit indices were examined in evaluating CFA. These were ꭕ2 /df, Comparative Fit Index (CFI), Normed Fit Index (NFI), Goodness-of-Fit-Index (GFI), standardized root mean square residual (SRMR), and root mean square error of approximation (RMSEA) [40]. For a good model fit, the ratio χ2 /df should be as small as possible. As there are no absolute standards, a ratio between 0 and 2 (0 ≤ χ2/df ≤ 2) is indicative of good data model fit whereas a ratio between 2 and 3 (2 < χ2 /df ≤ 3) is indicative of acceptable data-model fit. However, the problem of sample size dependency cannot be eliminated by this procedure. Other goodness-of-fit indices CFI > 0.97, NFI > 0.95, GFI> 0.95, SRMR < 0.05 and RMSEA < 0.05 indicate perfect fit, whereas CFI > 0.95, NFI > 0.90, GFI > 0.90, SRMR < 0.10 and RMSEA < 0.08 indicate acceptable fit [40].

Criterion validity
In order to evaluate the criterion validity of the new scale, the THLS which was based on the European Health Literacy Survey (HLS-EU) consortium (2012) was used [22]. For criterion validity, the correlation coefficient is expected to be strong values such as .70 -.80 in scales measuring the same conceptual structure. In terms of conceptual comparisons made with scales similar to the developed scale, correlation coefficients with a size of .50 -.70 showing a moderate relationship are considered to be a proof of validity, while values below .30 are not displayed since they are considered to be an expression of uncertainty [41,42].

Reliability
The reliability of the scale was examined with internal consistency method.

Internal consistency
In order to assess the internal consistency, Cronbach's alpha and item-total correlation was evaluated [37,42].
Cronbach's alpha: Cronbach's alpha is a good criterion indicating the homogeneity of the items supposed to measure a specific field. In the present study, for the total scale, a minimum Cronbach's alpha of 0.90 was regarded as ideal, but reliability coefficients of 0.80 were considered very acceptable and reliability coefficients > 0.70 were considered acceptable [43,44].
Item-total correlation: The corrected item-total score correlation coefficients of the developed scale are calculated in order to have an idea about whether the items in the scale measure the same quality in a better way and contribute to the conceptual structure [37,42]. Item-total correlation coefficient should not be negative and below .30. Acceptable corrected item-total correlations were those > 0.30 [42,45].

Turkey Health Literacy Scale (THLS)
The scale which was developed by Okyay et al. [23] consists of 32 structural items based on the HLS-EU Conceptual Framework. The scale, which has a matrix structure (2x4), consists of eight components which are two subdomains; treatment and service (1) and disease prevention / health promotion (2), and four processes (accessing, understanding, evaluating and applying health-related information). Items are rated on a 5-point Likert-type scale ranging from 1 (very easy) to 5 (I have no idea). Scale score ranging from 0 to 50 points is calculated with a formula. As a result of this calculation, 0 indicates the lowest health literacy while 50 indicates the highest health literacy. According to the score, the level of health literacy is categorized as follows; 0-25 points indicate inadequate health literacy, > 25-33 points indicate problematic/limited health literacy, > 33-42 points indicate sufficient health literacy, and > 42-50 points indicate excellent health literacy. The Cronbach's alpha coefficient used to evaluate the overall internal consistency of the scale was reported to be 0.927 [23] and was found to be 0.967 in this study.

Analysis
For the analysis of data, SPSS 22 (Statistical Package for Social Sciences Inc, IL, USA) was used. While evaluating the study data, the characteristics of the participants were defined by mean, frequency, percentage and standard deviation (SD). The content validity of the COVID-19 literacy scale was evaluated with the Davis technique and the CVI was calculated. The suitability of the data for factor analysis was examined using the Kaiser Meyer Olkin (KMO) Coefficient and Bartlett's test of sphericity. In order to evaluate the construct validity of the scale, exploratory factor analysis was performed using Principal Components and Rotated Component Matrix technique. The CFA of the structure obtained in the exploratory factor analysis was conducted with LISREL 8.50. Then, in order to assess the criterion validity, the relationship between the THLS and the In the study, a statistically significant moderate relationship was expected between THLS and Health Literacy Scale for Protect against COVID-19 was evaluated using Pearson's correlation coefficient. The Cronbach's alpha coefficient and item total correlation coefficient was used to determine the internal consistency of the scale.

Ethical Considerations
Prior to the conduction of the study, approval was obtained from the ethics committee (2021/3). Before the data was collected, the participants were informed about the aim of the study, and their consent was received through the electronic questionnaire form which stated that they agreed to participate in the study.

Content validity
The content validity index (CVI) value was determined to be 0.97 on the 22-item draft scale.

Face validity
A pilot study was conducted on a group of 28 people before finalizing the draft of the 22-item Health Literacy Scale for Protect against COVID-19. In the pilot study, the scale was sent to the participants via e-mail. This pilot group evaluated each item in terms of readability, comprehensibility of the terms, length of sentences, clarity and explicitness of meaning. The scale was reviewed in line with the group's recommendations and the necessary revisions were made. In accordance with the recommendations of the pilot study group, two items regarding the mobile application; "I can access information on the mobile application" and "I use the mobile application" were removed from the draft scale. Each item in the scale was graded as 5: Very easy, 4: Easy, 3: Difficult, 2: Very difficult, 1: I don't know. The final form of the scale with 20 items was added to the data collection form.

Construct validity
When the sample's adequacy for factor analysis was assessed, the following values were determined; KMO = .969, df: 190 and Bartlett's Test of Sphericity p <0.01. In the exploratory factor analysis of the COVID-19 literacy scale, a single factorial construct with an eigenvalue greater than 1 explained 70.639% of the total variance ( Table 2).

Criterion validity
The criterion validity of the Health Literacy Scale for Protect against COVID-19 was tested by examining its relationship with the THLS scale. A moderately significant correlation (r = .0569, p <0.001) was found between the total THLS score and Health Literacy Scale for Protect against COVID-19 score.

Reliability
It was determined that the Cronbach's alpha coefficient of the Health Literacy Scale for Protect against COVID-19 was .976 and the item-total correlation varied between 0.68-0.94 ( Table  4).

DISCUSSION
In the research, it has been determined that the Health Literacy Scale for Protect against COVID-19 is valid and reliable as a result of the analysis. Health literacy is very important to protect against the COVID-19 virus during the pandemic process and can help control the pandemic.

Validity
For the content validity, expert opinions are taken on whether the items in the measurement instrument are suitable for the purpose of measurement and whether they represent the domain of interest. Expert consensus indicates that the scale covers the health literacy for protect against COVID-19 and that the content validity is ensured [32,42]. In the study, the CVI was determined to be .97 and it was found that there was consistency among the experts' assessment scores. It was concluded that the items included in the draft scale was related to Health Literacy Scale for Protect against COVID-19 and that they represented the domain of interest [3,5,22,23].
In the literature, it is recommended that the draft scale, whose content validity was ensured by taking expert opinions, be applied to a small sample (10-30 people) with similar characteristics and the incomprehensible items be corrected [34]. Two items regarding the mobile application were removed from the draft scale in line with the views that the mobile application was not used in all segments of the society. Although mobile applications are quite common in Turkey, not everyone is capable of using them in the same manner.
EFA is conducted when the number of factors are not known beforehand in a scale development process. EFA is performed to see how the measurement variables group, in other words, which factors the scale items are related to [28,37,38]. Although it was expected that the newly developed Health Literacy Scale for Protect against COVID-19 would demonstrate a four-dimensional structure, it turned out to be one-dimensional which consisted of all the processes Chi-Square=500.65, df=129, P-value=0.00000, RMSEA=0.073   regarding COVID-19 literacy, namely; accessing, understanding, evaluating and applying information [3,5]. In addition, when the structure obtained by EFA was examined with CFA, it was seen that the goodness-of-fit indices were acceptable.
The criterion validity of the scale is conducted with another scale that assesses the same conceptual structure with the scale to be developed. While the THLS evaluates general health literacy, the newly developed Health Literacy Scale for Protect against COVID-19 assesses health literacy for protection against COVID-19. Both scales share similar conceptual structures. For this reason, the correlation between Health Literacy Scale for Protect against COVID-19 and the THLS total scores was investigated for criterion validity. As expected, a moderate correlation was found between both scales which indicated that the newly developed Health Literacy Scale for Protect against COVID-19 had criterion validity [38,42].

Reliability
It is stated that internal consistency reliability is necessary, though not sufficient for a newly developed scale; thus, other reliability measures should also be used according to the type of the instrument [42]. In the study, the internal consistency reliability of the scale was examined with Cronbach's alpha and item-total correlation. It was determined that the Cronbach's alpha coefficient of the scale was ideal. In addition, the fact that item-total correlation coefficients were greater than .65 indicated that each item contributed to the scale fairly well.

Limitations of the Study
In this study, online data collection may be considered as a limitation. Although online questionnaires are a simple and inexpensive method, respondents may be inattentive or get bored with questions and mark any option randomly. The fact that respondents may not always be honest about basic demographic information such as age, gender, and race, or that they submit the questionnaire twice may lead to inaccuracies in the data of the online questionnaires.
Another limitation of the study is that the mean age is young, indicating that mostly young people participated in the questionnaire. Last limitation of this study is that the EFA and CFA of the scale were done in the same group. In the literature, some studies suggest that it be done in the same group, while in some studies it is recommended to perform analyzes in different groups [46].

CONCLUSION
The Health Literacy Scale for Protect against COVID-19 showed a single-factor structure that covers four conceptual processes of health literacy. It is a valid and reliable scale that can be used to evaluate health literacy for protection against COVID-19 during the pandemic process. Currently, protection from COVID-19 is at the forefront all over the world, and there is a necessity to evaluate the health literacy of individuals in the society for protection. This scale is a valid and reliable measurement tool that can be used to evaluate the health literacy of the society in protection against COVID-19.
Author contributions: All authors have sufficiently contributed to the study, and agreed with the results and conclusions.

Funding:
No funding source is reported for this study.

Declaration of interest:
No conflict of interest is declared by authors.
Ethics committee approval: Ethics committee approval was received for this study from the Social and Humanities Ethics Committee of Istanbul Medeniyet University (Approval date: 14 January 2021).