|ORIGINAL RESEARCH PAPER
|Year : 2010 | Volume
| Issue : 3 | Page : 400
Evaluation of the Content Validity, Internal Consistency and Stability of an Instrument Designed to Assess the HIV/AIDS Knowledge of University Students
JA Balogun1, TC Abiona1, M Lukobo-Durrell1, A Adefuye1, S Amosun2, J Frantz3, Y Yakut4
1 College of Health Sciences, Chicago State University, Chicago, Illinios, USA
2 Faculty of Health Sciences, University of Cape Town, Cape Town, South Africa
3 Faculty of Community and Health Sciences, University of the Western Cape, Cape Town, South Africa
4 Faculty of Health Sciences, Hacettepe University, Ankara, Turkey
|Date of Submission||19-Sep-2009|
|Date of Acceptance||09-Nov-2010|
|Date of Web Publication||30-Nov-2010|
J A Balogun
College of Health Sciences, BHS 607, 9501 South King Drive, Chicago State University, Chicago, IL, 60628
Source of Support: None, Conflict of Interest: None
Objective: This study evaluated the content validity, internal consistency and stability of a questionnaire designed to assess the HIV/AIDS knowledge of young adults.
Methods: The questionnaire was administered on two occasions, within two week intervals, to 219 university students in the USA (n=66), Turkey (n=53) and South Africa (n=100). The psychometric instrument contained demographic information questions and 45 knowledge questions regarding the HIV virus, ways of transmitting it, symptoms of HIV/AIDS infection and methods of prevention. Factor analysis was conducted to assess construct validity; Cronbach alphas and Pearson's product moment correlation (r) were calculated to evaluate internal consistency and stability, respectively.
Results: The 45 knowledge questions produced a Kaiser-Meyer-Olkin (measure of sampling adequacy) value of 0.8133 and loaded on three factors with an overall Cronbach's alpha of 0.861. The r for the individual knowledge question was greater than 0.50 and the overall knowledge score was "almost perfect" (0.91; p<.001). For participating students from the USA, Turkey and South Africa, their r for the overall knowledge score was 0.80 (p<.001), 0.83 (p<.001) and 0.93 (p<.001), respectively.
Conclusions: Our findings revealed that the instrument is highly stable and internally consistent. The availability of this instrument may enhance HIV intervention studies internationally.
Keywords: Evaluation, knowledge, HIV/AIDS, psychometric instrument, university students, young adults
|How to cite this article:|
Balogun J A, Abiona T C, Lukobo-Durrell M, Adefuye A, Amosun S, Frantz J, Yakut Y. Evaluation of the Content Validity, Internal Consistency and Stability of an Instrument Designed to Assess the HIV/AIDS Knowledge of University Students. Educ Health 2010;23:400
|How to cite this URL:|
Balogun J A, Abiona T C, Lukobo-Durrell M, Adefuye A, Amosun S, Frantz J, Yakut Y. Evaluation of the Content Validity, Internal Consistency and Stability of an Instrument Designed to Assess the HIV/AIDS Knowledge of University Students. Educ Health [serial online] 2010 [cited 2020 Oct 26];23:400. Available from: https://www.educationforhealth.net/text.asp?2010/23/3/400/101474
Worldwide, human immunodeficiency virus/acquired immunodeficiency syndrome (HIV/AIDS) disproportionately affects young adults. In 2007, 5.4 million of the 1.7 billion persons between the ages of 15 and 24 years globally were estimated to be living with HIV1,2. In the same year, 45% of all new HIV infections occurred in persons under age 253. Young adults remain at the center of the HIV/AIDS epidemic in terms of vulnerability, impact and potential for change1. Social, political, cultural, biological and economic factors increase young adults’ vulnerability to HIV infection. Typically, because they have a feeling of invincibility, they are more likely to take risks.
Evaluation of the knowledge of young adults about HIV/AIDS is widely reported in the literature4-7. However, comparative data from different parts of the world is scarce. Following an exhaustive search of the literature on the CINAHL Plus, PsycINFO and MEDLINE databases (using the terms: HIV/AIDS knowledge; comparative data; young adults), we obtained no related “hit”. The paucity of comparative data on the HIV knowledge of young adults from different parts of the world may be attributed, in part, to the lack of culturally appropriate and reliable psychometric instruments. To better understand the HIV/AIDS knowledge of young adults from different parts of the world, a reliable and valid knowledge instrument is needed. Availability of comparative data on young adult’s HIV/AIDS knowledge can provide opportunities for sharing exchanges on intervention approaches.
Although knowledge about HIV and AIDS does not always translate to positive attitudinal and behavioral change, and knowledge alone is insufficient to produce and maintain positive behavior, an accurate knowledge of HIV transmission, risk and prevention is essential for behavior change and risk reduction to occur8. Exposure of young adults to appropriate HIV and AIDS information has been found to positively affect knowledge, attitudes and beliefs and to reduce risk behaviors that can predispose them to acquiring HIV infection1,7. Increasing the knowledge of young adults about HIV and AIDS is critically important in stemming the tide of the epidemic.
One of the objectives stated in the Millennium Development of the United Nations (goal # 6) is that all young adults have the knowledge and means to prevent HIV infection and to halt and reverse the spread of HIV/AIDS9. Similarly, as part of the Declaration of Commitment’s target for HIV prevention among young adults, resolution # 53 adopted by the United Nations General Assembly states that by 2005, it must be ensured that at “least 90%, and by 2010 at least 95% of young men and women aged 15 to 24 have access to the information, education, including peer education and youth-specific HIV education, and services necessary to develop the life skills required to reduce their vulnerability to HIV infection, in full partnership with young persons, parents, families, educators and healthcare”9. The United Nations General Assembly Special Session (UNGASS) on HIV/AIDS indicators for the implementation of this objective is the percentage of young women and men aged 15-24 years who both correctly identified ways of preventing the sexual transmission of HIV and who reject major misconceptions about HIV transmission3,8.
To realize the United Nations’ HIV prevention goals, it is necessary to continually study young peoples’ knowledge about HIV/AIDS and their sources of information about the disease. Such data will be useful in identifying existing gaps in knowledge and addressing misconceptions and myths. This study was designed to evaluate the psychometric properties (content validity, internal consistency and stability) of an HIV/AIDS knowledge questionnaire designed for use among university students in three target countries (United Sates of America, Turkey, South Africa) with varying HIV prevalence rates. HIV prevalence is considered to be “low level” in the USA and Turkey; it is “hyper-endemic” in South Africa.
In the USA, 1.1 million adults and adolescents were living with HIV at the end of 2006 and the HIV prevalence rate was 447.8 per 100,000 population10-12. HIV disproportionately affects minority populations: almost two-thirds (65.4%) of people living with HIV were ethnic minorities10-12. An estimated 13% of persons diagnosed with HIV or AIDS in the USA were young people aged 13 to 24 years old, and African Americans made up 55% of this figure13.
With an estimated 5.7 million South Africans infected14, the nation has the highest number of people in the world living with HIV/AIDS and current data indicate no reduction in HIV risk behaviors in that country3. In South Africa, the prevalence of HIV infection was 29.1% in 200614. In 2005, the prevalence of HIV among 15 to 19 year old South Africans was estimated at 3.2% among males and 9.4% among females; in the 20 to 24 year age group, the prevalence was 6.0% among males and 23.9% among females, while in the 25 to 29 year age group, the prevalence was 12.1% among males and 33.3% among females15.
Turkey is a developing country with conservative but rapidly changing social, cultural and moral norms4. However, the number of people living with HIV/AIDS in Turkey has continued to increase16,17 and about 76% of HIV infections occur among sexually active people between 15 to 49 years of age8. Turkey has 3,700 HIV infected persons in a population of 70 million. Youth between 15 and 24 years make up 317 (8.6%) of the 3,700 and less than 0.2% of the adult population (15 to 49 years) in Turkey is living with HIV/AIDS8.
Student volunteers were recruited from major public universities in the USA, Turkey and South Africa. The American university, with a population of 7,000 students, was a minority-serving public institution located in a cosmopolitan city in the Midwestern region of the country. The students from Turkey were recruited from an urban public university with a population of 25,800 students; over 98% of the students are of Turkish nationality. Students were recruited in South Africa from two public universities: one with a population of 21,562 predominantly White students and the other with a population of 12,129 mostly Black students.
This study, conducted in 2008, was a pilot study for a cross-sectional survey of the HIV/AIDS knowledge, perception of knowledge and sources of information among university students in the three countries18,19. The study was designed to evaluate the content validity, internal consistency and stability of a psychometric instrument to assess the HIV/AIDS knowledge of university students in the three target countries. As a test-retest design protocol, each study subject served as its own control20. Consequently, demographic characteristics such as age, gender, race/ethnicity, marital and enrollment statuses did not affect the outcomes.
We focused on four universities from the three countries, the selection based on differences in HIV prevalence rates as well as cultural, religious and ethnic differences. The universities in each country represent a “convenience sample”, based on mutual ongoing agreements between institutions. Study participants were volunteers and also a sample of convenience. A common feature of the study populations was that they were mostly young adults. Subjects were recruited in each university through an advertisement (flyers) posted on campus. To be eligible for the study, participants had to be at least 18 years old and available to repeat the survey within two weeks of completion of the first survey. Participation was on a first come, first served basis. The study protocol was approved by the Institutional Review Boards at the participating universities.
The HIV/AIDS knowledge items in this survey were adapted from two primary sources. We selected several items from the HIV knowledge questionnaire conceptualized by Carey and Schroeder21 and from the National Health Interview Survey of AIDS Knowledge and Attitudes from the USA Center for Disease Control and Prevention5. The resulting psychometric instrument sought socio-demographic information such as age, gender, marital status and year in school. There were 45 questions regarding knowledge of HIV infection: the HIV virus; ways of transmitting it; symptoms of HIV/AIDS infection; and methods of prevention. The demographic questions, except age, provided options for each item. All the knowledge questions had three options: “True”; “False”; and “Don’t Know”. An overall knowledge score was computed by adding the total number of correct responses on the 45 questions on HIV/AIDS, resulting in a range from 0 to 45.
To establish the content and face validity of the instrument and for cultural relevance, students and faculty members from selected universities in the USA and South Africa reviewed the questionnaire for clarity. Based on their input, several items of the initial draft of the questionnaire in the English language were restructured to improve comprehension. The English version of the questionnaire was translated into Turkish and back into English to ensure validity. We subsequently determined the readability of the two questionnaires using the Flesch-Kincaid assessment instrument on Microsoft Word software.
The English version of the survey instrument was used in the USA and South Africa and the translated Turkish version was used in Turkey. The purpose, significance, benefits and potential risks of the study were explained to the student participants. The students were then asked to complete a paper and pencil survey after providing verbal consent. Participants were instructed to answer the questions as honestly and as accurately as possible. A time limit was not imposed for completion of the questionnaire, but most participants completed the questionnaire in less than 30 minutes.
On the second occasion, similar standardized testing conditions and instructions were maintained. Testing was done by the same research staff members. Following the administration of the questionnaire, the staff retrieved the card provided during the first testing session from the participants before giving them the questionnaire to fill out. The number on the card was identical to the identification number on the questionnaire. Participants who did not bring their card during the second data collection were not allowed to participate in the survey. The interval between the sessions was two weeks. We selected the two weeks test-retest interval to enable comparison of our findings with previous studies21-26. The compliance rate of the study participants was very high; over 95% of the students in each country who volunteered for the pre-test session returned for the second, post-test session.
Upon completion of both sessions, study participants received a book voucher or phone card for their time. Participants’ questions were addressed before and after completing the survey, and those who required further services such as HIV testing were referred to the appropriate personnel. Less than 5% of the study participants sought HIV testing and no discernable difference was observed between the three countries.
Each item on the questionnaire was coded and entered on the computer. Raw data were cross-checked for accuracy prior to analyses. We calculated percentages to describe the demographic variables. The Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy, Bartlett’s test of sphericity, eigenvalue, % variance, cumulative %, mean, skewness, kurtosis and Cronbach’s alpha of each factor were examined using the STATA Intercooled 10.0 (STATA Corp). Internal consistency was evaluated by determining Cronbach’s alpha (α) for each factor retained; α &8805; 0.70 was accepted as adequate23,25,26.
The demographic profile of the students from the three target countries is presented in Table 1.
Table 1: Demographic Characteristics of the Study Participants in the Three Target Countries (n = 219)
Overall, the study participants from the USA were older (p<.05) than their peers from Turkey and South Africa. The university in the USA where this study was implemented traditionally attracts older students and over 90% were ethnic minorities (African Americans and Latino/Hispanics). Fifty-eight percent of the study participants were junior students, 12% sophomores, 16% seniors, 4% graduate students and 7% undisclosed. For the present study, participating students from all three countries ranged in age from 18 to 49 (mean=24;1, sd=5.8) years. The majority of study participants were women (77%) and young adults between 15 to 24 years old (74%).
Exploratory factor analysis
Factor analysis was conducted to assess the instrument’s construct validity. The KMO value for the sample was 0.8133. The KMO is a measure of sampling adequacy and is used for comparing the magnitudes of the observed correlation coefficients to the magnitudes of the partial correlation coefficients. KMO values >0.6 indicate that performing factor analysis is worthwhile (STATA Corp). We employed Bartlett's test of sphericity to test the null hypothesis that the items in the questionnaire correlation matrix are uncorrelated. The null hypothesis was rejected because of the strength of the calculated Chi-square and p-values that we obtained (X2=4089.81; p<0.001). The data revealed that the relationship among the items in our questionnaire was strong.
As a result of the high KMO value obtained from the Bartlett’s test of sphericity, we proceeded to perform the principal axis factor analyses on the knowledge questions. Principal components factor analysis with Varimax rotation was done to identify the nature and number of variables that explain the relationship between items. The criterion for retaining a factor was an eigenvalue greater than one. In addition, the items that were retained in the factors had to have factor loadings at least 0.40, inter-item correlations of <0.90 and had to load on only one factor. (Statistical Package for Social Sciences software version 16 for Windows). Of the 45 HIV/AIDS knowledge questions, 36 met all the factor and item criteria and a three factor solution emerged (Table 2).
Table 2: Principal Components of the Factor Analysis for the Knowledge Questions
The result of the factor analysis for the knowledge subscale revealed that 18 items loaded on Factor 1 (“Incorrect Facts about HIV Transmission and Prevention”) with Cronbach’s alpha of 0.904. Twelve items loaded on Factor 2 (“Correct Facts about HIV Transmission and Prevention”) with Cronbach’s alpha of 0.845. Six items loaded on Factor 3 (“Common or Everyday Myths about HIV Transmission and Prevention”) with Cronbach’s alpha of 0.501. Overall, Cronbach’s alpha for the HIV/AIDS knowledge subscale was 0.861. The eigenvalues indicated that the first factor explained about 19% of the variance, the second factor explained 15% and the third factor explained 4%. Combined, 38% of the variance was explained by the three factors, and all the loadings were positive and without cross loadings.
Composite scores were created for each of the three factors, based on the mean of the items which had their primary loadings on each factor. Higher scores indicated greater correct responses to the questions that make up the factors. The students had more correct responses to the survey questions that addressed incorrect facts about HIV transmission and prevention, while the survey questions relating to correct facts and myths received less correct answers and had positively skewed distributions. The skewness and kurtosis distribution of the data were within a tolerable range for assuming a normal distribution.
Out of the maximum possible score of 45, the mean (sd) knowledge score for the students from the USA, South Africa and Turkey were 34.4 (5.6), 27.1 (9.8) and 21.1 (8.3), respectively. Students from the USA and South Africa had better knowledge (p<.05) about HIV/AIDS than students from Turkey. The overall knowledge score of the respondents older than 25 years of age was significantly higher (mean=32.97; sd= 7.73) than for young adults (mean=26.0; sd= 9.7: t=4.907; p<.001).
Test-retest reliability of the HIV/AIDS knowledge questions
The Pearson’s product moment correlation coefficient (r) for each of the 45 HIV/AIDS knowledge items on the questionnaire is presented in the Appendix. The test-retest reliability coefficients for most of the items on the knowledge subscale were greater than 0.50. Among the USA students, three of the items (“A woman can get HIV if she has anal sex with a man”, “Eating healthy foods can keep a person from getting HIV”, “A person can get HIV even if she or he has sex with a person one time”) had poor test-retest reliability coefficients (p>.05). The Pearson product moment correlation coefficients for students from Turkey ranged from “fair” (0.20 to 0.40) to “almost perfect” (0.80 to 0.99). One item (“Pulling out the penis before a man climaxes or “cums” keeps a woman from getting HIV during sex”) was rated poorly (r=0.12, p>.05) by South African students. The overall Pearson product moment correlation coefficient for the knowledge score was “almost perfect” (0.91, p<.001) for the three groups combined; by country, the values were: USA, 0.80 (p<.001); Turkey, 0.83 (p<.001); and South Africa, 0.93 (p<.001).
While a number of studies on the HIV risk behaviors of young adults exist in the literature, cross-cultural comparative studies are scarce. This study attempted to evaluate the readability and stability of a psychometric instrument designed to assess the HIV/AIDS knowledge of university students in the three countries with varying HIV epidemic rates.
This evaluative study is significant because to our knowledge it represents the first attempt to determine the psychometric properties of an instrument designed to assess the HIV risk behaviors of young adults in the three different regions. The HIV epidemic is considered to be low level in the USA and Turkey; it is hyper-endemic in South Africa. In addition to disparities in prevalence rates, other selection criteria such as culture, religion and ethnicity were also considered. Both South Africa and the USA are ethnically diverse consisting of Whites, Blacks, Asians and other ethnic races and are both predominantly Christian nations. On the other hand, Turkey is predominantly White and ethnically a less diverse Islamic nation with conservative moral ideals that are being influenced by Western values. It is generally believed that the HIV/AIDS prevalence data in Turkey may not reflect the true scope of the problem because most people with sexually transmitted diseases do not generally seek medical care16. In the last decade, Turkey has witnessed increased access to pornographic materials and increased sexual promiscuity among adolescents. The known positive influence of religion in moderating morality and sexual activity of adolescents has decreased27,28.
Our findings revealed that the questionnaire requires an eighth grade reading level to comprehend. The instrument will find wider application among clinicians, counselors and researchers interested in evaluating the HIV risk behaviors of adolescents and young adults in the three countries. The questionnaire was relatively stable within the two-week test period. The level of agreement that we observed on retest compared favorably or better with the reliability coefficients reported in other HIV-specific research questionnaires21,22 with the same two-week retest interval.
The overall test-retest reliability coefficients for the items on the demographic subscale were generally high, but much lower test-retest reliability was obtained for the HIV/AIDS knowledge subscale. This finding can be attributed, in part, to the repeat testing phenomenon20. The study participants were aware that the survey would be re-administered. It is plausible that many of the students recalled the questions after the first administration of the survey and went in search of the correct answer.
Overall, no major discernible differences exist in the test retest reliability of the three groups. The instrument has potential for wide application internationally in assessing HIV/AIDS knowledge of adolescents and young adults with at least an eighth grade reading level. The availability of this instrument may spur future research that will enhance effective targeted HIV prevention intervention among university students.
Study limitations and future studies
Although the study participants were assured of the anonymity of their responses, and no planned HIV education was implemented, it is plausible that the study participants could have been exposed to more information about HIV/AIDS during the two-week interval. Our questionnaire was relatively stable within the two-week test period. The high test-retest reliability obtained in our study could be attributed, in part, to the recall bias phenomenon20. To address this limitation in future studies, the stability of the instrument should be evaluated over a longer period.
To enhance the external validity and applicability of the questionnaire, the psychometric properties of the instrument should be investigated by recruiting students from the other parts of the world. The evaluation of the instrument’s sensitivity and validity is needed in future research. However, based on the findings in this study and the findings from other previous studies18,19, the instrument should be of interest to clinicians and researchers conducting intervention studies involving HIV/AIDS knowledge of young adults in the three countries, and availability of this instrument may enhance HIV intervention studies internationally.
This work was supported by the Illinois General Assembly (Contract #: P.I. 94-0629) and the Illinois Department of Public Health (Contract #: 75781008). The information, content and conclusions drawn from our research are those of the authors and should neither be construed nor inferred to be an endorsement by the Illinois General Assembly or Illinois Department of Public Health.
1. UNFPA. Preventing HIV/AIDS. Young People: The greatest Hope for Turning the Tide. 2009. Retrieved April 15, 2009, from: http://www.unfpa.org/hiv/people.htm#why.
2. UNICEF. Preventing Infection among Adolescents and Young People. 2009. Retrieved April 15, 2009, from: http://www.unicef.org/aids/index_orphans.html.
3. UNAIDS. Report on the global AIDS epidemic. 2008. Retrieved April 15, 2009 from: http://www.unaids.org/en/KnowledgeCentre/HIVData/GlobalReport/2008/2008_Global_report.asp.
4. Gökengin D, Yamazhan T, Özkaya D, Aytuğ S, Ertem E, Bilgin A. Sexual knowledge, attitudes, and risk behaviors of students in Turkey. Journal of School Health. 2003; 73:258-263.
5. Center for Disease Control and Prevention. National college health risk behavior (NCHRB) survey codebook. 1995. Retrieved May 2, 2008 from: http://www.cdc.gov/HealthyYouth/yrbs/data/college/nchrbs95.pdf.
6. Center for Disease Control and Prevention. HIV/AIDS Surveillance Report, Vol. 16. Atlanta. US Department of Health and Human Services, CDC: 2005:1–46. 2004. Retrieved April 16, 2009 from: http://www.cdc.gov/hiv/surveillance/resources/reports/2004report.
7. Bastien S, Sango W, Mnyika KS, Masatu MC, Klepp KI. Changes in exposure to information, communication and knowledge about AIDS among school children in Northern Tanzania, 1992-2005. AIDS Care. 2008; 20: 382-387.
8. UNAIDS. UNGASS Indicator Country Report, Turkey. 2006. Retrieved April 15, 2009, from: http://data.unaids.org/pub/Report/2006/2006_country_progress_report_turkey_en.pdf
9. United Nations. Goal #6: Combat HIV/AIDS, Malaria and Other Diseases. 2009. Retrieved April 14, 2009, from: http://www.un.org/millenniumgoals/aids.shtml, and http://www.un.org/millenniumgoals/2008highlevel/pdf/newsroom/Goal%206%20FINAL.pdf.
10. Center for Disease Control and Prevention. HIV/AIDS among youth. 2008a. Retrieved September 17, 2008 from: http://www.cdc.gov/hiv/resources/factsheets/youth.htm
11. Center for Disease Control and Prevention. HIV/AIDS in the United States. 2008b. Retrieved September 17, 2008 from: http://www.cdc.gov/hiv/resources/factsheets/us.htm
12. Centers for Disease Control and Prevention. Subpopulation Estimates from the HIV Incidence Surveillance System --- United States. Morbidity and Mortality Weekly Report. 2006; 57: 985-989.
13. Center for Disease Control and Prevention. HIV Prevention in the Third Decade. Atlanta: US Department of Health and Human Services, CDC. 2005. Retrieved April 16, 2009 from: http://www.cdc.gov/hiv/resources/reports/hiv3rddecade
14. Department of Health (South Africa). National HIV and Syphilis antenatal sero-prevalence survey in South Africa. Pretoria: Department of Health, 2007.
15. Shisana O, Rehle T, Simbayi LC, Parker W, Zuma K, Bhana A, Connolly C, Jooste S, Pillay V. South African National HIV prevalence, HIV Incidence, Behavior and Communication Survey. Cape Town: HSRC Press; 2005.
16. Duyan V, Agalar F, Sayek I. Surgeons’ attitudes toward HIV/AIDS in Turkey. AIDS Care. 2001; 13: 243-250.
17. Ungan M, Yaman H. AIDS knowledge and educational needs of technical university students in Turkey. Patient Education and Counseling. 2003; 51:163-167.
18. Balogun JA, Abiona T, Adefuye A, Amosun SL, Frantz J, Yakut Y. Readability and test retest reliability of a psychometric instrument designed to assess HIV/AIDS attitudes, beliefs, behaviors and source of HIV prevention information of young adults. Health Education Journal. 2010a; in press.
19. Balogun JA, Abiona TC, Yohannes E, Adefuye A, Amosun SL, Frantz J, Yakut Y. HIV/AIDS knowledge, perception of knowledge, and sources of information among university students in USA, Turkey, South Africa and Nigeria. Health Education Journal. 2010b; in press.
20. Campbell DT, Stanley JC. Experimental and Quasi-Experimental Designs for Research. Chicago: Houghton-Mifflin College; 1966.
21. Carey MP, Schroeder KEE. Development and psychometric evaluation of the brief HIV knowledge questionnaire (HIV-KQ-18). AIDS Education and Prevention. 2002; 14:174-184.
22. Jaworski BC, Carey MP. Development and psychometric evaluation of a self-administered questionnaire to measure knowledge of sexually transmitted diseases. AIDS Behavior. 2007; 11:557-574.
23. Cronbach L. Coefficient alpha and the internal structure of tests. Psychometrika. 1951; 16:297–333.
24. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977; 33:159-174.
25. Lohr KN, Aaronson NK, Alonso J. Evaluating quality-of-life and health status instruments: development of scientific review criteria. Clinical Therapy. 1996; 18:979–992.
26. Lohr KN. Assessing health status and quality-of-life instruments: attributes and review criteria. Quality Life Research. 2002; 11:193–205.
27. Bulut A, Güvenli O. Cinselik ve condom kullamini. In: Proceedings of the 5th Turkish Congress on AIDS, Istanbul, Turkey; 2001.
28. Özcebe H. Universite gencliğinde güvenli cinsel yasam. HIV/AIDS. 2002; 5:51-58.
Appendix: Test-Retest Reliability of the HIV/AIDS Knowledge Questions by Country and in Total