|ORIGINAL RESEARCH PAPER
|Year : 2013 | Volume
| Issue : 1 | Page : 48-53
Evaluating internet information on attention-deficit/hyperactivity disorder (ADHD) treatment: Parent and expert perspectives
Alonso Montoya1, Sergio Hernández2, Montserrat Pamias Massana3, Oscar Herreros4, Marta Garcia-Giral5, Esther Cardo6, Jose Angel Alda Diez7, Cesar Soutullo8, Alberto Fernández-Jaén9, Joaquín Fuentes10
1 Clinical Research, Lilly Research Laboratories, Alcobendas, Spain
2 Facultad de Psicología, Universidad de La Laguna, Tenerife, Spain
3 Servei de Psiquiatria Infantil i Juvenil. Corporació Sanitària Parc Taulí, Sabadell, Spain
4 Unidad de Psiquiatría Infanto-Juvenil, Hospital Virgen de las Nieves, Granada, Spain
5 Servei de Psiquiatria i Psicologia infantil i juvenil. Hospital Clínic, Barcelona, Spain
6 Servicio de Neuropediatría, Hospital Son Llàtzer, Palma de Mallorca, Spain
7 Servicio de Psiquiatría y Psicología Infanto-Juvenil, Hospital Universitario Sant Joan de Déu, Barcelona, Spain
8 Unidad de Psiquiatría Infantil y Adolescente, Departamento de Psiquiatría y Psicología Médica de la Clínica, Clínica Universitaria, Universidad de Navarra, Pamplona, Spain
9 Servicio de Neuropediatría, Hospital Universitario Quiron, Madrid, Spain
10 Servicio de Psiquiatría Infantil y Adolescente. Policlínica Guipúzcoa, San Sebastián, Spain
|Date of Web Publication||31-May-2013|
Lilly Research Laboratories, Avenida de la Industria, 30, 28108 Alcobendas (Madrid)
Source of Support: This work has been supported by Lilly Research Laboratories, Alcobendas, Spain., Conflict of Interest: None
Background: The Internet is increasingly used as a source of health-related information. The objective of this study was to assess the quality of web-based information on treatments for attention-deficit/hyperactivity disorder (ADHD). Methods: Sixteen expert health professionals in ADHD and 35 parents of paediatric patients with a recent diagnosis of ADHD assessed the information contained in the 10 highest ranked websites in Spanish, using the Spanish version of the DISCERN tool - a validated questionnaire designed to assess the quality and reliability of web-based information on treatment choices (rating scores from 15 to 75). Results: DISCERN scores given by parents and experts were low (total mean scores [standard deviation]: 35.9 [13.1] and 43.4 [13.7], respectively) and inter-rater agreement was poor/moderate (weighted kappa for the global assessment between -0.69 and +0.93, average = 0.29). There was a significant change on the ADHD-knowledge and motivation for treatment (ADHD-KMT) basic knowledge sub-scale score after the assessment of the different websites by parents (total mean scores [standard deviation]: 49.09 [9.46] and 63.21 [9.45]). Conclusions: Despite a poor/moderate inter-rater agreement between parent and expert opinions, all agreed that the quality of the web-based information on treatment choices for ADHD is generally poor.
Keywords: ADHD, DISCERN, information, internet, reliability
|How to cite this article:|
Montoya A, Hernández S, Massana MP, Herreros O, Garcia-Giral M, Cardo E, Diez JA, Soutullo C, Fernández-Jaén A, Fuentes J. Evaluating internet information on attention-deficit/hyperactivity disorder (ADHD) treatment: Parent and expert perspectives. Educ Health 2013;26:48-53
|How to cite this URL:|
Montoya A, Hernández S, Massana MP, Herreros O, Garcia-Giral M, Cardo E, Diez JA, Soutullo C, Fernández-Jaén A, Fuentes J. Evaluating internet information on attention-deficit/hyperactivity disorder (ADHD) treatment: Parent and expert perspectives. Educ Health [serial online] 2013 [cited 2021 Jan 25];26:48-53. Available from: https://www.educationforhealth.net/text.asp?2013/26/1/48/112801
| Introduction|| |
Parents of children with neurobehavioral disorders have particular needs for relevant and accessible information.  Attention-deficit/hyperactivity disorder (ADHD) is one of the most common chronic conditions of childhood, characterised by developmentally inappropriate symptoms of inattention, hyperactivity and impulsivity. ADHD has a childhood onset of symptoms that typically results in a chronic and pervasive pattern of impairment in school, work, social and daily adaptive functioning. 
ADHD management requires a multimodal approach with a combination of pharmacologic and non-pharmacologic therapies, with medication often the first intervention used to treat ADHD.  As parents share with physicians the decision-making process regarding the treatments their children receive, parent's knowledge and opinions on ADHD and its treatments play a significant role in treatment choices for their children. Therefore, there is an increasing need for parents of patients to receive good quality information upon which to base their decisions. 
This information can come from a variety of sources. Parents traditionally seek and respect their children's physicians' advice and opinions about health-related information;  however, studies show that parents of patients are increasingly consulting other sources of information, mainly Internet, even before visiting a healthcare professional.  Although the Internet represents an unprecedented opportunity to provide accessible health-related information to patients and their families, the uncontrolled volume of information available frequently leads to doubts about its reliability and quality. , This is particularly important in ADHD because there is strong evidence for different treatments, but ADHD remains a controversial area around which sensational disclosures often appear in sources of information, mainly on the Internet. This can be confusing for parents. In this context, the evaluation of the degree of agreement between healthcare professionals and family members' opinions about the quality of the information found on the Internet can be important in defining the potential for controversy, which in turn has a direct impact on the physician-parent/patient relationship. Although some studies have attempted to objectively rate the quality of consumer health information on ADHD, no studies have looked specifically at web-based information on treatment options for children with ADHD. ,
DISCERN is a standardised instrument (16-item questionnaire) designed to help patients and professionals assess the reliability and quality of written information for the public on treatment choices.  To evaluate Spanish web-based information on medical treatment, we previously created and validated a Spanish version of DISCERN to be used by health professionals, all of them ADHD experts, and parents of patients with a recent diagnosis of ADHD.  The aim of this descriptive cross-sectional study was to evaluate the quality of information on treatment options for ADHD available on the Internet, from a family and health professional perspective, and the degree of agreement between family members' and physicians' ratings using a validated instrument. As a secondary objective we evaluated whether this review exercise influenced parents' basic knowledge of ADHD.
| Methods|| |
Subjects: Twenty expert health professionals (child psychiatrists, child neurologists or paediatricians) were invited to participate in the study. Four of them declined to participate due to overload with other research and clinical duties. As experts in ADHD, they had to be leaders in the ADHD field, had participated in ADHD clinical trials and voluntarily agreed to participate in the design and implementation of this research. Additionally, 35 parents of paediatric patients with a recent diagnosis of ADHD participated in this study. Parents of patients were invited to participate by their children's physicians and also through a local advocacy group. This study was approved by an institutional review board from H.U.P.H. in Madrid, Spain (Acta no. 258).
Website selection: To identify potential websites for our study, we performed a search in the most popular search engines used in Spain (MSN, Yahoo, Lycos, Google, Altavista, Excite, Infoseek)  introducing search terms 'ADHD, Attention Deficit Hyperactivity Disorder, treatment' in Spanish ('TDAH, Trastorno por déficit de atenciσn con hiperactividad, tratamiento'). Keywords were selected to represent terms that consumers may use when they search for information about ADHD. The search was done in May 2010.
We included the first 200 results yielded by each search engine for suitable websites. The Rate (number of times or frequencies that a particular site appears among different engines, where a higher rate means a bigger number of appearances) and Score (the sum of the position number that a particular website appears among the different engines, where a lower number means a higher order rank among the different engines) were calculated. These two measurements produced a total score for each site, and sites were then ranked from highest to lowest scoring. The 10 highest ranked sites were selected by a consensus between main investigators and contrasted with link analysis algorithms ([Figure 1] presents a flow diagram of the process).
|Figure 1: Flow diagram on selection of the websites assessed in the study|
Click here to view
Assessment Tools: The DISCERN questionnaire consists of 16 questions, on a continuous rating scale (Likert type scale) from 1 to 5, where 1 = definite NO and 5 = definite YES. Any rating in between (such as a 2, 3 or 4) suggests that some of the elements asked are present to a certain extent. These questions are categorised into three sections. Section 1 (questions 1 to 8) assesses reliability, dependability and trustworthiness of a website; Section 2 (questions 9 to 15) focuses on the quality of information about treatment choices and Section 3 (question 16) evaluates overall quality rating for the online website on a continuous rating scale with ratings from 1 = Low to 5 = High. The rating for question 16 is done independently based on the previous 15 question ratings.  We translated the tool into the Spanish language and validated this translation of the DISCERN questionnaire. To do this, we used a modified version of a method for validating translated instruments in which the original source-language version is formally compared with the back-translated source-language version in terms of comparability of language, similarity of interpretability, and degree of understanding. ,
The ADHD-knowledge and motivation for treatment (ADHD-KMT) is a parent questionnaire designed to assess ADHD-specific knowledge and motivation for treatment. The questionnaire comprised of 37 items on three scales: Basic knowledge, differentiated knowledge and motivation for medical treatment. The basic knowledge scale was used before and after parents' assessment of the quality and reliability of the web-based information on treatments for ADHD. 
Statistical analysis: Descriptive statistics of the scores given by health professionals and patients' caregivers were calculated separately. This included also the pre- and post-test ADHD-KMT scores of patients' relatives. The websites evaluated were ranked according to the mean total DISCERN scores given by both groups of raters. To analyse the degree of agreement between these scores, a weighted (quadratic) kappa score was calculated for each possible pair of professionals and caregivers. Then an overall kappa score and the associated standard error were calculated according to the procedures described by Fleiss et al.  This process was repeated for each of the 16 items forming the DISCERN tool. Additionally, as internal measures of reliability, some scores of consistency (Spearman's correlation between item 16 ratings and total DISCERN scores and Kendall's W of item 16 ratings) and agreement (intra-class correlation coefficient of agreement of total DISCERN scores) were calculated for healthcare professionals on the one hand and patients' caregivers on the other. Finally, a paired t-test was used to compare the pre- and post-test scores of the ADHD-KMT questionnaire. In order to avoid having too many missing data to calculate coefficients that required crossing sums between all evaluators and/or sites, all available data were used in each calculation. Some evaluators were removed from the study because they evaluated very few sites and some websites were removed because they were evaluated only by very few experts. In contrast, missing data were not imputed.
| Results|| |
Demographic and background characteristics of the parents and ADHD experts are presented in [Table 1]. Participants involved in the study were mainly 30- to 50-year-old mothers with a superior educational level and child psychiatrists from public clinical practice.
|Table 1: Participating parents' and ADHD experts' demographic characteristics|
Click here to view
A total of 25,000 websites were retrieved. Considerable overlap was seen in the search results from each engine. After removal of duplicates, and excluding websites without relevant information with regard to ADHD treatment, inaccessible sites, blogs or pay sites, 100 sites were identified. After calculating Rate and Score, and contrasting with link analysis algorithms, 10 individual, relevant websites were selected for evaluation [Figure 1]. Descriptions are summarised in [Table 2].
DISCERN scores given to the 10 websites by healthcare professionals and parents of patients with ADHD were low (total mean scores [standard deviation]: 35.9 [13.1] and 43.4 [13.7], respectively, over a total possible score of 75). Websites ranked according to DISCERN scores (total mean scores) are presented in [Table 3]. Although there are some small differences in the order categorised, in general, websites ranked from the highest to the lowest DISCERN score were almost the same for both groups. The website that showed the highest DISCERN score was www.nlm.nih.gov in both groups. The inter-rater agreement between healthcare professionals and patients' caregivers was moderate. The overall weighted kappa scores ranged from 0.03 (item 12) to 0.35 (item 4) and were moderate (0.30) for item 16 (overall assessment of quality). They ranged between -0.69 and +0.95, with average (arithmetic mean) of 0.29.
|Table 3: Websites ranked by experts and parents according to DISCERN Scores|
Click here to view
The correlation between the total mean scores and item 16 (global assessment) was moderate/good. The Spearman's correlation coefficients ranged from 0.54 (site 8) to 0.87 (site 3) among professionals and from 0.56 (site 1) to 0.89 (site 9) among parents of patients. Kendall's W for item 16 ratings were also moderate/good (0.56 and 0.58 for professionals and parents, respectively), as were the intra-class correlation coefficients of agreement for total DISCERN scores (0.94 and 0.76 for professionals and relatives, respectively). Finally, the change in the ADHD-KMT basic knowledge sub-scale score after the assessment of the different websites by parents (total mean scores [standard deviation]: 49.09 [9.46] and 63.21 [9.45]) was statistically significant, showing an improvement in their basic knowledge on ADHD.
| Discussion|| |
Child health information is easily found on the Internet from many reputable sources; however, a broad assortment of information about child health also available has uncertain reliability. This information likely conflicts with physician-provided advice,  as some studies have reported that in more than 70% of those who use Internet the health information they find influences their decision about their children's treatment.  The Internet is a powerful tool for accessing information about health topics such as ADHD, but its use also puts forth the risk that inaccurate, harmful and perhaps obsolete information will be disseminated to patients and their families.
Within the past decade, researchers have begun to assess the quality of information on the internet related to a variety of paediatric health conditions. To our knowledge, this is the first study that analyses the information on ADHD websites in Spanish using a highly reliable tool for that purpose.
As shown in our results, according to experts and parents of patients, the overall quality of the information on treatment options for ADHD provided by the majority of reviewed sites was low. We used the 10 highest ranked Internet websites on the main search engines in Spain. Because these websites were selected after a thorough method on the most common engines used in Spain, it is likely that these are the websites that any parent would visit and use to make decisions regarding their children's ADHD management. Our results highlight the fact that not all the highly ranked websites on popular search engines are necessarily the most reliable when it comes to accessing accurate information regarding ADHD treatment.
Relevant websites selected for evaluation consisted of a very heterogeneous group of sites. It is interesting to note that within these highly ranked websites, only a minority came from medical institutions or governmental organisations (i.e. Cleveland Clinic, The National Library of Medicine), only one site came from a Spanish advocacy group (www.still-tdah.com) and the rest came from private and even personal sites.
The website whose quality was better evaluated by the experts and parents was the site of the National Library of Medicine (NLM) (www.nlm.nih.gov). As part of the National Institutes of Health, and the Department of Health and Human Services in the United States, NLM is probably the world's largest medical library, which collects materials and provides information in all areas of biomedicine and healthcare. It is possible that the governmental nature of this site grants implicit values of seriousness and rigorousness that may contribute to its high evaluation of quality.
In this study, the DISCERN questionnaire was used to evaluate the quality of ADHD websites. DISCERN has been used  in a number of studies. This questionnaire has shown good inter-rater agreement when used by healthcare professionals, ,,, and lower when used by patients. It has been suggested that the greater level of inter-rater agreement between healthcare professionals may be explained by the fact that they are more familiar with the information they are rating and so will be able to assess its quality more consistently.
Although, in our study, we found some discrepancies between parents' and experts' ratings (poor/moderate agreement between experts and parents of ADHD patients), all agreed that the quality of the web-based information on treatment choices for ADHD was generally poor. Also, despite small differences in the order categorised, the ranking from the highest to the lowest DISCERN score was almost the same for both groups.
In this study, we also explored whether a relatively simple intervention such as asking parents to assess the quality and reliability of web-based information on ADHD treatments could modify their basic knowledge of ADHD. Parents' scores on the ADHD-KMT basic knowledge scale showed that their ADHD-specific knowledge improved significantly after this relatively simple exercise, suggesting that the review of quality of information on a few websites had a significant effect on parents' basic knowledge.
As joint decision-making between parents of patients and physicians becomes an established part of modern medical care in children, there is an increasing need for patients to receive good quality information upon which to base their decisions. Because of shorter appointment times, parents are often less able to spend time in the physician's office receiving information and may often have questions that arise after the visit has concluded.
It is important that healthcare professionals inform patients of the variable quality of health-related information on the Internet. Parents of patients are generally not aware of characteristics that indicate quality information on the Internet,  therefore education about quality indicators (such as a clear statement of the purpose and target audience for the website, core standards for information provision such as authorship, disclosure, attribution and currency, citation of the best available evidence, acknowledgement of risks, other treatments and the effect of no treatment, references to further sources and clear disclosure of sponsorship, affiliations or conflicts of interest) may have the potential to increase parents ability to access and resource quality information.
Because there is such a vast amount of constantly changing health information available on the Internet, this study is limited by the cross-sectional design and the small number of websites included. Websites are being updated or removed constantly and new ones are emerging, all of which may change the quality of websites analysed in this study, thus the validity of the results may be considered relative and temporal (time-dependent).
As this was a pilot study, the results are presented descriptively. As a pilot study, the relatively small number of websites and evaluators can also limit the validity and generalisation of the results, but the use of the first-ranked websites in the most popular web search engines addresses, in part, this limitation, because they are the pages that people tend to use when seeking information from Internet.
Overall, the Internet provides great opportunities for healthcare education, but it carries the risk that inaccurate and/or outdated information will be disseminated to patients and their families. As parents of patients with ADHD share the decision-making process regarding treatment, this information is important as the basis for their decisions; however, its reliability and quality is frequently questioned. In the present study we investigated the quality of web-based information on treatments for ADHD. The DISCERN scores given by parents of patients and experts were low and inter-rater agreement between experts and parents was poor/moderate. Despite there being a poor/moderate inter-rater agreement between parents and experts opinions, all agreed that the quality of the web-based information on treatment choices for ADHD is generally poor. This study demonstrates the variability in the quality of treatment option information available on websites in Spanish.
| Acknowledgements|| |
The authors are indebted to all the parents of patients who kindly provided the data necessary for their analysis, and also to all expert participants. The authors thank Dr. Sasha Shepperd for her valuable input, Marco Municio for his technical support and also Medicxact that acted as an external statistical services provider.
| References|| |
|1.||Sethuram R, Weerakkody AN. Health information on the internet. J Obstet Gynaecol 2010;30:119-21. |
|2.||Biederman J, Spencer T, Wilens T. Evidence-based pharmacotherapy for attention-deficit hyperactivity disorder. Int J Neuropsychopharmacol 2004;7:77-97. |
|3.||Bukstein OG. Clinical practice guidelines for attention-deficit/hyperactivity disorder: A review. Postgrad Med 2010;122:69-77. |
|4.||Moseley KL, Freed GL, Goold SD. Which sources of child health advice do parents follow? Clin Pediatr (Phila) 2011;50:50-6. |
|5.||Sciberras E, Iyer S, Efron D, Green J. Information needs of parents of children with attention-deficit/hyperactivity disorder. Clin Pediatr (Phila) 2010;49:150-7. |
|6.||Peterson G, Aslani P, Williams KA. How do consumers search for and appraise information on medicines on the Internet? A qualitative study using focus groups. J Med Internet Res 2003;5:e33. |
|7.||Sonnenberg FA. Health information on the Internet. Opportunities and pitfalls. Arch Intern Med 1997;157:151-2. |
|8.||Ademiluyi G, Rees CE, Sheard CE. Evaluating the reliability and validity of three tools to assess the quality of health information on the Internet. Patient Educ Couns 2003;50:151-5. |
|9.||Kisely S, Ong G, Takyar A. A survey of the quality of web based information on the treatment of schizophrenia and Attention Deficit Hyperactivity Disorder. Aust N Z J Psychiatry 2003;37:85-91. |
|10.|| Reavley NJ, Jorm AF. The quality of mental disorder information websites: A review. Patient Educ Couns 2011;85:16-25 |
|11.||Charnock D, Shepperd S, Needham G, Gann R. DISCERN: An instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health 1999;53:105-11. |
|12.||Montoya A, Llopis N, Gilaberte I. Validation of the translation of an instrument to measure reliability of written information on treatment choices: A study on attention deficit/hyperactivity disorder (ADHD). Educ Health (Abingdon) 2011;24:577. |
|13.||NETPROVIDER. Ranking de buscadores de internet en España (2007). Available from: NetProvider ( http://www.netprovider.es/ranking_buscadores.asp ) [Last accessed on 2009 Jan]. |
|14.||Sperber AD. Translation and validation of study instruments for cross-cultural research. Gastroenterology 2004;126(1 Suppl 1): S124-8. |
|15.||Hellwig-Brida S, Mangold S, Goldbeck L. The parent questionnaire "ADHD-knowledge and motivation for treatment" - development and first results. Z Kinder Jugendpsychiatr Psychother 2009;37:441-9. |
|16.||Fleiss JL, Levin B, Paik MC. The measurement of inter-rater agreement. Statistical methods for rates and proportions. Hoboken, NJ: John Wiley and Sons, Inc.; 2003. p. 235-8, p. 598-626. |
|17.||Roshan A, Agarwal S, England RJ. Role of information available over the internet: What are the parents of children undergoing tonsillectomy likely to find? Ann R Coll Surg Engl 2008;90:601-5. |
|18.||Fox S, Rainie L. The Online Health Care Revolution: How the Web Helps Americans Take Better Care of Themselves. Washington, DC: Pew Charitable Trusts; 2000. |
|19.||Godolphin W, Towle A, McKendry R. Evaluation of the quality of patient information to support informed shared decision-making. Health Expect 2001;4:235-42. |
|20.||Rees CE, Ford JE, Sheard CE. Evaluating the reliability of DISCERN: A tool for assessing the quality of written patient information on treatment choices. Patient Educ Couns 2002;47:273-5. |
|21.||Griffiths KM, Christensen H. The quality and accessibility of Australian depression sites on the World Wide Web. Med J Aust 2002;176(Suppl):S97-104. |
|22.||Berland GK, Elliott MN, Morales LS, Algazy JI, Kravitz RL, Broder MS, et al. Health information on the Internet: Accessibility, quality, and readability in English and Spanish. JAMA 2001;285:2612-21. |
[Table 1], [Table 2], [Table 3]
|This article has been cited by|
||Where to find accurate information on attention-deficit hyperactivity disorder? A study of scientific distortions among French websites, newspapers, and television programs
| ||Sébastien Ponnou,Héloïse Haliday,François Gonon |
| ||Health: An Interdisciplinary Journal for the Social Study of Health, Illness and Medicine. 2020; 24(6): 684 |
|[Pubmed] | [DOI]|
||How parents use, search for and appraise online health information on their childæs medical condition: A pilot study
| ||Benedicta Benedicta,Patrina HY Caldwell,Karen M Scott |
| ||Journal of Paediatrics and Child Health. 2019; |
|[Pubmed] | [DOI]|
||Online Information-Seeking Behaviors of Parents of Children With ADHD
| ||Adam Sage,Delesha Carpenter,Robyn Sayner,Kathleen Thomas,Larry Mann,Sandy Sulzer,Adrian Sandler,Betsy Sleath |
| ||Clinical Pediatrics. 2018; 57(1): 52 |
|[Pubmed] | [DOI]|
||Avaliação da qualidade da informação em sites de tuberculose: análise de uma experiência participativa
| ||Rodolfo Paolucci,André Pereira Neto,Rafaela Luzia |
| ||Saúde em Debate. 2017; 41(spe): 84 |
|[Pubmed] | [DOI]|
||Seeking Web-Based Information About Attention Deficit Hyperactivity Disorder: Where, What, and When
| ||Sara Rosenblum,Elad Yom-Tov |
| ||Journal of Medical Internet Research. 2017; 19(4): e126 |
|[Pubmed] | [DOI]|
||Evaluation of an educational website for parents of children with ADHD
| ||Gemma Sinead Ryan,Munib Haroon,Gail Melvin |
| ||International Journal of Medical Informatics. 2015; 84(11): 974 |
|[Pubmed] | [DOI]|
||Acute posttraumatic stress symptoms but not generalized anxiety symptoms are associated with severity of exposure to war trauma: A study of civilians under fire
| ||Liat Helpman,Avi Besser,Yuval Neria |
| ||Journal of Anxiety Disorders. 2015; 35: 27 |
|[Pubmed] | [DOI]|
||Circumcision on the web: A comparison of quality, content, and bias online
| ||Karl Coutinho,Kristian Stensland,Grace Hyun |
| ||Journal of Pediatric Urology. 2014; 10(4): 688 |
|[Pubmed] | [DOI]|