Test-Retest Reproducibility of the WebCas Electronic Questionnaire

for Assessing Health-Related Behaviors in Adolescents

Rosimeide Francisco dos Santos Legnani*

legnanirosi@gmail.com

Elto Legnani**

legnanielto@gmail.com

Eva Luziane Denkewicz Gustave***

evadenkewicz@yahoo.com.br

Rafael Vieira Martins+

rafa_rvm@yahoo.com.br

elianebacil@hotmail.com

wagner-campos@hotmail.com

*Profa. Adjunta da Universidade Estadual de Ponta Grossa/PR (UEPG)

Doutorado em Educação Física pela Universidade Federal do Paraná (UFPR)

Mestrado em Educação Física pela Universidade Estadual de Londrina

Especialista em Fisiologia do Exercício pela UFPR

Graduado em Licenciatura em Educação Física

pela Universidade de Passo Fundo/RS e Faculdades de Palmas/PR,

atual Instituto Federal do Paraná (IFPR), Palmas/PR

Membro do GPAFE-Tec (UTFPR - Curitiba)

**Professor Titular da Universidade Tecnológica Federal do Paraná (UTFP)

Graduado em Educação Física pela Universidade Estadual do Oeste do Paraná

Especializado em Treinamento Desportivo

pela Universidade Federal do Rio de Janeiro (UFRJ)

Especializado em Avaliação da Performance Motora

pela Universidade Estadual de Londrina (UEL)

Mestrado em Educação Física pela Universidade Federal de Santa Catarina

Doutorado em Educação Física pela UFPR

Atua no ensino da graduação (Educação Física)

Orienta nos Programas de Pós-Graduação em Engenharia Biomédicas

e Educação Física da Universidade Tecnológica Federal do Paraná

Líder do grupo de pesquisa Gpafetec

***Mestre em Educação Física na área de saúde e qualidade de vida

pela Universidade Tecnológica Federal do Paraná

Bacharel em Educação Física pela Universidade Cesumar (Unicesumar)

Licenciada em Educação Física pela Universidade Estadual de Ponta Grossa

Instrutora do método Pilates pelo grupo Voll

Pós-graduação latu sensu em Educação Inclusiva

pela Faculdade Batista de Minas Gerais

Graduação em Licenciatura em Educação Física

pela Universidade Estadual de Ponta Grossa

+Graduado em Educação Física pela UFPR

Mestrado em Educação Física pela UFPR

Especializado em Fisiologia do Exercício na Uninter-Curitiba

Doutorando no Programa de Pós graduação em Educação Física da UFPR

Atua como professor de Educação Física na Prefeitura Municipal de Curitiba

É sócio-proprietário e professor de Dança de Salão

do Espaço de Dança R&G-Curitiba

Membro do Centro de Estudos em Atividade Física e Saúde (CEAFS - UFPR)

e do Grupo de Pesquisa em Atividade Física, Esporte

e Tecnologia (GPAFETec - UTFPR)

++Pós doutora em Educação Física

na Área de Atividade Física e Saúde (2023) pela UFPR

Doutora em Educação Física na Área de Atividade Física e Saúde pela UFPR

Mestre em Saúde Pública na Área de Epidemiologia

pela Universidade Federal de Santa Catarina (UFSC)

Especialista em Educação Física

na Área de Atividade Física e Saúde UFSC

Licenciada Plena em Educação Física

pela Universidade Federal de Pelotas (UFPEL)

Atualmente é professora do Departamento de Educação Física

da Universidade Estadual do Centro- Oeste (UNICENTRO)

Integrante do Grupo de Pesquisa em Atividade Física e Saúde (CEAFS/UFPR),

do Grupo de Pesquisa em Atividade Física e Saúde Pública (FAMed/FURG)

e do Grupo de Pesquisa em Mecânica e Energética

do Movimento Humano (MEMH/UNICENTRO)

+++Licenciado em Educação Física pela UEL

Mestrado e Doutorado e Estágio Pós-Doutoral (2005)

na área do Desenvolvimento Motor e Estudos do Esporte

realizados na Universidade de Pittsburgh, EUA

Professor titular na Universidade Federal do Paraná

(Brasil)

Reception: 07/21/2024 - Acceptance: 09/07/2024

1st Review: 09/03/2024 - 2nd Review: 09/04/2024

Accessible document. Law N° 26.653. WCAG 2.0

Suggested reference: Legnani, R.F. dos S., Elto Legnani, Gustave, E.L.D., Martins, R.V., Bacil, E.D.A., & Campos, W. de (2024). Test-Retest Reproducibility of the WebCas Electronic Questionnaire for Assessing Health-Related Behaviors in Adolescents. Lecturas: Educación Física y Deportes, 29(318), 67-82. https://doi.org/10.46642/efd.v29i318.7793

 

Abstract

    Purpose: The goal of the study was to evaluate the reproducibility between the application-replication of the electronic version of the WebCas questionnaire. Methods: The sample consisted of 152 students (10 to 15 years old) of both sexes, enrolled in a public school in the city of Curitiba (PR). Anthropometric data (weight and height) were collected to calculate the body mass index. The reproducibility of the questionnaire was assessed by means of concordance between the repeated measures obtained by applying the WebCas, with a 7-day interval between applications. Results: Most of the variables showed percentages of reproducibility above 50% (7/12 variables). Eight variables presented Kappa values considered “very good”: commuting to school (K=0.95); alcohol intake in the last 30 days (K=0.92); alcohol intake in doses (K=0.92); smoking (K=0.93); intoxication (K=0.92), nap (K=0.83), illness (K=0.94) and medication use (K=0.90). Daytime sleepiness, frequency of consumption of fruits, vegetables, sweets, snacks and soft drinks showed “moderate” Kappa values (0.41 to 0.65). Regarding accuracy, the variables hours of sleep, time to wake up on weekdays and bedtime on weekdays showed substantial values (>0.95 to <0.99). For the other variables, the values were almost perfect (≥0.99), with the exception of the variable “bedtime the night before” which showed low accuracy (Cb=0.90). Conclusion: The results showed that the WebCas questionnaire showed satisfactory evidence of reproducibility.

    Keywords: Questionnaire. Reproducibility. Teenagers. Health-related behaviors.

 

Resumo

    Objetivo: O objetivo do estudo foi avaliar a reprodutibilidade entre as réplicas de aplicação da versão eletrônica do questionário WebCas. Métodos: A amostra foi constituída por 152 escolares (10 a 15 anos) de ambos os sexos, matriculados em uma escola pública da cidade de Curitiba (PR). Foram coletados dados antropométricos (peso e estatura), para cálculo do índice de massa corporal. A reprodutibilidade do questionário foi avaliada por meio concordância entre as medidas repetidas obtidas pela aplicação do WebCas, com 7 dias de intervalo entre as aplicações. Resultados: A maioria das variáveis apresentaram indicadores de reprodutibilidade acima de 50% (7/12 variáveis). Oito variáveis apresentaram valores de Kappa considerados “muito bons”: deslocamento para a escola (K=0,95); ingestão de álcool nos últimos 30 dias (K=0,92); ingestão de álcool em doses (K=0,92); tabagismo (K=0,93); intoxicação (K=0,92), cochilo (K=0,83), doenças (K=0,94) e uso de medicamentos (K=0,90). A sonolência diurna, frequência de consumo de frutas, verduras, doces, salgadinhos e refrigerantes apresentaram valores “moderados” de Kappa (0,41 a 0,65). Em relação à acurácia, as variáveis ​​horas de sono, hora de acordar em dias de semana e hora de dormir em dias de semana apresentaram valores substanciais (>0,95 a <0,99). Para as demais variáveis, os valores foram quase perfeitos (≥0,99), com exceção da variável “hora de dormir da noite anterior” que apresentou baixa acurácia (Cb=0,90). Conclusão: Os resultados demonstraram que o questionário WebCas apresentou evidências de reprodutibilidades satisfatórias.

    Unitermos: Questionário. Reprodutibilidade. Adolescentes. Comportamentos relacionados com a saúde.

 

Resumen

    Objetivo: El objetivo del estudio fue evaluar la reproducibilidad entre replicaciones de aplicaciones de la versión electrónica del cuestionario WebCas. Métodos: La muestra estuvo compuesta por 152 estudiantes (de 10 a 15 años) de ambos sexos, matriculados en una escuela pública de la ciudad de Curitiba (PR). Se recogieron datos antropométricos (peso y talla) para calcular el índice de masa corporal. La reproducibilidad del cuestionario se evaluó mediante la concordancia entre mediciones repetidas obtenidas mediante la aplicación de WebCas, con un intervalo de 7 días entre aplicaciones. Resultados: La mayoría de las variables presentaron indicadores de reproducibilidad superiores al 50% (7/12 variables). Ocho variables presentaron valores Kappa considerados “muy buenos”: desplazamiento a la escuela (K=0,95); consumo de alcohol en los últimos 30 días (K=0,92); ingesta de alcohol en dosis (K=0,92); fumar (K=0,93); intoxicación (K=0,92), siestas (K=0,83), enfermedades (K=0,94) y uso de medicamentos (K=0,90). La somnolencia diurna, la frecuencia de consumo de frutas, verduras, dulces, snacks y refrescos presentaron valores Kappa “moderados” (0,41 a 0,65). En cuanto a la precisión, las variables horas de sueño, tiempo para despertarse entre semana y tiempo para dormir entre semana presentaron valores sustanciales (>0,95 a <0,99). Para las demás variables, los valores fueron casi perfectos (≥0,99), a excepción de la variable “hora de dormir de la noche anterior” que mostró baja precisión (Cb=0,90). Conclusión: Los resultados demostraron que el cuestionario WebCas presentó evidencia de reproducibilidad satisfactoria.

    Palabras clave: Cuestionario. Reproducibilidad. Adolescentes. Comportamientos relacionados con la salud.

 

Lecturas: Educación Física y Deportes, Vol. 29, Núm. 318, Nov. (2024)

---

 

Introduction 

 

    During adolescence, several physiological and behavioral changes occur in young people (Mountjoy et al., 2011). This is a phase where they seek for autonomy and decision-making power, which can lead them into adopting behaviors that can have negative impacts to their health. (Mountjoy et al., 2011; Spear, & Kulbok, 2001). In this perspective, it is extremely important that healthy habits are encouraged from childhood and adolescence so that these positive behaviors be relevant in fighting negative effects to health such as the development of chronic or degenerative diseases. (Bezerra et al., 2021; Marques, Loureiro, & Avelar-Rosa, 2020)

 

    In the last years, scientific studies have shown that main behaviors help young people to keep adequate health levels, of which we can highlight: the practice of physical activity, reduced time spent in sedentary activities, adequate eating and sleeping habits, and non-contact with alcoholic beverages and tobacco. (Agostinis-Sobrinho, Oliveira, & Kievišienė, 2021)

 

    The interest of studying the behaviors that result in healthy outcomes for the young population has led to the creation of several instruments by the scientific community to assess such patters (Farias Junior et al., 2010; 2012; Pires, Pires Junior, & Oliveira, 2014). Among those various instruments, paper questionnaires had greater prominence. Through these questionnaires, the researcher can evaluate many individuals, however, they still require additional work in data entry and analysis. (De Jesus et al., 2017; Legnani et al., 2013)

 

    In that regard, many technological resources have emerged to support scientific research, such as electronic questionnaires. These instruments have been providing greater agility, by enhancing collections and research data storage, optimizing time, and, mainly, being better accepted by young population. (De Jesus. De Assis, & Kupek, 2017; De Jesus et al., 2016)

 

    Technological tools that replace paper questionnaires such as web questionnaires are very important in assisting researchers in evaluating healthy behaviors during adolescence especially in large scale samples (Legnani et al., 2020). The electronic version of the questionnaire can be more attractive the adolescents and to facility in its application, as well as provide more accurate and reliable results, in addition to saving paper and data typing (De Jesus, De Assis, & Kupek, 2017; De Jesus et al., 2016; Legnani et al., 2020)

 

    As a way of proposing an instrument that could help researchers to investigate health-related behaviours in Brazilian schoolchildren from the age group between 9-15 years old, previous studies were carried out to develop the WebCas questionnaire (Legnani et al., 2017, 2020). The printed version (Legnani et al., 2019), was tested and validated in Brazil with samples only and presented satisfactory reproducibility for all indicators, categorical and continuous variables. Recently the electronic version (Legnani et al., 2020), was made available (www.legnaniwebcas.com.br). So, the aim this study was assessed the reproducibility of the electronic version (EV) of the WebCas questionnaire (WebCas).

 

Methods 

 

    The target population of the study included students from the 5th to the 9th year of elementary school, duly enrolled in public and private, in the city of Curitiba, Paraná. The sample consisted of 152 students aged between 10 and 15 years of both sexes, selected intentionally, enrolled in the daytime period of a public school in the urban area of Curitiba, Paraná. The number of subjects necessary to analyze the reproducibility of the questionnaire was calculated a priori, taking into account type I error α=0.05, type II error 1-β=0.95, and effect size of 0.30, medium effect size (Faul et al. 2007), being established a minimum number for the sample of 143 students, plus 30% for sample losses, totalizing a sample of 186 students. Initially, five classes were selected (±40 students per class).

 

    Data collection was carried out between September and October (cross-sectional survey) by a trained team from the Center for Physical Activity and Health Studies at the Federal University of Paraná (UFPR), supervised by the main researcher. The study was approved by the Ethics Committee in Research with Human Beings of UFPR, under the opinion number: 684.147/2014 of June 11, 2014.

 

    To fill in the EV WebCas a computer room at the school was used with computers connected to the internet. Data collection was carried out in three stages. A previous visit to the school was carried out in order to deliver the two versions of the Free and Informed Term of Consent (FITC), one for the children’s parents or guardians, and the other for the schoolchildren over the age of 12 years old, the FITC was read and confirmed directly at www.legnaniwebcas.com.br. Then the students registered their personal data and received a login and password to enter the EV WebCas. This first contact with the questionnaire was considered as an “environment” to the questionnaire.

 

    In the second stage, the instrument reproducibility was tested upon the application of the WebCas questionnaire at two different moments (repeated measures), observing a 7-day interval between applications, with the same methodological criteria being used in both collections (Legnani et al., 2020). Both procedures were performed by the main researcher, assisted by two researchers previously trained to help the students in relation to filling it out. The average time to answer the EV WebCas was 21 minutes. From the schoolchildren invited, 152 returned and signed the FITC, and answered the questionnaire in both applications.

 

    The Webcas electronic questionnaire used in this study is part of the transposition from the paper version to the EV WebCas version (Legnani et al., 2020), keeping all its original structures. This instrument was prepared from the combination of various instruments, resulting in a questionnaire with seven sections: 1) anthropometric and sociodemographic variables; 2) duration of sleep the night before; 3) practice of physical activity (recall of the previous day), enabling the calculation of energy expenditure (EE), represented by a continuous variable in metabolic equivalent (MET), measured in kilocalories per day (Kcal/day), transformed into a categorical variable (in the case of Physical Activity Level (PAL) classifying students into: sedentary, little active, active and very active(ref)); 4) and 5) refer to locomotion habits (home/school/home), sleeping and eating habits, using the last week as a reference; 6) Sleep habits were investigated with questions related to: daytime sleepiness, nap or nap, time you usually wake up and sleep; 7) information regarding the frequency of alcohol consumption in the last 30 days, number of drinks per occasion and cigarette consumption for the seven days prior to the survey.

 

    The socioeconomic level was evaluated following the recommendations of the Brazilian Association of Research Companies (ABEP, 2014), using the classification criterion; ownership of locomotives and parent’s educational level, which classified students into classes A, B1, B2, C1, C2, D and E. For this study, the samples were stratified into three classes: high (A+B1); medium (B2 + C1 + C2); and low (D + E). The stages of elaboration and validation of the WebCas questionnaire, as well as the validity procedures of the PA measures of the WebCas questionnaire were presented in previous studies (Legnani et al., 2019, 2020). Body mass was collected with a digital scale (plena brand) with precision of 100 grams. Height was measured with a measuring tape attached to the wall and a wood support placed on the upper part of the skull (Vertex). (Guedes, & Guedes, 1995)

 

Statistical procedures 

 

    For sample description, descriptive analysis procedures were used frequency distribution, as well as the (Kolmogorov-Smirnov) normality test. The reproducibility and measurement agreement were assessed by the adjusted Kappa statistic test were performed in the PABAK-OS18 spreadsheet OS Calculator (Vannest et al., 2016). The categorical variables were analyzed by the percent of agreement and the Kappa index, considering the following criteria: poor: Kappa values <0.20; weak: 0.21-0.40; moderate: 0.41-0.60; good; Kappa of 0.61-0.80, and very good for values of 0.81-1.0018.

 

    Continuous variables were analyzed using the Concordance Correlation Coefficient (McBride, 2005) (CCCρc) technique between the two applications of the WebCas questionnaire. The CCCρc is composed of two measures: the first is the precision measure (ρ), which corresponds to Pearson's Correlation Coefficient; the second is the accuracy measure (Cb), which represents a bias and Correction factor. This accuracy measure assesses how far the best-fit line deviates from the 45° line passing through the origin. The values suggested for the interpretation of the Coefficients of Agreement (ρc) indicate the strength of agreement between the first application and the second application of the WebCas questionnaire (<0.90= poor; 0.90 - 0.95; moderate; 0.95 - 0.99, substantial and >0.99 = Nearly Perfect). In cases in which the variables did not present a normal distribution, data were normalized using the statistical program Medcalc 15.2 for Windows, adopting a significance level of 5% (p<0.05). The analysis was performed in the Medcalc program (version 15.2), considering the significance values p<0.05.

 

Results and discussion 

 

    The sample was composed in a balanced way in relation to gender and period of study, with the majority belonging to social classes "Medium" and "Low", two thirds were classified as sedentary and most of them declared that completing the EV WebCas it was easy and fun (Table 1).

 

Period	%	n
Morning	50.70	77
Afternoon	49.30	75
Sex
Boys	52.00	79
Girls	48.00	73
Social-Economic status
High	8.60	13
Medium	21.70	33
Low	68.70	106
Physical Activity Level
Sedentary	66.50	101
Little Active	13.80	21
Active	19.70	30
Very Active	0	0
Enjoyed answering the WebCas
Yes	94.10	143
No	5.90	9
Difficulty in answering the WebCas
Very difficult	7.90	12
Reasonably difficult	5.30	8
Easy	57.20	87
Very Easy	29.60	45

Source: Authors

 

    Among the 15 variables analyzed, six showed a very good Weighted Kappa value (> 0.70). Nine variables had a weighted kappa value between 0.40 and 0.70, considered good. The Kappa PABAK analysis showed that eight variables had very good Kappa values. The variables related to the frequency of food consumption, such as fruits, vegetables, sweets, snacks and soft drinks showed moderate Kappa values.

 

	Percent of agreement % (n)	Weighted Kappa* (IC: 95%)	Kappa PABAK** (IC: 95%)
School commuting	53.30 (81)	0.95 (0.91 - 0.99)	0.95 (0.88 - 1.01)
Fruits	38.80 (59)	0.61 (0.51 - 0.70)	0.50 (0.43 - 0.56)
Vegetables	40.80 (62)	0.67 (0.58 - 0.76)	0.57 (0.50 - 0.63)
Sweets	40.10 (61)	0.48 (0.37 a 0.58)	0.41 (0.34 - 0.48)
Snacks	50.70 (77)	0.49 (0.39 a 0.60)	0.46 (0.39 - 0.53)
Soft drinks	37.5 0(57)	0.50 (0.39 a 0.61)	0.42 (0.36 - 0.49)
Alcohol intake 30 days	91.40 (139)	0.57 (0.32 a 0.82)	0.92 (0.86 - 0.98)
Alcohol intake doses	92.80 (141)	0.72 (0.60 a 0.85)	0.92 (0.86 - 0.98)
Intoxication	90.10 (137)	0.72 (0.57 a 0.88)	0.92 (0.86 - 0.99)
Smoking	95.4 (145)	0.59 (0.28 a 0.89)	0.93 (0.86 - 1.00)
Daytime Sleepiness	57.90 (88)	0.62 (0.51 a 0.74)	0.65 (0.58 - 0.72)
Napping (siesta)	73.70 (112)	0.73 (0.61- 0.85)	0.83 (0.75 - 0.92)
Health perception	53.30 (81)	0.75 (0.66 - 0.84)	0.76 (0.69 - 0.83)
Illnesses	86.80 (132)	0.82 (0.68 - 0.96)	0.94 (0.86 - 1.01)
Drug use	92.10 (140)	0.58 (0.34 - 0.81)	0.90 (0.82 - 0.97)

*Analysis performed in MedCalc Statistical Program; **Analysis performed in the Pabak spreadsheet - OS Calculator; 

Percent of agreement refers only to the category with the highest percentage of events (frequency of food intake). 

Source: Authors

 

    About the continuous variables, it was observed that the values obtained between the EV WebCas application showed good reproducibility. The Coefficient Agreement Correlation (CAC) and precision (ρc) between the measures ranged from moderate to strong correlation. Regarding accuracy (ρc) measure between two applications EV WebCas was almost perfect for most variables (Cb=0.98).

 

	ρc	Cb	CAC	IC 95%
Time woke up today (1x2)	0.70	0.99	0.70	0.61 - 0.77
Time went to sleep last night (1x2)	0.31	0.90	0.31	0.16 - 0.44
Sleeping hours (1x2)	0.45	0.98	0.45	0.31 - 0.56
Wake up time on Weekdays (1x2)	0.55	0.97	0.54	0.42 - 0.64
Sleeping time on weekdays (1x2)	0.32	0.98	0.32	0.17 - 0.45
Wake up time on weekends (1x2)	0.69	0.99	0.69	0.60 - 0.77
Sleeping time on weekends (1x2)	0.57	0.99	0.56	0.45 - 0.66
GEDA 1 x GEDA 2	0.45	0.98	0.45	0.31 - 0.56

GEDA=Energy Expenditure Previous Day; 1=first WebCas application; 

GEDA 2=Energy Expenditure Day 2=second WebCas application; 

ρc=error correction factor; Cb=accuracy; CAC=Correlation Agreement Coefficient. 

Source: Authors

 

    The objective of this study was to evaluate the reproducibility of the EV WebCas. Of the 15 variables analyzed, six showed a percentage of agreement greater than 90%, six variables had a very good Weighted Kappa value (>0.70). Nine variables had a weighted kappa value between 0.40 and 0.70, considered good. The Kappa PABAK analysis showed that eight variables had very good Kappa values. The variables related to the frequency of food consumption, such as fruits, vegetables, sweets, snacks and soft drinks showed moderate Kappa values. In this context, it was observed that the EV WebCas presented indicators of agreement of moderate to strong intensity for most of the variables analyzed.

 

    International evidence shows that the use of electronic questionnaires in different population groups has been gaining prominence (De Jesus, De Assis, & Kupek, 2017; De Jesus et al., 2016; Legnani et al., 2013; 2020; Pires, Pires Junior, & Oliveira, 2014), mainly questionnaires to assess the practice of PA in children and adolescents and food consumption (Legnani et al., 2013; McBride, 2005; McLure et al., 2009; Parker, Vannest, & Davis, 2013; Tabacchi et al., 2015). From this perspective, the use of these instruments seems effective, as they have good internal consistency and stability, being viable for epidemiological research, mainly due to their low cost and wide coverage. (Casamali et al., 2019; Legnani et al., 2013; Tabacchi et al., 2014)

 

    In the reproducibility analysis of categorical variables, ten variables: transportation to school, frequency of consumption of fruits, vegetables, sweets, snacks, soft drinks, alcohol consumption, drunkenness, cigarette consumption, consumption for 30 days and nap presented almost perfect agreement, however, when considering the same variables from the perspective of the PABAK analysis, only six variables presented higher values, results that are in agreement with other studies that used similar methodological procedures (Jesus; Assis; Kupek, 2017; Skogen et al., 2013; Tabacchi et al., 2014). Studies to evaluate the food consumption of schoolchildren, similar to this one, were carried with schoolchildren from six countries Belgium, Denmark, Iceland, Norway, Portugal, Spain (Jesus, Assis, & Kupek, 2017; Skogen et al., 2013; Tabacchi et al., 2015) reported reproducibility indicators ranging from moderate to strong, and the authors considered that the reproducibility indicators presented were satisfactory.

 

    In the reproducibility analysis of the continuous variables of the EV WebCas, six of the eight variables presented substantial or almost perfect accuracy values, two of them presented moderate accuracy values. On the other hand, 62.5% of the variables showed correlation values between moderate and strong and two variables showed strong values of correlation. These indicators are slightly higher than the values found in the reproducibility study of the printed version of the WebCas questionnaire (Legnani et al., 2013). International evidence shows that the use of electronic questionnaires has been gaining prominence (Casamali et al., 2019; De Jesus, De Assis, & Kupek, 2017, De Jesus et al., 2016, McLure et al., 2009, Legnani et al., 2013; 2020, Pires, Pires Junior, & Oliveira, 2014), especially to assess outcomes on health, mainly, regarding food intake and regular physical activity (McBride, 2005; McLure et al., 2009; Legnani et al., 2013; Parker, Vannest, & Davis, 2013). In this perspective, the use of these instruments seems effective, as they show good internal consistency and stability, being viable for epidemiological research, especially for their low cost and wide coverage. (Legnani et al., 2013; Tabacchi et al., 2014)

 

    Therefore, when considering the methodological differences, as well as the different socioeconomic contexts between the studies and, above all, the different statistical procedures adopted, the EV WebCas showed acceptable reproducibility indicators similar to the indicators found in the scientific literature. On the other hand, it was found that the electronic version presented significantly better results than those found in the printed version, especially regarding the weighted Kapa, and Kapa PABAK accuracy (Cb) procedures.

 

    The convenience sample selection method, the number of schoolchildren participating in the study and the fact that data collection was carried out in a city in the southern region of Brazil can be considered the main limitations of the study. Therefore, it is recommended to carry out more studies related to the application of the electronic instrument involving students from different types of educational institutions and regions of the country.

 

    From another perspective, EV WebCas had some advantages over the printed version, as the student's previous contact with the platform (registration and setting) reduced the number of incomplete or erroneous questionnaires. Likewise, the completion of the questionnaire by the students, reduced costs of typing and printing forms, eliminating incomplete questionnaires, or typing errors and allowing immediate access to the collected data. Finally, after filling in the EV WebCas, the students get instantaneous feedback for their answers, and this tool can also be applied to group answers per school (general report), age and sex.

 

Conclusions 

 

    The EV WebCas showed satisfactory reproducibility values in most of the variables analyzed (categorical and continuous). In this context, EV WebCas can be considered a good instrument to be used in epidemiological and monitoring research, as it is an easy-to-use, low-investment tool. Furthermore, we emphasize that this instrument has the proper parameters to be used in research aiming at to assess health risk behaviors in children and adolescents of 10-15 years old from the southern region of Brazil.

 

References 

 

ABEP - Associação Brasileira de Empresas de Pesquisa (2014). Critério de Classificação Econômica do Brasil. http://www.abep.org

 

Agostinis-Sobrinho, C., Werneck, AO, Kievisiene, J., Moreira, C., Ramírez-Vélez, R., Rosário, R., Norkiene, S., Lopes, L., Mota, J., & Santos, R. (2021). Estado de saúde cardiovascular ideal e qualidade de vida relacionada à saúde em adolescentes: estudo Labmed. Revista Paulista de Pediatria, 39, 1-7. https://doi.org/10.1590/1984-0462/2021/39/2019343

 

Bezerra, MKA, Carvalho, EF, Oliveira, JS, Cesse, EÂP, Lira, PIC, Cavalcante, JGT, Leal, VS, & Santos, MP (2021). Lifestyle of adolescent students from public and private schools in Recife: ERICA. Ciências e Saúde Coletiva, 26(1), 221-32. https://doi.org/10.1590/1413-81232020261.34622018

 

Casamali, F.F.C., Schuch, F.B., Scortegagna, S.A., Legnani, E., & De Marchi, A.C.B. (2019). Accordance and reproducibility of the electronic version of the WHOQOL-BREF and WHOQOL-OLD questionnaires. Experimental gerontology. 125, 110683. https://doi.org/10.1016/j.exger.2019.110683

 

De Jesus, G.M., De Assis, M.A.A., & Kupek, E. (2017). Validade e reprodutibilidade de questionário baseado na internet (Web-CAAFE) para avaliação do consumo alimentar de escolares de 7 a 15 anos. Caderno de Saúde Pública, 33(5), 1-16. https://doi.org/10.1590/0102-311X00163016

 

De Jesus, G.M., De Assis, M.A.A., & Kupek, E. (2016). Avaliação da atividade física de escolares com um questionário via internet. Revista Brasileira de Medicina do Esporte, 22(4), 261-6. https://doi.org/10.1590/1517-869220162204157067

 

De Jesus, G.M., De Assis, M.A.A., Kupek, E., & Dias, L.A. (2017). Avaliação da consistência de valores de peso, altura e índice de massa corporal no questionário consumo alimentar e atividade física de escolares: O controle da qualidade da entrada de dados no sistema. Revista Brasileira de Epidemiologia, 20(4), 573-82. https://doi.org/10.1590/1980-5497201700040002

 

Farias Junior, J.C., Lopes, A.S., Florindo, A.A., & Hallal, P.C. (2010). Validity and reliability of self-report instruments for measuring physical activity in adolescents: a systematic review. Caderno de Saúde Pública, 26(9), 1669-91. https://doi.org/10.1590/S0102-311X2010000900002

 

Farias Junior, J.C., Lopes, A.S., Mota, J., Santos, M.P., Ribeiro, J.C., & Hallal, P.C. (2012). Validade e reprodutibilidade de um questionário para medida de atividade física em adolescentes: uma adaptação do Self-Administered Physical Activity Checklist. Revista Brasileira de Epidemiologia, 15(1), 198-210. https://doi.org/10.1590/S1415-790X2012000100018

 

Faul, F., Erdfelder, E., Lang, A.G., & Buchner, A. (2007). G* Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior research methods, 39(2), 175-191. https://link.springer.com/article/10.3758/BF03193146

 

Guedes, D.P., & Guedes, J.E.R.P. (1995). Atividade física, aptidão física e saúde. Revista Brasileira de Atividade Física & Saúde, 1(1), 18-35. https://doi.org/10.12820/rbafs.v.1n1p18-35

 

Legnani, E., Legnani, R.F.S., Rech, C.R., Guimaraes, R.F., & De Campos, W. (2013). Instrumentos eletrônicos para avaliar atividade física em crianças: Uma revisão sistemática. Motricidade, 9(4), 90-9. http://dx.doi.org/10.6063/motricidade.9(4).594

 

Legnani, R.F.S., Camargo, E., Legnani, E, Bacil, E. Neto, N.M., & Campos, W. (2017). Students’ and teachers’ understanding of the WebCas questionnaire for assessing health related behaviors. Revista Brasileira de Atividade Física e Saúde, 22(3), 251-260. https://doi.org/10.12820/rbafs.v.22n3p251-260

 

Legnani, R.F.S., Legnani, E., Andrade, T.A., Martins, R.V., Gustave, E.L.D., & Campos, W. (2020). WebCas electronic questionnaire to evaluate health-related behaviors of schoolchildren. Motriz Revista Educação Física, 26(3), 1-5. https://doi.org/10.1590/s1980-6574202000030041

 

Legnani, R.F.S., Legnani, E., Quentino, R.A., Pereira, M.S., Bacil, E., & Campos, W. (2019). Reprodutibilidade da versão impressa do Questionário Webcas. Adolescência & Saúde, 16(4748), 7-15. https://www.researchgate.net/publication/334683825

 

Marques, A., Loureiro, N., Avelar-Rosa, B., Matos, A. de, & Gaspar, M. (2020). Adolescents healthy lifestyle. Journal of Pediatric, 96(2), 217-24. https://doi.org/10.1016/j.jped.2018.09.002

 

McBride, G.B. (2005). A proposal for strength-of-agreement criteria for Lin’s Concordance Correlation Coefficient. NIWA Client Report: HAM2005-062. https://api.semanticscholar.org/Corpus ID:125225327

 

Mclure, S.A., Reilly, J.J., Crooks, S., & Summerbell, C.D. (2009). Development and evaluation of a novel computer-based tool for assessing physical activity levels in schoolchildren. Pediatric Exercise Science, 21(4), 506-19. https://doi.org/10.1123/pes.21.4.506

 

Mountjoy, M., Andersen, LB, Armstrong, N., Biddle, S., Boreham, C., Bedenbeck, HPB, Ekelund, U, Engebretsen, L., Hardman, K., Hills, AP, Kahlmeier, S., Kriemler, S., Lambert, E., Ljungqvist, A., Matsudo, V., Mckay, H., Micheli, L., Pate, R., Riddoch, C., Schamasch, P., Sundberg, CJ, Tomkinson, G., Van Sluijs, E., & Van Mechelen, W. (2011). International Olympic Committee consensus statement on the health and fitness of young people through physical activity and sport. British Journal of Sports Medicine, 45(11), 839-848. https://doi.org/10.1136/bjsports-2011-090228

 

Parker, R., Vannest, K., & Davis, J. (2013). Inter-rater reliability for multi-category scales: problems and solution in PABAK-OS. Journal of School Psychology, 51(2), 217-29. https://doi.org/10.1016/j.jsp.2012.12.003

 

Pires, A.A.P.P., Junior, R.P., & De Oliveira, R.F. (2014). Concordância entre os formatos impresso e eletrônico do IPAQ-L. Revista Brasileira Medicina do Esporte, 20(6), 474-9. https://doi.org/10.1590/1517-86922014200602134

 

Skogen, J.C., Boe, T., Knudsen, A.K., & Hysing, M. (2013). Psychometric properties and concurrent validity of the CRAFFT among Norwegian adolescents. Ung@hordaland, a population-based study. Addict Behav, 38(10), 2500-2505. https://doi.org/10.1016/j.addbeh.2013.05.002

 

Spear, H.J., & Kulbok, P.A. (2001). Adolescent health behaviors and related factors: A review. Public health nursing, 18(2), 82-93. https://doi.org/10.1046/j.1525-1446.2001.00082.x

 

Tabacchi, J., Filippi, AR, Amodio, E., Napoli, G., Breda, J., Bianco, A., Jemni, M., Censi, L., Mammina, C., & De Jardim, T. (2014). The web-based ASSO-food frequency questionnaire for adolescents: Relative and absolute reproducibility assessment. Nutrition Journal, 13(1). https://doi.org/10.1186/1475-2891-13-119

 

Tabacchi, J., Filippi, AR, Breda, J., Censi, L., Amodio, E., Napoli, G., Bianco, A., Jemni, M., Firenze, A., & Mammina, C. (2015). Comparative validity of the ASSO-Food Frequency Questionnaire for the web-based assessment of food and nutrients intake in adolescents. Food Nutrition Research, 15(29), 1-15. https://doi.org/10.3402/fnr.v59.26216

 

Vannest, K.J., Parker, R.I., Gonen, O., & Adiguzel, T . (2016). Single Case Research: calculadoras baseadas na web para análise de SCR. (Versão 2.0) [Aplicativo baseado na web]. College Station, TX: Texas A&M; University. https://singlecaseresearch.org/calculators/

Lecturas: Educación Física y Deportes, Vol. 29, Núm. 318, Nov. (2024)