The Journal of School Nursing2024, Vol. 40(2) 135–143© The Author(s) 2021Article reuse guidelines:sagepub.com/journals-permissionsDOI: 10.1177/10598405211057588journals.sagepub.com/home/jsn
Data on school health policy implementation are limited due to the absence of a validated measurement tool. The purpose of this study was to create and pilot a school health policy implementation survey. A modified, four-round Delphi process was used to achieve consensus on content and format of the survey. The final 76-item survey was piloted in 655 schools with a return rate of 57.1% (n = 378). Seven schools participated in environmental audits. Based on the audits, survey responses represented an accurate description of school practices for 84.2% (n = 64) of questions. The remaining 15.8% (n = 12) of survey items were eliminated or revised. This measurement tool begins to fill the research gap between the evaluation of written school health policy and implementation. Further, this tool may be used by school nurses in alignment with the Framework for 21st Century School Nursing Practice.
Keywordsschool health, policy, delphi, survey development, framework for 21st century school nursing practice
Creating a healthier generation within the United States (U.S.) must include efforts that begin early in children’s lives. The school environment, where children spend a majority of their day, is an ideal setting to implement health and wellness strategies (Story et al., 2009). In 2004, Congress passed the Child Nutrition Act (CNA) and required all school districts participating in the National School Lunch Program to develop local school wellness policies by 2006 (The Child Nutrition and WIC Reauthorization Act of 2004). This act encouraged a number of school districts to pass local wellness policies, however, very few were strong enough to make an impact in health and wellness outcomes of students (Belansky et al., 2009). In 2010, Congress reauthorized the CNA and required school districts to implement, evaluate, and publicly report on the progress of their wellness policies (United States Senate, 2010). This was in response to several studies documenting the clear link between student participation in school-based wellness activities and academic success (Basch, 2011; Bradley & Greene, 2013; Lam, 2014; Michael et al., 2015; Rasberry et al., 2011).
Policy passage alone, especially in the school environment, does not guarantee policy implementation nor does it equate to practice changes among school staff as a wide variety of barriers exist (Cygan, et al., 2019; Schuler et al., 2018; Weatherson et al., 2017). Barriers to school health and wellness policy implementation include limited availability of school nurses, community resistance to breaking tradition and changing school culture, and inadequate budgets (Cygan et al., 2019). While barriers to implementation are documented, evaluation of policy implementation is limited in the literature.
School wellness policies have previously been evaluated in terms of policy language (Chriqui et al., 2013; Schwartz et al., 2012; Wall, et al., 2012); however, more studies of implementation of school health policies should be conducted to determine impact and effectiveness (Castelli et al., 2014; Cygan et al., 2019). Several tools exist to assess school-level policy needs; however, these assessment tools were not designed to evaluate policy implementation or impact on student health (Centers for Disease Control & Prevention [CDC], 2019). One study assessed school health policy implementation within an international setting (Pinto et al., 2016). However, literature on the assessment of school health policy implementation in the U.S. is limited, leaving a gap between the measurement of policy language or passage and evaluation of policy implementation and impact. One reason for this gap in the literature is the lack of a validated, school health policy implementation measurement tool.
Since 2013 the Chicago Public Schools (CPS) Office of Student Health and Wellness, which serves the third largest school district in the U.S., has been a national leader in school health and wellness policy. In 2016, the CPS Office of Student Health and Wellness created the Healthy CPS Initiative, a component of the mayor’s Healthy Chicago initiative that seeks to ensure that all residents have access to resources, opportunities, and environments that maximize health and well-being (Chicago Public Schools, 2016). Healthy CPS was created to help schools understand and implement health-related federal, state, and local policy requirements (Chicago Public Schools, 2016).
Monitoring of CPS’s health and wellness policy implementation includes annual administration of the Healthy CPS Survey. A version of this survey has been distributed to measure district-level policy alignment since 2013. The survey is completed by each school’s administrator or designee (Chicago Public Schools, 2016). While the Healthy CPS Survey has filled a gap in local policy implementation measurement, there remained a need for a policy measurement tool that was designed to be generalizable for consistent evaluation of school health and wellness policy implementation nationwide. The purpose of this study was to create a school health policy implementation survey, achieving consensus on survey content and format, through a Delphi study and pilot the tool within CPS schools.
Researchers in this study utilized a modified, four-round Delphi process to achieve consensus on the content and format of a school health policy implementation survey. The Delphi method is an iterative process used to collect anonymous judgements and reach consensus among a panel of content experts (Humphrey-Murto et al., 2017; Kennedy, 2004). The Delphi process in this study was modified by beginning the process with a set of predetermined items to facilitate collection of expert opinion. This type of modification is established as acceptable in the literature (Custer et al., 1999).
The Delphi method has been used in a variety of fields, including health care and education, to develop curriculum guidelines, professional standards, and to develop or validate measurement tools (Barr et al., 2014; Falzarano & Pinto Zipp, 2013; Marshall & Allegrante, 2017; Pinto et al., 2016). Further, the Delphi method has been used in school health to develop competency-based education for professionals caring for students with chronic conditions (Berget et al., 2019; Cicutto et al., 2017). This method is often used to come to a consensus on something that is unknown or to obtain a judgment about a specific complex topic, such as school health (Falzarano & Pinto Zipp, 2013).
The final survey developed through the Delphi process was piloted at CPS. A small sample of CPS schools participated in environmental audits. The purpose of the environmental audits was to compare self-reported survey responses with observed in-school policy implementation. This study was approved by the primary author’s Institutional Review Board and CPS Research Review Board.
Researchers used purposive sampling to invite school health policy experts from across the country and CPS to participate as members of the Delphi panel via e-mail and a professional listserv. Panelists were also invited to participate through secondary recruitment as potential panelists were asked to share the opportunity with qualified colleagues. Potential panelists received a study information sheet and application. Return of the completed application was considered passive consent to participate in the study. All applications and CVs were reviewed by a member of the research team to verify the following two inclusion criteria were met. One, they committed to completing at least four Delphi rounds over a one-year time period. Two, they had at least five years of experience in school health policy expertise, as evidenced by review of a CV or resume, in one of the following areas: school health policy, school administration, teaching or school nursing. All qualified panelists were sent an e-mail verifying their inclusion in the Delphi panel and an explanation of the phases of the project.
Participation incentives for the completion of each round became available and were disclosed to panelists after participant recruitment closed. Panelists were offered the following incentives: $20.00 gift card for completion of Round 1 by the due date, $20.00 gift card for completion of Round 2 by the due date, $30.00 gift card for completion of Round 3 by the due date, and a $40.00 gift card for completion of Round 4 by the due date. In addition, panelists who completed all four rounds were entered in a raffle to receive one of two $50.00 gift cards.
The schools recruited for environmental audits were selected using random, stratified sampling. All CPS schools were stratified into three categories based on the previous year’s Healthy CPS survey results, including: high, medium, or low implementation of health and wellness policies. Fifteen elementary schools (K – 8th grade) and eight high schools (9th–12th grade) were randomly selected from each category and invited to participate in an audit of their survey responses. Participation was voluntary and schools were offered $1,000 for completing the survey and allowing researchers to conduct an environmental audit to evaluate alignment between survey response and observed policy implementation.
The Delphi method is an inherently mixed methods research approach (Humphrey-Murto et al., 2017; Kennedy, 2004). Figure 1 illustrates the Delphi process of this study. The first round of data were collected and managed using REDCap (Harris et al., 2019). The purpose of Round 1 was to gain expert opinion on the topic areas to be included in the final survey, and the components of each topic area that should be addressed. Once the Round 1 data were received from the Delphi panelists, the research team members independently conducted qualitative content analysis, coding the Delphi panel members’ responses. After completing the independent coding, the research team met to discuss their coding to ensure that any discrepancies were resolved, and 100% agreement was reached regarding the higher-order and lower-order themes.
In Round 2 the results of the content analysis of Round 1 responses were disseminated to the expert panelists. The panelists were asked to rate the accuracy/clarity (1 “not at all clear” to 4 “very clear”) of the representation of their original comments and the importance (1 “not at all important” to 4 “very important”) of the components within each category. These data were collected to ensure the survey developed included the components deemed most important by the diverse expert panel. To simplify this round, the panelists were emailed an Excel spreadsheet of the themes and asked to rate and provide clarifying feedback when they deemed it necessary. Topics rated below a three on accuracy, clarity, or importance were reviewed by the research team. Researchers compared these lower scoring topics to national guidelines (Centers for Disease Control and Prevention [CDC], 2021) to determine inclusion, and revised wording for clarity.
A pool of survey items representing the topic areas were then developed by the research team. For Round 3, this pool of survey items was sent to the expert panelists via Qualtrics (Qualtrics, 2021). Panelists were asked to rank the importance (1 “not at all important” to 4 “very important”) of each item on the survey. The panelists also provided qualitative feedback regarding item type/format (e.g., multiple choice, multipleselection, open-ended), item wording and/or modification, and response options. Delphi panelists had the opportunity to provide comments regarding anything they felt had not been included that should have been, plus identify any items they felt were not needed. This feedback was reviewed by the research team to ensure the items (i.e., format, wording, and response options) were clear regardless of individuals’ geographical location and role in the school district. The feedback from the panelists informed revisions of items, as well as consolidation, separation, or removal of items from the survey. Items were removed if deemed repetitive or if they were not in alignment with national guidelines (CDC, 2021).
For Round 4, the revised survey was sent to the Delphi panel for a final qualitative review. The focus of this final round was to gather feedback regarding the overall survey and item-specific revisions before it was implemented in a pilot validation study. Qualitative feedback gathered from the expert panelists in this round focused primarily on clarity of questions and grammar. This feedback was incorporated into the development of the final survey. This survey was piloted at CPS in Spring 2019. The survey was distributed to all schools within the district (n = 655). A small sample of schools was invited to participate in an environmental audit to compare survey responses to observed behaviors and practices within each school.
One hundred percent (N = 20) of potential participants who responded to recruitment efforts met the inclusion criteria and were included in the study. Participants reported a range of 5–25 years of experience in the school health and wellness field (mean: 12.9 years). Participants reported a wide variety of careers and employers. Forty-five percent of participants (n = 9) worked in primary education, forty percent (n = 8) worked in health care, and the remaining fifteen percent (n = 3) worked in non-profit, public health roles. Of the nine primary educator participants, 67% (n = 6) were employed by CPS and 33% (n = 3) were employed by other school districts. Among participants, 5% (n = 1) had a bachelor’s degree, 65% (n = 13) had a graduate degree, and 30% (n = 6) had a doctoral degree.
The response rate for Round 1 was 95% (n = 19). Researchers used a predetermined set of items to solicit expert opinions. These items were in alignment with the School Health Guidelines set forth by the CDC (CDC, 2021). For example, panelists were asked, “On a health and wellness policy implementation survey, what is most important to include about student physical activity?” Panelists identified a total of 324 unique, possible survey components. Researchers categorized these components into 165 themes, within 16 topic areas. Table 1 shows the number of topic areas and themes identified for this, and subsequent, Delphi rounds.
Given the large number of unique responses provided by the expert panelists in Round 1, it was critical in Round 2 to gain their opinion on the accuracy of the Round 1 data analysis and the importance of including each component on the final survey. The Round 2 response rate was 90% (n = 18). The average accuracy ranking for all components (n = 324) was 3.52 (1 to 4 Likert scale). Ninety-eight percent (n = 319) of the individual components received an accuracy rating of over 3.0, supporting the accuracy of the researchers’ Round 1 qualitative analysis. The average importance ranking for all components was 3.47 (1 to 4 Likert scale). Eighty-nine percent (n = 289) of the individual components averaged more than 3.0 (out of 4), demonstrating that panelists found a large proportion of the identified components to be important or very important to include on the final survey. Due to the high importance rating for all components, researchers identified two needs: 1) inclusion of as many components as possible on the final survey, and 2) prioritization of components.
Based on Delphi panelists’ feedback from Round 1 and 2, researchers developed an 84-item pool across 12 topic areas, encompassing nearly all identified themes and in alignment with national guidelines (CDC, 2021). When similar to panelist feedback, items were modeled after the previous year’s
In several instances, items from different topic areas were combined to decrease the total number of items and topic areas on the survey. For example, several items in the topic area of LGBTQ student health were included in the sexual health education topic area. Health education items were moved into physical education or nutrition education. Items related to fundraisers were included in the school food and meal topic area. Researchers agreed to limit the survey to student health and wellness and therefore did not develop items related to employee wellness.
The purpose of Round 3 was to prioritize items to be included on the final survey and gather qualitative feedback on each item for the content and format of both question and response options. The response rate for this round was 80% (n = 16). Delphi panelists ranked items in order of importance within each of the topic areas, allowing researchers to understand which items were the most important to include on the final survey. Panelists offered suggestions for edits on 100% (n = 84) of the items. Suggestions included grammatical edits, rewording for clarity, inclusion or exclusion of response options, and changes in formatting (e.g., multiple choice vs. open-ended questions).
After examination and revision, researchers reached consensus on a revised survey with 64 items, across nine topic areas. To simplify the survey, researchers made organizational decisions that changed the number of topic areas and items included. For example, several items in physical activity and physical education were combined and placed under the physical activity topic heading, eliminating the physical education topic area all together. Items from the topic areas of fundraisers, reward and punishment, nutrition education, and school meals and food were combined and included in the school meals and food topic area. Items originally under identification of chronic conditions were moved under management of chronic conditions.
The final 64-item survey was then sent out to the expert panelists in Round 4 to gain qualitative input on the clarity and readability of each item. The response rate in this round was 80% (n = 16). Based on suggestions of the Delphi panelists, researchers clarified several items by separating them into two or more items, and in some cases recategorizing items under different topic areas. The final result was a 76-item school health and wellness implementation policy survey.
The final survey, developed as a result of four Delphi rounds, was piloted at CPS in Spring 2019. The survey was sent to 655 schools, with a return rate of 57.1% (n = 378). Once completed, 23 randomly selected schools were invited to participate in environmental audits to assess how well their survey responses reflected practices within the school. Among invited schools, 30% (n = 7) agreed to participate in the audits. Based on these audits, researchers determined that the survey responses represented an accurate description of school policy implementation for 84.2% (n = 64) of questions.
For the 15.7% (n = 12) of survey items where discrepancies were identified in more than 50% of environmental audits, plans were made to either eliminate questions or adapt question wording to make the survey more districtspecific moving forward. For example, one question asked, “How many lessons of health education were taught during class time throughout the school year?” While this question was identified as important by the Delphi panel, it was not found to be relevant to CPS and was therefore eliminated from future CPS surveys. The question was not included in the CPS survey because there was not a policy in place the specified the number of health education lessons required per grade. Because of this there were no clear criteria for what constituted a health education lesson in this context, nor was there a procedure in place to track the number of lessons taught. While this specific question did not reflect the policies in place at CPS, it was included in the final survey to increase generalizability of the tool to other school districts. The final Healthy CPS Survey is included as a supplement to this article. It should be noted that due to survey software branching logic, several questions were revised to be single questions with two or more parts. This changed the overall number of questions on the survey but did not change the content.
The purpose of this study was to create a school health policy implementation survey to measure health and wellness policy implementation in schools nationwide. Given that most measures in school health policy assess only policy language and passage (Chriqui et al., 2013), there is a need for a tool school districts can use to effectively measure policy implementation. We utilized a Delphi process to achieve consensus on survey content. The final survey, piloted at CPS in Spring 2019, included 76 items across a wide spectrum of school health and wellness topics.
In the first Delphi round, panelists identified over three hundred unique, potential survey components. This simultaneously highlighted the wide variety of school health and wellness practices in place across the United States (Bryan et al., 2021) and the importance of prioritizing survey items based on both expert opinion, and in alignment with established best practices, such as the School Health Guidelines and emerging literature (CDC, 2021; Long et al., 2020). In this study, researchers asked panelists to prioritize survey components twice and reviewed current guidelines to develop a survey that aligned with both expert opinion and national standards. This approach ensured the final survey is one that will allow districts nationwide to determine success in implementing policies that align with nationally recognized best practices. By developing one survey for use across districts, uniform data collection based on national guidelines is possible and may be used to drive change at the district, state or federal levels.
Use of this survey will also allow school districts to assess the translation of health and wellness policies into practice. This is the essential first step to a district-wide quality improvement process in which school administrators can utilize data to respond to gaps within their schools’ health and wellness practices. While a quality improvement approach to assess policy implementation is not novel, reports of quality improvement for school health policy are limited in the literature and often focused on specific programs, rather than overarching policies (Klostermann et al., 2000; Mickel et al., 2017). Use of this survey as a data collection tool to drive quality improvement projects specificto health and wellness policy implementation should be explored.
By using this survey to measure school health and wellness policy implementation, school administrators can identify areas in which school-level practice does not align with national guidelines. In response, they may work with school nurse and other school health leaders to develop and prioritize strategies to improve policy implementation in identified areas. School health initiatives are most effective when supported by school administrators (Lee & Welk, 2021). In collaboration with school nurses and school health educators, administrators are well-positioned to support policy agendas that will positively impact student health outcomes.
Although national school health guidelines and the need for a uniform data collection tool are established (Davis et al., 2019), it is important to note that effective school health policies reflect the needs and culture of the district. Because of this, some variation in health and wellness policies across districts is expected and it may be necessary to adapt this survey to allow for meaningful data collection. In fact, the poor performance of several items on the survey during environmental audits serves as a reminder that school districts may need to adapt this tool to fit districtspecific characteristics. This is an area in which school nurses can play an important role. Working in alignment with the Framework for 21st Century School Nursing PracticeTM school nurses understand policy development and implementation (National Association of School Nurses [NASN], 2016). Further, school nurses act as public health professionals who understand the needs of the district and school community (NASN, 2016). This knowledge, coupled with their expertise in quality improvement processes, that is guided by data and seeks to measure meaningful outcomes, allows school nurses to bridge the gap between national guidelines and measurement of districtspecific policies at the school level. For example, school nurses may review the survey and replace generic policy terms used in national guidelines with associated districtspecific policy names. This may facilitate more accurate survey responses. School nurses can also evaluate the survey and suggest elimination of questions that are not relevant to district-level policies. All adaptations should occur at the district level to ensure consistent policy measurement across all schools within a district. More research is needed to understand how this survey can be adapted to gather uniform data across various districts.
The Framework for 21st Century School Nursing PracticeTM promotes school nurses as change agents who engage in data collection for the meaningful evaluation of student outcomes and take leadership in policy development and implementation (NASN, 2016). However, school nurses often report difficulty in incorporating the Framework into their daily practice (Maughan et al., 2016; Best et al., 2017). This has led to the call for practical tools to guide school nurses in practicing alignment with the Framework (Best et al., 2017; Reising & Cygan, 2020). Specifically, researchers recognized the need for tools that focus on population-level evidence-based practice (Davis et al., 2019). The health and wellness policy implementation measurement tool described in this paper fills this identified gap. The tool may be used by school nurses and other school health educators to understand how evidence-based practices for health and wellness policies are being implemented within their schools. Data collected from this tool may be used to identify areas in which further strategies for policy implementation are needed. This tool can guide a quality improvement process that will allow school nurses to lead change in the area of school health and wellness policy with the intention of improved student health and learning outcomes. Ultimately, this tool can be used as a practical guide for realizing all key principles of the Framework for 21st Century Nursing PracticeTM.
There are limitations to this study that may have impacted the development of the survey. The Delphi study lasted 10 months from the start of Round 1 to the completion of Round 4. At some points there were up to 12 weeks in between rounds as the researchers applied careful qualitative methods to ensure only the essential content areas and items were promoted within the survey. The length between rounds could have caused expert panelists to have inconsistent rankings for each round, impacting overall results (Hsu & Sandford, 2007).
The panelists were purposively selected to participate given their expertise and experience in school health and wellness work. Their professions varied from researchers to teachers and from nurses to administrators. This varied expertise was intentional to gather expert perspectives on the wide range of topics the survey needed to cover to be a comprehensive assessment. However, the limitation of this varied expertise is that each panelist may not have been able to provide expert opinion equally across all content areas (Linstone & Turoff, 2002). This may have contributed to the large number of items included on the final survey.
The survey was piloted in one, unique school district within a limited number of schools. For the tool to be generalizable to other districts throughout the country, additional research must occur in other districts to support the external validity of the final survey. Further, the environmental audits were conducted in just seven schools. While randomly selected, schools voluntarily agreed to participate. This may have led to inclusion bias, including mostly schools that believed their survey responses were in alignment with operational practices in the school, skewing audit results toward positive outcomes.
The use of a Delphi process to create a survey, while feasible, is a complex and time-intensive methodology. As a result of using the Delphi method, researchers developed a comprehensive measurement tool to fill the research gap between the evaluation of written policy and the assessment of policy implementation. The next step in this line of research is to conduct additional studies with the measurement tool in various school districts to build support for the generalizability of the survey to accurately evaluate policy implementation. Additional testing of the survey within other school districts would support its utility for schools to compare school health policy implementation across districts. Further, this tool may serve as a practical guide for school nurses to practice alignment with the Framework for 21st Century School Nurse Practice School PracticeTM. Harnessing the expertise of school nurses through the use of this tool will lay the foundation for assessing the true impact of health and wellness policy on student outcomes across the country.
This project is part of the Building Healthy Urban Communities Project funded by BMO Harris Bank. The content of this article is solely the responsibility of the authors.
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the BMO Harris Bank
Heide R. Cygan https://orcid.org/0000-0003-2811-2170
Supplemental material for this article is available online.
Barr, E. M., Goldfarb, E. S., Russell, S., Seabert, D., Wallen, M., & Wilson, K. L. (2014). Improving sexuality education: The development of teacher-preparation standards. The Journal of School Health, 84(6), 396–415. https://doi.org/10.1111/josh.12156
Basch, C. E. (2011). Healthier students are better learners: A missing link in school reforms to close the achievement gap. The Journal of School Health, 81(10), 593–598. https://doi.org/10.1111/j.1746-1561.2011.00632.x
Belansky, E., Chriqui, J., & Schwartz, M. (2009). Local school wellness policies: How are schools implementing the congressional mandate? http://www.bridgingthegapresearch.org/_asset/0myri8/RWJF_Brief_WellnessPolicies_2009.pdf
Berget, C., Nii, P., Wyckoff, L., Patrick, K., Brooks-Russell, A., & Messer, L. H. (2019). Equipping school health personnel for diabetes care with a competency framework and pilot education program. The Journal of School Health, 89(9), 683–691. https://doi.org/10.1111/josh.12806
Best, N. C., Oppewal, S., & Travers, D. (2017). Exploring school nurse interventions and health and education outcomes: An integrative review. The Journal of School Nursing, 34(1), 14–27. https://doi.org/10.1177/1059840517745359
Bradley, B. J., & Greene, A. C. (2013). Do health and education agencies in the United States share responsibility for academic achievement and health? A review of 25 years of evidence about the relationship of adolescents’ academic achievement and health behaviors. The Journal of Adolescent Health, 52(5), 523–532. https://doi.org/10.1016/j.jadohealth.2013.01.008
Bryan, L. N., Brener, N., Barker, L., Lo, A., & Underwood, J. M. (2021). US Nationwide assessment of school health policies and practices using state-level data. Health Education Journal, 80(6), 724–733. https://doi.org/10.1177/00178969211008553
Castelli, D. M., Centeio, E. E., Beighle, A. E., Carson, R. L., & Nicksic, H. M. (2014). Physical literacy and comprehensive school physical activity programs. Preventive Medicine, 66, 95–100. https://doi.org/10.1016/j.ypmed.2014.06.007
Centers for Disease Control and Prevention (2019). School Health Index FAQs. https://www.cdc.gov/healthyschools/shi/FAQ.htm
Centers for Disease Control and Prevention (2021). School Health Guidelines. https://www.cdc.gov/healthyschools/npao/strategies.htm
Chicago Public Schools (2016). Healthy CPS. https://cps.edu/Programs/HealthyCPS/Pages/HealthyCPS.aspx
Chriqui, J. F., Resnick, E. A., Schneider, L., Schermbeck, R., Adcock, T., Carrion, V., & Chaloupka, F. J. (2013). School district wellness policies: Evaluating progress and potential for improving children’s health five years after the federal mandate. School Years 2006–07 through 2010-11. (Vol. 3). Chicago, IL: Bridging the Gap Program. University of Illinois at Chicago. www.bridgingthegapresearch.org.
Cicutto, L., Gleason, M., Haas-Howard, C., Jenkins-Nygren, L., Labonde, S., & Patrick, K. (2017). Competency-based framework and continuing education for preparing a skilled school health workforce for asthma care: The Colorado experience. The Journal of School Nursing, 33(4), 277–284. https://doi.org/10.1177/1059840516675931
Custer, R. L., Scarcella, J. A., & Stewart, B. R. (1999). The modified delphi technique - a rotational modification. Journal of Vocational and Technical Education, 15(2). http://doi.org/10.21061/jcte.v15i2.702
Cygan, H, & Tribbia, C (2019). School health policy implementation: Facilitators and challenges. Journal of School Nursing, 36(5). https://doi.org/10.1177/19846089
Cygan, H., Tribbia, C., & Tully, J. (2019). School health policy implementation: Facilitators and challenges. The Journal of School Nursing, 36(5), 330–338. https://doi.org/10.1177/1059840519846089
Davis, D., Maughan, E. D., White, K. A., & Slota, M. (2019). School nursing for the 21st century: Assessing scope of practice in the current workforce. The Journal of School Nursing, 37(5), 374–386. https://doi.org/10.1177/1059840519880605
Falzarano, M., & Pinto Zipp, G. (2013). Seeking consensus through the use of the delphi technique in health sciences research. Journal of Allied Health, 42(2), 99–105.
Harris, P. A., Taylor, R., Minor, B. L., Elliott, V., Fernandez, M., O’Neal, L., McLeod, L., Delacqua, G., Delacqua, F., Kirby, J., & Duda, S. N., & REDCap Consortium (2019). The REDCap consortium: Building an international community of software platform partners. Journal of Biomedical Informatics, 95, 103208. https://doi.org/10.1016/j.jbi.2019.103208
Hsu, C., & Sandford, B. (2007). The delphi technique: Making sense of consensus. Practical Assessment, Research and Evaluation, 12(10), 1.
Humphrey-Murto, S., Varpio, L., Wood, T. J., Gonsalves, C., Ufholz, L. A., Mascioli, K., & Foth, T. (2017). The use of the delphi and other consensus group methods in medical education research: A review. Academic Medicine, 92(10), 1491–1498. https://doi.org/10.1097/ACM.0000000000001812
Kennedy, H. P. (2004). Enhancing delphi research: Methods and results. Journal of Advanced Nursing, 45(5), 504–511. https://doi.org/10.1046/j.1365-2648.2003.02933.x
Klostermann, B. K., Perry, C. S., & Britto, M. T. (2000). Quality improvement in a school health program: Results of a process evaluation. Evaluation & the Health Professions, 23(1), 91–106. https://doi.org/10.1177/01632780022034507
Lam, L. T. (2014). Risk factors of internet addiction and the health effect of internet addiction on adolescents: A systematic review of longitudinal and prospective studies. Current Psychiatry Reports, 16(11), 508-014-0508-2. https://doi.org/10.1007/s11920-014-0508-2
Lee, J. A., & Welk, G. J. (2021). Association between comprehensive school physical activity program implementation and principal support. Health Promotion Practice, 22(2), 257–265. https://doi.org/10.1177/1524839919862767
Linstone, H., & Turoff, M. (2002). The delphi method: Techniques and applications. Addison-Wesley. https://web.njit.edu/∼turoff/pubs/delphibook/delphibook.pdf
Long, S. J., Littlecott, H., Hawkins, J., Eccles, G., Fletcher, A., Hewitt, G., Murphy, S., & Moore, G. F. (2020). Testing the “zero-Sum game” hypothesis: An examination of school health policies and practices and inequalities in educational outcomes. Journal of School Health, 90(5), 415–424. https://doi.org/10.1111/josh.12889
Marshall, M. G., & Allegrante, J. P. (2017). Achieving consensus on principles of good practice for school health in independent schools: A delphi study. The Journal of School Health, 87(10), 732–742. https://doi.org/10.1111/josh.12545
Maughan, E. D., Duff, C., & Wright, J. (2016). Using the framework for 21st-century school nursing practice in daily practice. NASN School Nurse, 31(5), 278–281. https://doi.org/10.1177/1942602X16661558
Michael, S. L., Merlo, C. L., Basch, C. E., Wentzel, K. R., & Wechsler, H. (2015). Critical connections: Health and academics. The Journal of School Health, 85(11), 740–758. https://doi.org/10.1111/josh.12309
Mickel, C. F., Shanovich, K. K., Evans, M. D., & Jackson, D. J. (2017). Evaluation of a school-based asthma education protocol: Iggy and the inhalers. The Journal of School Nursing, 33(3), 189–197. https://doi.org/10.1177/1059840516659912
National Association of School Nurses (2016). Framework for 21st century school nursing practice: National association of school nurses. NASN School Nurse, 31(1), 45–53. https://doi-org.ezproxy.rush.edu/10.1177/1942602X15618644
Pinto, R. O., Pattussi, M. P., Fontoura Ldo, P., Poletto, S., Grapiglia, V. L., Balbinot, A. D., & Horta, R. L. (2016). Validation of an instrument to evaluate health promotion at schools. Revista De Saude Publica, 50, 2-8787.2016050005855. Epub 2016 Mar 8. https://doi.org/10.1590/S01518-8787.2016050005855
Qualtrics (2021). Qualtrics XM. Retrieved from https://www.qualtrics.com/
Rasberry, C. N., Lee, S. M., Robin, L., Laris, B. A., Russell, L. A., Coyle, K. K., & Nihiser, A. J. (2011). The association between school-based physical activity, including physical education, and academic performance: A systematic review of the literature. Preventive Medicine, 52(Suppl 1), S10–S20. https://doi.org/10.1016/j.ypmed.2011.01.027
Reising, V., & Cygan, H. (2020). School nurse job descriptions in urban districts: Alignment with the framework for 21st-century school nursing practice. The Journal of School Nursing. E-pub ahead of print. https://doi.org/10.1177/1059840520963621
S.2507 - child nutrition and WIC reauthorization act of 2004, 108 Cong (2004). https://www.congress.gov/bill/108th-congress/senate-bill/2507
Schuler, B. R., Saksvig, B. I., Nduka, J., Beckerman, S., Jaspers, L., Black, M. M., & Hager, E. R. (2018). Barriers and enablers to the implementation of school wellness policies: An economic perspective. Health Promotion Practice, 19(6), 873–883. https://doi.org/10.1177/1524839917752109
Schwartz, M. B., Henderson, K. E., Falbe, J., Novak, S. A., Wharton, C. M., Long, M. W., & Fiore, S. S. (2012). Strength and comprehensiveness of district school wellness policies predict policy implementation at the school level. The Journal of School Health, 82(6), 262–267. https://doi.org/10.1111/j.1746-1561.2012.00696.x
Story, M., Nanney, M. S., & Schwartz, M. B. (2009). Schools and obesity prevention: Creating school environments and policies to promote healthy eating and physical activity. The Milbank Quarterly, 87(1), 71–100. https://doi.org/10.1111/j.1468-0009.2009.00548.x
United Sates Senate (2010). S. rept. 111-178 - HEALTHY, HUNGER-FREE KIDS ACT OF 2010. https://www.congress.gov/congressional-report/111th-congress/senate-report/178
Wall, R., Litchfield, R., Carriquiry, A., McDonnell, E. T., & Woodward-Lopez, G. M. (2012). Local wellness policy strength and perceived implementation of school nutrition standards across three states. Childhood Obesity, 8(4), 331–338. https://doi.org/10.1089/chi.2012.0047
Weatherson, K. A., McKay, R., Gainforth, H. L., & Jung, M. E. (2017). Barriers and facilitators to the implementation of a school-based physical activity policy in Canada: Application of the theoretical domains framework. BMC Public Health, 17(1), 835-017-4846-y. https://doi.org/10.1186/s12889-017-4846-y
Heide R. Cygan, DNP, RN, PHNA-BC, is an associate professor, in the Rush University College of Nursing’s Department of Community, Systems and Mental Health. She also co-directs the Advanced Public Health Nursing and Transformational Leadership: Population Health DNP programs. For the past 16 years Cygan’s work has focused on addressing the public health needs of young people through a variety of community and school-based programs. She is nationally recognized through professional presentations and peer-reviewed publications.
Rachael D. Dombrowski, PhD, MPH, designs, implements, and evaluates multilevel interventions to advance social justice and improve community health. Dombrowski is a faculty member of the Wayne State University Center for Health and Community Impact and the College of Education Community Health program where she conducts community-based participatory research, including program development and evaluation within the Midwest and nation-wide. She is a graduate of the University of Illinois at Chicago (PhD) and the University of Michigan (MPH). She has worked in the public health field for over 15 years.
E. Whitney G. Moore, Ph.D. is an associate professor in the Division of Kinesiology, Health & Sport Studies at Wayne State University in Detroit. Over the past 10 years, Whitney has authored over 40 research articles and three book chapters on motivation in physical activity contexts. She has specialized in developing and testing new measures/scales related to physical activity and health contexts and applying advanced quantitative analysis techniques to answer applied research questions.
Jamie Tully, MS, MPH, is the director of Health Information and Response at Chicago Public Schools, leading the district’s efforts to make data-driven and evidence-based strategies to remove health-related barrier to learning. She has over 10 years of experience as an educator and public health professional. She implemented the nationally recognized Healthy CPS, a success policy implementation policy, and started the district’s first Health Information Team to create data-driven public health solutions. Ms. Tully is also a key partner in a variety of evaluation and research projects.
Kimberly Kin holds a master’s degree in education: Community Health, through Wayne State University. For the last seven years, her work has been focused on positively impacting as many lives as possible, through education and promotion of healthy behaviors. One of the many ways she has been able to complete this work in the past was through her Research Assistant position with Wayne State University. She is currently works for a nonprofit organization, Leaders Advancing and Helping Communities, as a health educator.
Elizabeth Hansen, MPH, is a program manager and epidemiologist at the University of Chicago Medicine’s Urban Health Initiative. She has worked in public health for 6 years, in health policy compliance, infection control, asthma research, Medicaid data management, and program development/evaluation.
1 Community, Systems and Mental Health Nursing, Rush University College of Nursing, Chicago, IL
2 Division of Kinesiology Health and Sport Studies, College of Education, Wayne State University, Detroit, MI
3 Kinesiology, Health & Sport Studies, Achievement Motivation Theory Specialist, College of Education, Wayne State University, Detroit, MI
4 Office of Student Health and Wellness, Chicago Public Schools, Chicago, IL
5 Kinesiology, Health & Sport Studies, College of Education, Wayne State University, Detroit, MI
Corresponding Author:*Heide R. Cygan, DNP, RN, PHNA-BC Associate Professor, Community, Systems and Mental Health Nursing, Rush University College of Nursing, 600 S. Paulina, Suite 1080, Chicago, IL 60612.Email: Heide_Cygan@rush.edu
