Methodology 2017

One of PHA’s main goals is to ensure that commitments made are commitments kept. To assess the progress of PHA partner commitments, we work with a team of external verifiers, including:

• Altarum
• Center for Active Design
• Food, Nutrition & Policy Consultants LLC
• Hudson Institute
• RTI International
• Rudd Center for Food Policy and Obesity

Together, PHA and our verifiers approach the process with three goals:

• To design verification methodologies that will yield valid, meaningful data and ensure partner accountability;
• To establish processes that are feasible to implement in real-world settings without undue data collection and/or reporting burdens on partners; and
• To strive for consistency in reporting across indicators and data sources, while maintaining a level of flexibility in approaching the verification process such that unique circumstances, abilities, and data systems can be accommodated as necessary to document progress.

The general methodology in approaching the verification process is as follows:

• Gather background information on commitments
• Operationalize commitment elements and identify appropriate indicators and data sources
• Work with PHA and its partners to establish or confirm methods for collecting and reporting data
• Develop tools to assist partners in data collection and in the reporting process
• Collect, verify and summarize data

Specific methodologies are developed in tandem with each commitment and include everything from using standardized tools to menu and nutrient analyses to GIS software that maps new grocery stores against USDA-designated food desert census tracts or low supermarket access areas.

Increasing Physical Activity:

For Dick’s Sporting Goods and Dick’s Sporting Goods Foundation, verifiers obtained summary data on the amount of funding, projects, and reach for the Sports Matter project from the DonorsChoose.org website. No detailed information about these projects was provided. Presence of the Sports Matter website was confirmed by going to www.sportsmatter.org. Details on who could apply through the website were identified in the “Apply” section of the website. Verifiers reviewed names and affiliations of the grant-making panel, as well as the complete scoring rubric and points of consideration for the panel. For Dick’s Sporting Goods Foundation marketing, verifiers reviewed a Sports Matter media report with itemized data on the type of advertising and signage and amount of funding spent. Data were summed across all media spent in 2017. Dick’s Sporting Goods Foundation’s involvement in creating a documentary film was verified using a copy of the agreement to create the film. Completion of the documentary was confirmed via viewing of the unreleased film.

To assess Mercedes-Benz, USA’s progress towards meeting its commitment, verifiers reviewed data on distribution of funds to various organizations and key activities for funds. Information on coaches was assessed via review of coach training log information including the number of coaches trained and city of placement. Engagement of Mercedes-Benz USA employees was assessed via review of a report providing dates, locations, and number of employees or dealers for engagement events. Data were summed to calculate number of opportunities and participants.

To measure New York Road Runners’ progress, verifiers reviewed a list of school names along with their addresses, type of program, free and reduced-price lunch eligibility, and Mighty Milers enrollment. Participating schools in 2016-2017 were compared to lists of schools from prior years and duplicates were removed to determine the number of new schools participating in Mighty Milers. The number of Mighty Milers participating from 2013-2014 to 2016-2017 school years were added to determine the number of new participants since baseline. Verifiers calculated the total number of trainings provided and the number of training participants from a training log that included training date, topic, and participants per class. Promotion of Let’s Move! Active Schools was assessed by review of the New York Road Runner’s website.

NIKE provided information on investments made, including award type, category, amount, date, recipient, city and state and key activities to be completed with the investment. Verifiers reviewed all data and calculated total investments for all areas.

The launch of Net Generation by United States Tennis Association (USTA) was assessed via Net Generation website, app, and copies of Net Generation curriculum and promotional materials. Number of coaches engaged by USTA was assessed using a report with date of sign up, city, state, and NCSI approval status for each coach that signed up for Net Generation. Data were summed to assess the number of coaches and NCSI-approved coaches that had signed up for Net Generation. Provision of tennis equipment was assessed via detailed information on each type of material or equipment donated and total cost, which were summed across all donations. Safe play provisions were verified using public information available on Net Generation’s website outlining Safe Play program guidelines.

For the Blue Goji commitment, provision and description of funding by the Coleman Fung Foundation to the University of California, Berkeley was verified based on a Fund Terms document signed by both parties that identified the total funding amount and the purpose of the funding. Additional information about the purpose and activities of the Fung Foundation Wellness & Technology Innovations Fellowship were obtained from the University of California, Berkeley website. Verifiers reviewed design plans for the Go Wings Safari game for young children which were created by Fellowship students. Plans for community partnerships, equipment donations, and Blue Goji’s identification of a project to be introduced to the marketplace were outlined in a progress report submitted by Blue Goji.

Verifiers reviewed the 2016 Explore Fund Impact Report supplied by The North Face and calculated the number of awards, unique award recipients, and total amount of funding awarded. The number of individuals, children, low-income children, at-risk children, and children with adaptive needs reached across all grant-funded programs were estimated and reported by grantees and then summed across all programs by verifiers. The North Face did not provide grantees with definitions for low-income, at-risk or adaptive needs; estimates were up to the interpretation of each grantee organization.

Transforming the Marketplace:

To verify progress on Hyatt’s commitment, calorie, sodium and total sugar averages were calculated for the top two selling a la carte items for breakfast, lunch and dinner from a random sample of 24 hotels of three-meal restaurants and in room dining. Calorie, sodium and total sugar values were averaged across the sample to monitor baseline values and annual changes throughout the commitment. Hyatt also provided three-meal restaurant and in-room dining menus to verify affordability and local procurement commitments.

Verifiers reviewed the Westin Eat Well Menu for Kids Brand Program Guide which outlines company-wide guidance on nutrition parameters for children’s menus in all hotels. Progress on the commitment was assessed by comparing all Eat Well Menu for Kids standardized recipes with nutrition analysis for each recipe to determine if Westin nutrition standards were met. Additionally, verifiers ensured only Eat Well Menu for Kids recipes were available in hotels by comparing a sample of nine hotel children’s menus to the approved recipe list. To measure progress on the Mars Food U.S. commitments, verifiers reviewed the Mars Food Nutrition Criteria, SKUs and whole grain content for all grain products in the Mars Food U.S. product portfolio and the online recipe database and nutrition analysis for all Uncle Ben’s Everyday meal recipes. Verifiers examined a list of all available U.S. retail products and conducted an online search of press releases and trade press to verify that Mars Food U.S. did not add any new products in 2016. Additionally, verifiers reviewed a summary of Mars Foods media impressions, screenshots of the dotMars internal associate website providing recaps of employee education programs and photos and screenshots of online articles referencing U.S. site cooking kitchens and fitness facilities. The menu from Mars Uncle Ben’s café in was also reviewed to determine compliance with nutrition and affordability commitments for all U.S. site cafeterias.

To measure Mushroom Council’s progress, a verifier reviewed toolkit materials developed by Mushroom Council, as well as Google Analytics data detailing web user accessing of the toolkit materials. The purpose, timing, and eligibility criteria for the “Blend the Rules” contest were verified through documentation on the Mushroom Council’s website as well as publically available reports describing the contest. A verifier reviewed the entries submitted by school districts that participated in the contest. Recipes developed for schools were reviewed and tallied from the Mushroom Council’s website. New meat/mushroom blended products were verified based on publically available information from company websites on availability of mushroom blended products and from distributor tear sheets containing product descriptions, ingredients, and nutrition information.

Nutri Ventures USA, Inc.’s progress was assessed by reviewing the Nutri-Guardians website created by the organization. An independent verifier extensively reviewed website material, documenting availability of various types of content and information and applicability to various key audiences, including professionals and educators.

Verifiers reviewed a copy of One Medical’s affiliation agreement with Iowa State University Dietetic Internship Program, which articulated the terms of intern involvement in the provision of nutrition coaching to Rise App participants in support of One Medical’s commitment. A list of de-identified Rise App participants that received free nutrition coaching, as well as the number of children they have, their BMI, qualification for low-income programs, willingness to commit to a three-month program and iPhone status, was reviewed and any participants not meeting all criteria were excluded before summing the eligible participants that received free coaching.

Progress on the commitment between PHA and Sesame Workshop and the Produce Marketing Association was assessed via review of data on sub-licensee applicants, which included names of organizations seeking use of Sesame Workshop Assets, as well as character requests and associated fruits and vegetables. These findings were totaled and reported.

Progress on commitments made by Kwik Trip, Sheetz, Vintners Inc. Distributers, Tri Star Energy, Ricker’s, Cumberland Farms, Aloha Petroleum, enmarket and U-Gas was determined by reviewing shipping logs and reports from stores to confirm that a minimum number of types of fruits, vegetables, whole grain products and low-fat dairy options were available. When applicable, verifiers reviewed inventory lists or store planograms and spreadsheets of products’ nutritional information to ensure stores offered a minimum number of products and grab and go items that met PHA’s definition of a Healthier Food and Beverage Product Criteria and to calculate linear feet of shelf space dedicated to fruits, vegetables, whole grains and low fat dairy. For healthier combo meals, verifiers reviewed recipes, nutritional information and store menus to ensure options are available that meet PHA’s Healthier Recipe Criteria. Verifiers reviewed employee handbooks to confirm commitment elements around employee benefits, and installation of bicycle racks, if applicable, was verified through review of building plans showing location of bicycle racks.

To measure progress on food and beverage distributors, Core-Mark International provided its marketing program book, a list of all products that meet PHA’s healthier food and beverage product criteria, access to its online ordering website and online mobile ordering application and examples of “Promo Power” and “Good Health to Go” promotional materials for verification. Verifiers also reviewed Core-Mark International’s employee wellness promotion materials, an employee health discount program overview, and the company’s healthier food catering guide. Progress on ESSTAR’s commitments was assessed based upon review of planograms, photos, nutrition information for all products that meet PHA’s product criteria and examples of in store and at the pump advertisements and signage.

PepsiCo’s progress towards meeting commitments to reduce added sugars in its beverage portfolio and reduce saturated fat and sodium in its foods portfolio were reviewed by an independent verifier during a meeting held at PepsiCo headquarters in Purchase, New York in November 2017. The verifier reviewed SKU-level data on beverages and foods sold in PepsiCo’s top 10 global markets in 2016, as well as governance documents detailing the methodology used by PepsiCo to determine the total beverage and food volumes sold and nutritional information related to products sold, including definitions, assumptions, calculations, and rounding procedures. Product data and methods were reviewed for completeness against commitment guidelines agreed upon by PepsiCo and Partnership for a Healthier America, as well as publically available information on PepsiCo products. Calculations were reviewed for accuracy and reproducibility using a random sample of beverage and food products from PepsiCo’s portfolio and publically available data on nutrients in the selected products.

The Hudson Institute Obesity Solutions Initiative (OSI) is serving as third-party validator of the National Confectioners Association’s Always a Treat commitment. In addition, Richard Black, PhD, Tufts University has provided input to the analytical methodology and examined the final conclusions and analyses.

IRI data for the 52 weeks ending December 25, 2016 were utilized to identify and analyze participating companies’ Instant Consumable candy and chocolate item sales (excluding gift items, gum and mints) according to two discreet categories: items that are ≤ 200 calories per pack, and those that are > 200 calories per pack. For purposes of this study, Instant Consumables are defined as all candy and chocolate items (excluding gift packs, gum and mints) sold in packages ≤ 3.2 ounces (90 grams).

A tracking of whether calorie counts appear on the front of participating companies’ candy and chocolate items, except gift items, very small packages (< 12 sq. inches of total available label space), gum and mints, was performed. Companies supplied physical examples of packages by SKU in order for Hudson Institute to verify whether calorie information has been placed on the front of pack (FOP). To cross-check the packaging information supplied by each company, the Hudson team commissioned visual audits of confection products at retail checkout lanes in 9 cities across the country. Audits were conducted by teams of Independent Registered Dietitians between December 8, 2017 and December 15, 2017.

Creating Healthier Places:

Hospital Healthier Food Initiative

For the Hospital Healthy Food commitment, documentation of total food purchases and the percentage of those purchases accounted for by fruits and vegetables was calculated using spreadsheets supplied by hospitals containing dollar amounts for all foods or categories of foods purchased in a baseline year designated by PHA.

Hospitals supplied photographs of cafeteria advertising and copies of general and pediatric patient menus that reviewers used to verify alignment with healthier food marketing criteria. Hospital-supplied photographs and lists of products within five feet of all cafeteria cash register stations were used by reviewers to verify whether hospitals adhered to healthier check out criteria. Independent verifiers utilized recipes, nutrient analyses, menus, and cafeteria pricing supplied by hospitals for three days that were selected by the independent verifier for children’s wellness meals and adult wellness meals. The verifier reviewed all meals to determine whether they met nutrient and food profiles and were offered at lunch and dinner. In order to meet each wellness meal commitment element, hospitals had to offer one to three meals (depending on year of commitment) in the cafeteria and on the patient menu during lunch and dinner that met nutrient and food profiles for all three days that data were collected. To determine whether hospitals achieved percentage goals for healthier foods available in the cafeteria and on patient menus, independent verifiers reviewed spreadsheets prepared by hospitals that contained lists of all foods available over the course of two days selected by the independent verifier. The verifiers reviewed the accuracy of product categorization of entrée or side dish according to protocol, and nutrient criteria to determine whether the product met healthier standards. The verifiers then calculated the percentage healthier entrees and healthier sides. Independent verifiers reviewed photos of cafeteria menu boards, food and beverage stations, and cafeteria menus to assess whether hospitals met requirements for nutrition labeling of all foods in the cafeteria footprint. Finally, hospitals submitted before/after photos to demonstrate removal of fryers or other data supporting removal of fryers and all fried foods, which were reviewed by verifiers. If fried products were still offered by the hospitals, nutrition information was requested to assess whether each product met nutrient requirements.

Morrison conducted a survey with its hospital clients in 2017. In the survey, hospitals self-reported whether they were meeting each of the Wellness Platform criteria to provide healthier hospital environments. Morrison provided the raw survey results to an independent verifier who calculated the number and percentage of hospitals meeting each Wellness Platform criteria and those meeting 100% of the criteria.

Sodexo and independent verifiers worked together to add verification questions to existing data collection instruments already in place at Sodexo. Sodexo provided verifiers with site names, locations, and responses to the survey, which verifiers used to calculate the percentage of sites selling or serving Mindful products. Achievement of Mindful Healthy Dining program gold status in corporate, government, hospital, and university settings was assessed via survey. Survey instruments were developed by Sodexo and an independent verifier and specific to each account type. A total of 495 of 595 corporate accounts (83% response rate), 6 of 24 government accounts (25% response rate), 207 of 607 hospital accounts (34% response rate), and 130 of 329 university accounts (response rate 40%) participated in the survey. Sodexo provided raw survey response data, which were analyzed by an independent reviewer to calculate the number of accounts achieving gold status. Implementation of PHA Hospital Healthy Food Initiative guidelines was assessed via survey and document review. The survey was developed by Sodexo and an independent verifier and used to collect data from hospital account managers on nutrition labeling, food marketing, check out stations, and fried food practices. A total of 607 hospital accounts were eligible to complete the survey and 603 completed the survey (99% response rate). Raw survey data were supplied to the independent verifier for analysis and determination of guidelines met. The percentage of food purchases that were fruits or vegetables for each hospital was calculated by independent reviewers based on procurement data submitted by hospitals that detailed foods and food purchase amounts for one calendar year, July 2016-June 2017. The percentage of hospital beverage purchases that consisted of healthier beverages, according to specific nutrition criteria, was calculated by independent reviewers based on procurement data submitted by hospitals that details all beverage types, nutrition facts, and purchase amounts for one calendar year, July 2016-June 2017. After the percentage of healthier beverages was calculated, hospitals were given credit for offering free water to customers by multiplying healthier beverage purchases by 5%, 10%, or 15% if the hospital provided photographic documentation that they had a drinking water fountain or station with free cups; chilled water, free cups, and ice; or a water station with free fruit slices or promotional messaging encouraging water consumption, respectively.

Trinity Health’s progress towards meeting its commitment was assessed via review of a grant report that included grant award recipients, amount of award, amount of cash match, date of award, length of award, and key activities to be completed. Number of grants, total value of grants, and total cash match were summed across all awards.

Active Design Verified

To verify the progress of our partners in meeting the active design criteria, verifiers reviewed the technical building plans, photographs, and related documentation for each affordable housing site. Verifiers ensured that each building was designed using the evidence-based strategies outlined in the developer’s commitment.

Food Banks

To verify future progress on Des Moines Area Religious Council, Capital Area Food Bank, San Antonio Food Bank and Atlanta Community Food Bank, verifiers reviewed baseline information on produce distributions, school food pantries, family markets, summer food sites and meals served, after-school meals and weekend meals served. San Antonio Food Bank provided monthly reports on nutrition education classes, gardening activities and a list of healthier recipes for review. Verifiers also examined Capital Area Food Bank’s wellness metrics, Partners for Wellness criteria and partner list, and wellness food distributions.

Healthier Campus Initiative

Each post-secondary education institution participating in the Healthier Campus Initiative selected twenty-three elements across the following categories: food and nutrition, physical activity and movement, and wellness programming.

For food and nutrition guidelines, independent verifiers reviewed two days, that were selected by the independent verifier, of breakfast, lunch, and dinner menus for each campus dining venue. Menus, which included nutrient information, allowed verifiers to ensure nutrient targets were met for calories, percent calories from saturated fat, grams of trans fat, and milligrams of sodium in wellness meals. Verifiers utilized production records, planograms, recipes, nutrition facts panels, product specifications, and additional photographs to determine whether campuses offered fruits, vegetables, whole grains, and plant-based food options and limited fried foods and desserts. Procurement records, sales records, and nutrition information were also reviewed and evaluated to verify healthy vending and beverage purchases. Campuses provided additional photographic evidence to verify their use of healthy icons and calorie labeling at the point of purchase. Policies were reviewed to determine the use of local procurement programs, tray-less dining initiatives, and merchandising programs to promote nutritious options. To verify availability of water in each dining, recreation, and educational facility, campuses provided an inventory of water fountains, bottle filling stations, and access points for filtered water in at least five percent of the campus buildings, selected by the verifier.

Physical activity elements were verified through campus maps, photographs, and policies outlining walking and biking infrastructure, access to public transportation, and recreation, physical activity, and competitive sports opportunities.

Wellness elements were verified by examining policy and program documentation to determine whether integrated, comprehensive wellness programming including breastfeeding support, strategies to reduce food insecurity, health insurance benefits, and access to hands-on nutrition education and cooking classes were provided.

Early Childhood and Out-of-School Time

Progress on commitments made by Learning Care Group were assessed via scoring of corporate policies using the Wellness Child Care Assessment Tool, and an online survey of directors of childcare centers was used to assess the nutrition and physical activity environment of each center (adapted from Henderson et al., 2011).

Progress on commitments for the Boys and Girls Clubs of America (BGCA) was measured via the Healthy Eating and Physical Activity (HEPA) inventory and document review. BGCA clubs with early childhood or after-school programs that had committed to implementing the HEPA standards in their programs participated in the inventory. Respondents reported whether they were “Fully Demonstrating,” “Not Fully Demonstrating” or it was “Not Feasible” to demonstrate each HEPA standard individually at their program site. A random sample of 28 clubs that completed the inventory were asked to submit program documents, including daily schedules, menus, program handbooks, and parent outreach materials, which were then reviewed by verifiers against definitions of meeting each of the HEPA standards. Verifiers analyzed the inventory to determine the percentage of clubs meeting each of the standards and the document review data to provide a confidence level on whether clubs had the documentation showing that they were meeting the standards. The confidence level is based on the document audit of 28 clubs and reflects the proportion of audited clubs that submitted clear or plausible evidence to support a claim that they were fully implementing a commitment element. We established four categories: Highly confident=76 to 100% of clubs that said they were demonstrating a commitment element submitted clear or plausible evidence to support their claim; Confident= 51 to 75%; somewhat confident= 26 to 50%; not confident= 0 to 25%. To assist clubs in implementation, BGCA with the Alliance developed training courses and technical assistance resources to facilitate standards implementation.

Progress on commitments made by the U.S. Olympic Committee was verified via a review of child participation in free introductory classes, open house demonstrations, race events, school-based programs, camps, clinics, tournaments and coach training for 14 Olympic and sports activities.

Next: Increasing Physical Activity