Consumer Picks 2013: Understanding the numbers

This issue of Nation’s Restaurant News features the third annual Consumer Picks report. Like any good brand manager, we continuously strive to improve our offer. In the case of Consumer Picks, we work to refine this study on a yearly basis to make it more useful and insightful for our readers. Here are the notable changes you’ll find in this year’s study:

Release date

In response to reader feedback, we decided to move the release date up to March to ensure that the data coincides with the end of the calendar year. This year’s research was fielded the third and fourth weeks of January. As such, it represents restaurant visits made in the second half of calendar-year 2012. Look for future issues of Consumer Picks to follow the same time frames and reporting dates.

Demographic reporting

Another improvement was the decision to report age groups in years instead of in generational cohorts, such as Millennials, baby boomers, etc. We made this change because of the lack of uniformity around what specific years of age define each generation.

Sampling

We also refined a few of the internal sampling processes to make state-level geographic data more accessible to those brands that opt to purchase the full data sets on their brands or competitors’ brands.

Filtering

We added a few more “hurdles” to help ensure that any respondents who were not being diligent in filling out the survey would show up in our various filtering devices. This refinement added one more layer to our vetting of responses, increasing the data’s reliability. In fact, we rejected about one-sixth of the responses as being suspect, which left us with almost 5,500 thoroughly vetted and reviewed surveys.

More brands

This year’s report includes four new brands, pushing the total to 154 chains. Morton’s the Steakhouse is one of the newly added brands, which makes for a nice addition to the Fine Dining category.

At the same time, there are a few chains that were in the 2012 Consumer Picks that did not make it into this year’s survey.  Among them, Jet’s Pizza, a limited-service pizza chain that made it into the 2012 survey with slightly more than 100 responses — the minimum threshold. It came in with just under 100 responses this year, excluding it from the rankings. 

Categories

As in prior years, there are always a few chains that do not fit neatly into any one category.  We do our best to segment them according to our guidelines, which have an initial filter of full service or limited service. Menu acts as a secondary filter. Note that we had an opportunity to add Barbecue as a new category this year.

The study allows a user to move any brand to any other subsegment without loss of accuracy, so long as the brand is not moved between the primary levels of Limited Service, Family Dining or Casual Dining. Crossing these primary categories will introduce some error in Overall Scores as they are weighted by the consumer-reported importance attributes for the segment in which the brand appears. 

Consistency

These changes aside, we kept the rest of the survey consistent in format and reporting with previous years in order to facilitate year-to-year comparisons. These comparisons will, no doubt, be a rather common use of the data. For that reason I would like to offer a few suggestions and warnings. 

If you look at only the difference in a brand’s scores from last year to this year, you run the risk of drawing incorrect conclusions. To look only at one year over the other for any one brand does not take into account the differences between research collection times, or economic and social factors that took place during the year that may have impacted consumers’ responses.  

A more accurate assessment comes from comparing one brand’s score against the average score for an identical group of competitors for both this year and last year, looking at the relative difference between the subject brand and its group of competitors. WD Partners has created a blank Excel file to aid you in making such comparisons. It is available as a download from the WD Partners site.

WD Partners and NRN hope you find Consumer Picks 2013 to be a useful and valuable tool. Please feel free to contact us if you have questions or are interested in more detailed data.

Dennis Lombardi is executive vice president of foodservice strategies for WD Partners in Dublin, Ohio. He can be reached at [email protected].

Consumer Picks 2013: Methodology

(Continued from page 1)

The Consumer Picks survey was developed by WD Partners’ Insights group and is designed to provide relative benchmark ratings on major attribute ratings across restaurant brands.

Respondents, who participated online, were given a list of up to 174 restaurant chains — depending on the brands with operations in their states of residence — which were organized into groups depending on the chains’ service models. They were asked to identify which restaurants they had patronized in the last six months, or since July 15, 2012, and how many times they visited during that time period, be it one to two, or three or more times. Respondents were then asked to rate their experiences at the restaurants for up to eight of the restaurants they had patronized in the last six months. The six-month time frame increased the likelihood of reasonable recall by respondents and also ensured that respondents would represent a broader range of consumers and not just a brand’s raving fans.

Responses were scrubbed for 1) inconsistent answers, 2) “straight lining,” or selecting the same response repeatedly, and 3) excessive haste in completing the questionnaire. This scrubbing process reduced the number of acceptable responses to 5,467 from a total of 6,496.

The accepted responses represented approximately 126,000 restaurant visits over the last six months, which resulted in nearly 43,000 restaurant ratings of the 174 chains.

Twenty of the 174 restaurants had fewer than 100 ratings and are not included in this report. The remaining 154 restaurant chains yielded 42,332 ratings.

Twenty-seven of these 154 restaurant chains had between 100 and 149 ratings and are included in this report, but are marked with an asterisk to indicate they received fewer ratings than the desired quota of 150. Consequently, the final report features 127 restaurant chains that were rated by at least 150 respondents, with the vast majority, or 91 restaurant brands, rated by more than 200 respondents.

The questionnaire was designed in-house by WD Partners and pretested by staff members familiar with survey methodologies. We worked with Survey Sampling International, which supplied panel respondents, and SurveyGizmo, which provided the survey administration software.

Results are shown as the percentage of top-two-box ratings received, based on a standard five-box scale. Overall Scores are a weighted average of attribute scores as calculated using their respective customer-reported importance ratings.

The survey addresses 10 attributes: Atmosphere, Cleanliness, Craveability, Food Quality, Likelihood to Recommend, Likelihood to Return, Menu Variety, Reputation, Service and Value.

Demographic information on the respondents was obtained to align the survey results with the U.S. population based on 2010 U.S. Census and 2010 Bureau of Labor Statistics reports.

As in all surveys of this type, closely ranked restaurant chains should be considered statistically equivalent.

The detailed data for 2013 — as well as that for prior years — is available for purchase. Additional survey results, such as percentage of top-box scores and custom data analysis, also are available by contacting [email protected].

WD Partners is a Dublin, Ohio-based firm specializing in the customer experience that helps global food and retail brands innovate through strategy and design. Research conducted by WD Partners’ Insights group is part of the company’s integrated approach to enhancing the performance of foodservice brands.