With 170,000 transactions or more moving through the online ordering systems of Domino’s Pizza during Super Bowl Sunday, the chain’s information technology department needs to have its game face on.
Lance Shinabarger, vice president of infrastructure and security for Ann Arbor, Mich.-based Domino’s Pizza Inc., spoke with Nation’s Restaurant News just ahead of the Super Bowl about the pressure to perform for the 57 employees on hand at chain headquarters during the big game.
Domino’s, a system of more than 9,541 worldwide restaurants, including at least 4,891 in the United States, monitors all aspects of its online operation during one of the largest pizza ordering and delivery days of the year, from pre-game load testing to after-the-game trash talking.
Shinabarger gives NRN a peak behind the curtain.
How many pizzas does Domino’s expect to move on Super Bowl Sunday and how does that compare to last year?
About 11 million slices, or 1.4 million whole pies. That’s up by about 80 percent from a typical Sunday, and it’s about the same as last year.
What has Domino’s done, IT systemwise, to prepare?
Every year we go through and do extreme load testing. We set up a bunch of synthetic transactions that basically emulates a human placing an order on our system. Because the process is automated, it gives us the ability to ramp up the number of orders to the point where it would be greater than what we’d expect to see during the Super Bowl. We do that because while you might have great estimates [of game-related business], you never know exactly what to expect and you definitely don’t want [systems] to be down during the Super Bowl because about 30 percent of all our orders come through the online channel.
How will Domino’s monitor the online ordering system’s health during the game?
We’ll continue those synthetic transaction tests and watch the results. We also will have some of our folks in the command center actually placing orders into the system. In addition to that, we have a number of monitors that we watch, so every piece of our equipment is monitored by somebody.
What networking components will you be watching closest?
There are the load balancers that let us redirect traffic from one data center to another, if we need to; the firewalls that provide some layers of security; routers; and the actual servers, themselves. We have what we call ‘playbooks’ for every component. They give the normal ranges of operation, so we can do a little health check. If something falls out of its normal range, we sound the alarm.
Continued from page 1
When a problem arises, what happens?
At that point, it is all hands on deck. A lot of these pieces have built in redundancy, so we can switch people placing orders from one piece of equipment to another. If you were in the middle of placing an order and we had to shift you from one piece of equipment to the other, you would not know that had happened. We have the people who are the experts on the system there, in the building, eating pizza and watching the game, so they can work on it, if necessary.
So you’ll have refreshments there?
Yes. We’ll be able to watch the game, like everyone else. Actually, a lot of the senior executives come in and make pizza for the people who are here. That’s a fun piece of the day. The CIO usually comes in, but this year it will be the CFO and that’s pretty high on the food chain around here. A lot of VPs are here, too.
What makes the Super Bowl special?
On a typical day, we deal with all the time zones. So a person on the East Coast starts ordering at 6 p.m., which is still 3 p.m. in the Pacific Time zone. But for the Super Bowl, there is only one kick off and one half time. It doesn’t matter what time zone you are in, everybody is ordering pizza right then. Normally you spread that order load over a four or five hour window, but for Super Bowl there are just those two peaks.
At the end of the game, what tells the IT team that it is the winner?
We monitor [traffic] down to the minute. What you see after half time, once the third quarter starts, is that those numbers really fall off and it becomes like a normal ‘busy’ night. So you hear a big sigh of relief after half time is done. That’s when the mood really lightens up and everyone gets up from their desk, starts to wonder around and the trash talking heats up.
Contact Alan J. Liddle at [email protected].
Follow him on Twitter: @AJ_NRN
