Metrics & Dashboards: Ace the Proactive Outage Prevention Interview Question
Learn how to nail the STAR-format behavioral question about using metrics to prevent outages — with real phrasing, pitfalls, and Marriott Tech Accelerator prep.
Loading...
Learn how to nail the STAR-format behavioral question about using metrics to prevent outages — with real phrasing, pitfalls, and Marriott Tech Accelerator prep.
When Marriott's Tech Accelerator panel asks you "Tell me about a time you used metrics and dashboards to proactively prevent a production outage," they're not just asking if you know what Grafana is. They're checking three very specific things:
Here's what the interviewer is really testing when they ask this: they want to know if you're the kind of engineer who notices the smoke before the fire, or the one who shows up after the sprinklers go off. For a hospitality tech company where a 10-minute outage during peak booking season can mean thousands of lost reservations, this distinction matters enormously.
Let me be honest with you — most candidates fumble this because they either:
Here's a quick comparison to calibrate your answer:
| Weak Answer | Strong Answer |
|---|---|
| "I saw the CPU was high so I restarted the service" | "I set up a p99 latency alert at 800ms — 40% below our SLA — giving us a 20-minute runway to investigate" |
| "Our dashboard showed errors" | "Error rate crossed 0.8% on the payment service — our threshold before customer-facing failures historically appeared" |
| "I fixed it before it became a problem" | "We rolled back the deployment at 11:42 AM, preventing an estimated 4-hour outage during our highest-traffic window" |
| No mention of teammates | "I pulled in the DB team because query times were the leading indicator" |
The interviewer is actually checking if you can speak in signals, not symptoms — and if you understand why you were watching what you were watching.
Let's walk through this together using the STAR framework — Situation, Task, Action, Result. I'll show you how to structure each layer, then give you a full example story you can adapt.
Don't just say "we had a microservices app." Give the interviewer a reason to care. Tell them what was at stake.
"This was during a major promotional weekend for our e-commerce platform — we were expecting 3x normal traffic volume over 48 hours. A previous incident during a similar event had cost the business roughly $80K in lost revenue in under two hours."
Notice how that immediately communicates business stakes? That's the kind of framing that makes interviewers lean in.
Be clear about your ownership. Avoid "we" soup where nobody can tell what you personally did.
"I was the on-call SRE for that weekend. My responsibility was to ensure the checkout and payment pipeline stayed healthy. I'd also recently taken ownership of our observability stack after the previous incident."
This is the bulk of your answer and where most candidates either shine or completely fall apart. Here's the structure that works:
1. Explain what you were monitoring and why those specific metrics
Don't just say "I watched the dashboard." Tell the interviewer why you chose those metrics. This signals observability maturity.
"After the last incident, I did a post-mortem and found that three metrics always preceded our checkout failures: p99 latency on the payment service, database connection pool saturation, and the queue depth on our order processing workers. I built a composite dashboard tracking all three with alerting thresholds set 30% below our SLA limits — essentially an early warning system."
2. Describe the specific signal you caught
Be precise. Numbers matter here.
"Around 2:15 PM on Saturday, the queue depth on the order processing workers jumped from our normal baseline of ~200 jobs to over 1,400 in under eight minutes. p99 latency hadn't moved yet — which was actually the key insight. The problem was upstream, not in the service itself."
3. Walk through your investigation process
Show your reasoning, not just your actions.
"I pulled the worker logs and cross-referenced with a recent deployment that had gone out at 1:50 PM. One of the worker configuration changes had reduced the thread pool size — a seemingly minor change that was fine under normal load but was creating a bottleneck as traffic ramped up toward our promotional peak."
Here's an example of the kind of monitoring query logic you might reference in your answer to sound concrete and technical:
# Example: PromQL query used to track queue depth rate of change
# This is the kind of alert rule that gave us the early warning
# Alert fires when queue is growing faster than 100 jobs/minute
# for more than 5 consecutive minutes
rate(order_queue_depth[5m]) > 100
# Composite alert: queue growing AND latency starting to climb
(
rate(order_queue_depth[5m]) > 50
AND
histogram_quantile(0.99, rate(payment_request_duration_seconds_bucket[5m])) > 0.6
)You don't need to paste code into a verbal interview, but referencing how you structured your alerts — especially composite conditions — signals real depth.
4. Describe the action you took
"I immediately pinged the deployment team and the DB lead on Slack, shared the dashboard link, and recommended we either roll back the thread pool config or hot-patch it. Within 12 minutes we had consensus to roll back. The rollback completed at 2:41 PM — queue depth normalized within 4 minutes."
This is where you land the plane. Give numbers and connect to business impact.
"We prevented what our post-incident simulation estimated would have been a 3-to-4 hour checkout degradation during peak traffic. Based on our revenue rate that afternoon, that was roughly $120K in protected revenue. More importantly, we had zero customer-facing errors — our error rate never crossed 0.1%. The incident never made it to a PagerDuty page for customer support."
Here's some exact phrasing you can borrow and adapt. Practice saying these out loud — they help you sound structured without sounding robotic.
Opening your answer:
"I want to tell you about a situation from [timeframe] where proactive monitoring genuinely prevented what would have been a significant outage. I'll walk you through what I was watching, what I caught, and what we did about it."
Transitioning to the technical detail:
"The reason I was monitoring those specific signals — not just generic CPU and memory — is because our post-mortem from the previous incident identified them as leading indicators. By the time the obvious metrics moved, we were already in trouble."
If you get nervous and lose your place:
"Let me make sure I'm giving you the full picture here — the key action I took was..."
Wrapping up with impact:
"The outcome I'm most proud of isn't the rollback itself — it's that we built a system where we caught it 40 minutes before it would have affected users. That's the difference between proactive and reactive."
I've watched hundreds of candidates stumble on this question. Here are the most common traps and how to escape them:
You start talking and realize halfway through that your story is actually about an outage you fixed, not one you prevented. Don't panic — pivot explicitly:
"Actually, let me give you a better example — one where the monitoring caught it before users were affected..."
It's far better to redirect than to push through a story that doesn't match the question.
"The dashboard showed things were off" is not an answer. Practice being specific: name the metric, give the threshold, give the actual value you saw. "p99 latency was 340ms against a threshold of 500ms" is ten times more compelling than "latency was getting high."
Engineers love to go deep on the technical fix. That's fine — but always come back up to: what would have happened to users/revenue/guests if you hadn't caught this? For Marriott specifically, think in terms of booking flows, loyalty points redemption, or property management system availability.
Interviewers are hiring you. They need to know what you built, you noticed, you escalated. Use "we" when describing team decisions and collective actions, but be explicit about your individual contribution.
Strong engineers always close with a learning. Did you update the runbook? Did you add a new alert? Did you do a blameless post-mortem? This shows growth mindset, which senior interviewers weight heavily.
After your STAR story, expect the interviewer to probe. Here are the most common follow-ups and how to answer them confidently:
"How did you decide which metrics to track?" This is a gift. Talk about post-mortem analysis, SLA-based thresholds, RED method (Rate, Errors, Duration) or USE method (Utilization, Saturation, Errors) — whatever framework guided you.
"What tool did you use for your dashboards?" Name it (Grafana, Datadog, New Relic, CloudWatch, etc.) and briefly describe why that tool was right for the context. Don't just list tools — show judgment.
"What would you have done if you hadn't caught it in time?" This tests your incident response knowledge. Talk about your runbook, escalation path, communication plan, and how you'd communicate to stakeholders.
"How did you make sure this type of issue wouldn't happen again?" This is where you talk about the post-mortem, alert tuning, and any process changes. Here's an example of the kind of monitoring-as-code approach you might reference:
# Example: Alert rule added post-incident (Prometheus/Alertmanager format)
# This became part of our standard deployment checklist
groups:
- name: order_pipeline_health
rules:
- alert: OrderQueueGrowthAnomaly
expr: |
rate(order_queue_depth[5m]) > 80
AND
rate(order_queue_depth[5m]) > (
rate(order_queue_depth[5m] offset 1w) * 2
)
for: 3m
labels:
severity: warning
team: platform
annotations:
summary: "Order queue growing faster than historical baseline"
description: "Queue depth rate {{ $value }} jobs/min — investigate worker capacity"
runbook_url: "https://wiki.internal/runbooks/order-queue-saturation"Referencing that you turned your incident learning into code — not just a Confluence page — is a massive signal of engineering maturity.
"Have you ever had a false alarm from your monitoring? How did you handle it?" Be honest. Talk about alert fatigue, how you tuned thresholds, and why some noise is better than missing a real signal. Show nuance.
These are the things that make interviewers quietly lose confidence in you — even if they don't say anything in the moment:
Here's what to lock in before you walk into that Marriott Tech Accelerator interview:
You've got this. The candidates who nail this question aren't the ones who've never had an outage — they're the ones who've thought deeply about why things go wrong and built systems to catch problems before they do.