**Parent Topic**: [[Software/README]] ## The Trap A system-call tracer (Solaris truss, Linux strace, FreeBSD ktrace) or DTrace is a peephole into what an application asks of the kernel and how long it waits. But Schlossnagle warns: looking at such output for the *first time* when something has gone wrong "provides basically no value except to the most skilled engineers; in fact, it can often cost you." ## Why Diagnosis under failure is an experiment with no control. If you have never run the tool under normal operating conditions, you have no basis for comparison — every pattern is unfamiliar, and patterns that *appear* correlated to the problem often are not. Engineers then waste time pursuing red herrings. ## The Rule Run your observability tools under normal conditions first, so you know what healthy looks like. A baseline is what converts an avalanche of trace output into signal. This grounds the "understand normal behavior" discipline echoed in [[Blameless Postmortem Culture]]. --- *Source: [[Web Operations]] (Allspaw & Robbins, O'Reilly 2010) — Ch 1 — Web Operations: The Career*