Making RUM actionable

Real User Measurements (or Monitoring) has seen a steep increase in use in the past year. Companies see it as the next Holy Grail in web performance monitoring and are shifting their attention to it. With current RUM techniques, it’s still impossible to pinpoint what the cause of a slowdown is. How do you get insights from the RUM data and how fast can you notice shifts in trends? The second post in our series about RUM will dive into it. Missed the first about the general introduction of RUM? Click here.

Dashboard of RUM events and timings A picture of our Dashboard, making RUM actionable for developers, performance and system engineering.

Getting the data

Before you can get insights, you need to have the data. For standard providers that measure pageload, this means implementing a piece of JavaScript. For custom events, you will need your own solution. At Spil we have Spil Event Tracker (SET) and the Graphite Data Collector.

Processing custom RUM events: batched and realtime

Standard RUM providers only have limited support for custom events. With our own Spil Event Tracker (SET), we are able to gain insights into data like this. SET runs with multiple Map/Reduce jobs in our Disco cluster, which is comparable to Hadoop. Writing the Map/Reduce jobs and scheduling them with our JobRunner (open source) still takes time and this can be an issue for small tests. The batched nature of the jobs is also not the optimal solution in some cases, like verifying business ideas on the spot.

For experimentation, we’ve set-up a system to get near real-time metrics from our RUM data. This is called the Graphite Data Collector. A whitelist and StatsD and used to collect the data that is gathered with JavaScript from the end-user.

Visualizing

After the data has been collected, you still need to make it available to the stakeholders. A good way is to visualize it in a graph. There are many ways to visualize the data. We went with Graphite because it’s fast and a popular tool for graphing metrics. Other visualization programs are also possible, like D3, HighCharts or Flash.

Graphite has a near-realtime refresh rate. Changes are shown when they happen. This gives very fast results and will make your business processes a lot more data-driven.

If you have people watching the graphs after a release and the metrics are specific enough to find regressions, you’ve succeeded in making the RUM actionable.

In future posts we will dive deeper in how we use RUM in the specific instances and how RUM relates to synthetic measurements.