Run Charts allow auditors to track performance measures over time to determine whether a problem exists, how significant it is, and whether performance is trending upward, downward, or fluctuating unpredictably. By displaying data visually, Run Charts make process variation, performance extremes, and gaps between actual and expected results easier to interpret. This step-by-step approach helps auditors gather meaningful data, identify trends, and support stronger findings.
This lesson is a preview from Graduate School USA's Analysis Techniques for Auditors Course.
The Run Chart is used to identify a potential problem and its extent. It does this by tracking and displaying changes in the aspect of performance being measured over a selected period of time (trend analysis). It creates a picture of what is happening in the situation the auditor is analyzing. The performance being tracked could be cycle time to complete a specific product or service, level of errors, efficiency, etc.
A Run Chart analysis can be used to identify a potential problem by showing:
- If performance is increasing
- If performance is decreasing
- Extremes in process variation
- how far performance is from a goal to criteria
- the extent of change in “before” and “after” performance due to installation of a new process, or some other intervention
Step 1: Determine What to Measure
The first step in constructing a Run Chart is selecting one key performance aspect to track over a period of time. Performance aspects subject to the Run Chart analysis include the following:
- Volume (amount or percent that occurs over a specified period)
- Timeliness (e.g., on time, cycle time & how long something takes)
- Errors (how many are incorrect)
- Waste (how much is reworked or rejected)
- Efficiency or cost per unit (is efficiency declining?)
- Economy (change in price)
After deciding what to measure, we need to decide on (1) the period of time over which we will collect data, and (2) the time interval of data collection (minute, hour, day, etc.).
U.S. Mint Example
The manager of one U.S. Mint location (that produces pennies, nickels, dimes, and quarters) has become concerned with the number of coins rejected over the past few months. Coins, after stamping, pass under an electronic sensor, which detects flaws in the stamping process. These flaws constitute rejects. The machine sensor does not count the number of rejects.
The manager has looked at the reject pile on a number of occasions over the past few months and has seen a wide variability in the volume of coins being rejected. He has asked you, as the internal auditor, to study the situation and determine the trend in the reject rate and ultimately what is causing the variations.
You decide to use a Run Chart analysis. You will want to measure the quantity of coin rejects at specific intervals over a specific time period. Because the Mint does not count the number of coins rejected, you choose to use the “rejection rate” rather than the actual coin reject quantity. The “rejection rate” is computed by dividing the pounds of coins rejected by the pounds of coins produced.
- Cycle time. You choose to measure rejects on a daily basis for one month, from the prior month or the next month
- Volume of rejects. You choose to weigh the pounds of coins rejected at the end of operations each day. Neither Mint staff nor its coin stamping equipment counts the number of coin rejects.
- Volume of coins produced. You choose to use the pounds of coins produced, including rejects. Daily production levels vary.
Step 2: Collect the Data
Collect data using a data collection instrument. The instrument should provide for recording data (1) on the total quantity of whatever is being measured (e.g., transactions), (2) for each selected time interval. If the concern is with errors, the instrument should provide for recording both the total quantity and the error quantity.
U.S. Mint Example
For the U.S. Mint example, your data collection instrument would provide for collecting data, for each day, on: (1) the weight of all coins produced, (2) the weight of coins rejected, and (3) the computed rejection rate.
You decide to use a Run Chart analysis and collect data over the next month to determine the variation in the number of rejects on a daily basis. You choose to determine the reject rate by weighing the rejected coins at the end of operations each day.
Step 3: Draw the Graph
Drawing the graph consists of three simple steps.
- Name the graph. Describe what the graph will be measuring
- Draw the vertical axis. The aspect being measured is always on the vertical axis and is shown in the number of occurrences or percentages
- Draw the horizontal axis. The time interval or other measurement base is always shown on the horizontal axis
For our example, the:
- Name of the chart is “Volume of coin rejections for May 20XX.”
- Vertical axis is labeled “Percent of coins rejected.” Due to the difference in coin weight, the number plotted on the vertical axis is derived by dividing pounds rejected by pounds produced
- The horizontal axis is labeled “Days in May.”
Step 4: Plot the Data
Collect data for your chosen measurement. Each data point should then be plotted on the graph in the appropriate location, given its level of occurrence (or percentage) and time interval. If you connect the points, it will be easier to interpret the graph. Finally, calculate the average occurrence or percentage and plot that on the graph. The average line will help you see variances in the volume of rejects on a daily basis.
