Real-time accuracy of the ads index

In this short note, we provide an assessment of the real-time accuracy of the ADS index using the information from the updates in the two-year period since the inception of the index. Our measure of accuracy in this context is whether or not the values of the index as it was computed in real time are close to the values as they appear today. In other words, we would consider the index accurate if the revisions to the index are small.
Since the index has a daily frequency and is updated at least once a week, we consider 1. A researcher who looks at the very last number produced in every update 2. A researcher who looks at the “right line”1 3. A researcher who looks at the average of the last complete calendar month.
Due to the timing of the underlying components of the index, in most updates the last available day is the Saturday that corresponds to the last day of the seven-day period covered by the latest initial jobless claims data. Also, since employment and industrial production (IP) are typically the …rst two of the monthly components to be released in a month, the right line moves when both of these components are released. This typically occurs with the release of IP around the middle of the following month.
Using the publicly available historical values of the index as they were released, we conduct the following exercise. We identify the days on which IP for the previous month is released. On these days we record the three numbers corresponding to the three variations listed above. We compare these numbers with the corresponding values from the December 3, 2010 update. There are two important caveats to this exercise. First, we use the latest values 1 The right line shown on some …gures on the ADS website shows the last day of the month for which at least two monthly indicators are available.
of the index as the “truth,”which means we compare the values of the index in two di¤erent updates. Since the variation in the underlying statistical model is virtually nonexistent, the di¤erences re‡ect data revisions. To the extent that there have been (unexpected) large data revisions, there may be large discrepancies. Second, this exercise is subject to what is commonly referred to as the “apples and oranges” problem in the real-time data literature.
In particular, the earlier values are based on more “mature” data – data that have been revised many times while the more recent values are still relatively preliminary.
Figures 1-3 show the results of this exercise.2 In all …gures the blue line shows the value of that particular measurement of the index (since they di¤er in each variation) as of the December 3, 2010 update and the red line shows the corresponding value in real time. The vertical distance between the two lines measures the revision to the index. The sample starts with November 2008, the …rst complete month as of the …rst update of ADS, and ends with the October 2010 observation. The gray shadings in the …gures re‡ect NBER recessions. We also provide the correlation of the two lines in every …gure. The results show that using any of the three ways of looking at the ADS index, the real-time version provides a lot of useful information. The real-time version and the current version are highly correlated, and a visual inspection reveals that the recovery of early 2009 and the relative calm of 2010 is very well captured in real time.3 We also conducted some simple statistical tests (not reported) that show that the real-time version is an unbiased forecast of the current version.
2 The underlying data for these …gures are posted on the ADS website.
3 One may be worried about the large discrepancy in November and December 2008, especially in Figure 2:99 for December 13, 2008 (the last observation as of the December 18, 2008 update), while in real time the value was 1:23: An inspection of later releases reveals that the update on January 16, 2009, the one that corresponds to the release of IP, shows a value of result of a large downward revision of the November value. Therefore, the discrepancy between the real-timeand the current versions of the index is due to large data revisions. It is unfortunate that this occurs rightaround the turning point, making the detection of the turning point slightly more di¢ cult. However, it isalso well known that initial releases have a hard time detecting turning points, and in that sense, the resultis not very surprising.
Figure 1 : Accuracy of the Last Point of the ADS Index
12 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11 Current VintageLast Observation for the Update Figure 2 : Accuracy of the "Right Line" of the ADS Index
11 12 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 Current VintageRight Line for the Month (Extracted Approx. 15 Days After the End of the Month) Figure 3 : Accuracy of the Average of the Previous Month of the ADS Index
11 12 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 Current VintageAverage of the Month Immediately Before the Right Line

Source: http://blog.zhaw.ch/idp/sefblog/uploads/ads_accuracy.pdf