Forecasting
Aggregation Bias Strikes Again!
November 07, 2018
The issue of aggregation bias should not be a new concept to any of us in the energy forecasting world. In April, my colleague David Simons posted about the potential pitfalls of aggregation bias when calculating weather concepts, pointing out the importance of calculating CDD and HDD concepts prior to averaging.
Traditionally, CDD and HDD concepts are constructed by first calculating daily average temperature— either averaging the 24-hourly observations or taking the average of the daily maximum and minimum— and then calculating CDDs and HDDs for the desired breakpoints. This is, by definition, aggregation bias, averaging before calculating the nonlinear values. This got me thinking, should we be calculating CDD and HDD concepts on the hourly values first and then averaging for the day? If you do not have historical hourly weather, the point is moot. But let’s assume you do. The table below contains 24 hourly temperature observations. The average temperature for the day is 63.8 degrees, calculating a CDD in base 65 would result in zero CDDs, but if you first calculated the CDDs and then averaged, you would end up with two CDDs. Was there cooling during the day or not? Thinking from a residential customer’s perspective, a few warm hours in the middle of the day wouldn’t necessitate turning on the air conditioner. During the height of summer or winter, it will not matter—when all hourly observations are above or below the breakpoint, the two methods will return the exact same value.
I set out to test if using hourly constructed CDD and HDD concepts would improve daily or monthly forecasting models. Using historical weather and system load data for a utility in the southeast, I built daily and monthly forecasting models. The daily model was simple—multiple HDD and CDD concepts, lags of weather, binaries for days of the week and holidays, hours of light, as well as a simple trend. I created two identical models, the only difference being how the daily weather concepts were constructed; one with the traditional approach of first averaging the daily temperature and then calculating weather concepts and the other calculated weather concepts from the hourly values, and then averaged. The models were estimated using three years of daily data, with the dependent variable being daily energy. The model with the traditional weather concepts had an Adj. R2 0.957 and a MAPE (Mean Absolute Percent Error) of 2.84 percent. The alternative model had an Adj. R2 0.951 and a MAPE of 3.03 percent. While not drastically different, the traditional weather concepts did provide a better fit and smaller errors. The monthly model was estimated with eight years of data and showed similar results with the traditional weather model having an Adj. R2 0.966 and a MAPE of 2.15 percent, while the alternative model had an Adj. R2 0.961 and a MAPE of 2.30 percent.
Conclusion: It appears there is nothing to be gained from calculating weather concepts from hourly values and then averaging.
Traditionally, CDD and HDD concepts are constructed by first calculating daily average temperature— either averaging the 24-hourly observations or taking the average of the daily maximum and minimum— and then calculating CDDs and HDDs for the desired breakpoints. This is, by definition, aggregation bias, averaging before calculating the nonlinear values. This got me thinking, should we be calculating CDD and HDD concepts on the hourly values first and then averaging for the day? If you do not have historical hourly weather, the point is moot. But let’s assume you do. The table below contains 24 hourly temperature observations. The average temperature for the day is 63.8 degrees, calculating a CDD in base 65 would result in zero CDDs, but if you first calculated the CDDs and then averaged, you would end up with two CDDs. Was there cooling during the day or not? Thinking from a residential customer’s perspective, a few warm hours in the middle of the day wouldn’t necessitate turning on the air conditioner. During the height of summer or winter, it will not matter—when all hourly observations are above or below the breakpoint, the two methods will return the exact same value.
I set out to test if using hourly constructed CDD and HDD concepts would improve daily or monthly forecasting models. Using historical weather and system load data for a utility in the southeast, I built daily and monthly forecasting models. The daily model was simple—multiple HDD and CDD concepts, lags of weather, binaries for days of the week and holidays, hours of light, as well as a simple trend. I created two identical models, the only difference being how the daily weather concepts were constructed; one with the traditional approach of first averaging the daily temperature and then calculating weather concepts and the other calculated weather concepts from the hourly values, and then averaged. The models were estimated using three years of daily data, with the dependent variable being daily energy. The model with the traditional weather concepts had an Adj. R2 0.957 and a MAPE (Mean Absolute Percent Error) of 2.84 percent. The alternative model had an Adj. R2 0.951 and a MAPE of 3.03 percent. While not drastically different, the traditional weather concepts did provide a better fit and smaller errors. The monthly model was estimated with eight years of data and showed similar results with the traditional weather model having an Adj. R2 0.966 and a MAPE of 2.15 percent, while the alternative model had an Adj. R2 0.961 and a MAPE of 2.30 percent.
Conclusion: It appears there is nothing to be gained from calculating weather concepts from hourly values and then averaging.
Si è verificato un errore nell'elaborarazione del modello.
The following has evaluated to null or missing:
==> authorContent.contentFields [in template "44616#44647#114455" at line 9, column 17]
----
Tip: It's the step after the last dot that caused this error, not those before it.
----
Tip: If the failing expression is known to legally refer to something that's sometimes null or missing, either specify a default value like myOptionalVar!myDefault, or use <#if myOptionalVar??>when-present<#else>when-missing</#if>. (These only cover the last step of the expression; to cover the whole expression, use parenthesis: (myOptionalVar.foo)!myDefault, (myOptionalVar.foo)??
----
----
FTL stack trace ("~" means nesting-related):
- Failed at: contentFields = authorContent.content... [in template "44616#44647#114455" at line 9, column 1]
----
1<#assign
2 webContentData = jsonFactoryUtil.createJSONObject(author.getData())
3 classPK = webContentData.classPK
4/>
5
6<#assign
7authorContent = restClient.get("/headless-delivery/v1.0/structured-contents/" + classPK + "?fields=contentFields%2CfriendlyUrlPath%2CtaxonomyCategoryBriefs")
8contentFields = authorContent.contentFields
9categories=authorContent.taxonomyCategoryBriefs
10authorContentData = jsonFactoryUtil.createJSONObject(authorContent)
11friendlyURL = authorContentData.friendlyUrlPath
12authorCategoryId = "0"
13/>
14
15<#list contentFields as contentField >
16 <#assign
17 contentFieldData = jsonFactoryUtil.createJSONObject(contentField)
18 name = contentField.name
19 />
20 <#if name == 'authorImage'>
21 <#if (contentField.contentFieldValue.image)??>
22 <#assign authorImageURL = contentField.contentFieldValue.image.contentUrl />
23 </#if>
24 </#if>
25 <#if name == 'authorName'>
26 <#assign authorName = contentField.contentFieldValue.data />
27 <#list categories as category >
28 <#if authorName == category.taxonomyCategoryName>
29 <#assign authorCategoryId = category.taxonomyCategoryId />
30 </#if>
31 </#list>
32 </#if>
33 <#if name == 'authorDescription'>
34 <#assign authorDescription = contentField.contentFieldValue.data />
35
36 </#if>
37
38 <#if name == 'authorJobTitle'>
39 <#assign authorJobTitle = contentField.contentFieldValue.data />
40
41 </#if>
42
43</#list>
44
45<div class="blog-author-info">
46 <#if authorImageURL??>
47 <img class="blog-author-img" id="author-image" src="${authorImageURL}" alt="" />
48 </#if>
49 <#if authorName??>
50 <#if authorName != "">
51 <p class="blog-author-name">By <a id="author-detail-page" href="/w/${friendlyURL}?filter_category_552298=${authorCategoryId}"><span id="author-full-name">${authorName}</span></a></p>
52 <hr />
53 </#if>
54 </#if>
55 <#if authorJobTitle??>
56 <#if authorJobTitle != "">
57 <p class="blog-author-title" id="author-job-title" >${authorJobTitle}</p>
58 <hr />
59 </#if>
60 </#if>
61 <#if authorDescription??>
62 <#if authorDescription != "" && authorDescription != "null" >
63 <p class="blog-author-desc" id="author-job-desc">${authorDescription}</p>
64 <hr />
65 </#if>
66 </#if>
67</div>
The following has evaluated to null or missing:
==> authorContent.contentFields [in template "44616#44647#114455" at line 9, column 17]
----
Tip: It's the step after the last dot that caused this error, not those before it.
----
Tip: If the failing expression is known to legally refer to something that's sometimes null or missing, either specify a default value like myOptionalVar!myDefault, or use <#if myOptionalVar??>when-present<#else>when-missing</#if>. (These only cover the last step of the expression; to cover the whole expression, use parenthesis: (myOptionalVar.foo)!myDefault, (myOptionalVar.foo)??
----
----
FTL stack trace ("~" means nesting-related):
- Failed at: contentFields = authorContent.content... [in template "44616#44647#114455" at line 9, column 1]
----
1<#assign
2 webContentData = jsonFactoryUtil.createJSONObject(author.getData())
3 classPK = webContentData.classPK
4/>
5
6<#assign
7authorContent = restClient.get("/headless-delivery/v1.0/structured-contents/" + classPK + "?fields=contentFields%2CfriendlyUrlPath%2CtaxonomyCategoryBriefs")
8contentFields = authorContent.contentFields
9categories=authorContent.taxonomyCategoryBriefs
10authorContentData = jsonFactoryUtil.createJSONObject(authorContent)
11friendlyURL = authorContentData.friendlyUrlPath
12authorCategoryId = "0"
13/>
14
15<#list contentFields as contentField >
16 <#assign
17 contentFieldData = jsonFactoryUtil.createJSONObject(contentField)
18 name = contentField.name
19 />
20 <#if name == 'authorImage'>
21 <#if (contentField.contentFieldValue.image)??>
22 <#assign authorImageURL = contentField.contentFieldValue.image.contentUrl />
23 </#if>
24 </#if>
25 <#if name == 'authorName'>
26 <#assign authorName = contentField.contentFieldValue.data />
27 <#list categories as category >
28 <#if authorName == category.taxonomyCategoryName>
29 <#assign authorCategoryId = category.taxonomyCategoryId />
30 </#if>
31 </#list>
32 </#if>
33 <#if name == 'authorDescription'>
34 <#assign authorDescription = contentField.contentFieldValue.data />
35
36 </#if>
37
38 <#if name == 'authorJobTitle'>
39 <#assign authorJobTitle = contentField.contentFieldValue.data />
40
41 </#if>
42
43</#list>
44
45<div class="blog-author-info">
46 <#if authorImageURL??>
47 <img class="blog-author-img" id="author-image" src="${authorImageURL}" alt="" />
48 </#if>
49 <#if authorName??>
50 <#if authorName != "">
51 <p class="blog-author-name">By <a id="author-detail-page" href="/w/${friendlyURL}?filter_category_552298=${authorCategoryId}"><span id="author-full-name">${authorName}</span></a></p>
52 <hr />
53 </#if>
54 </#if>
55 <#if authorJobTitle??>
56 <#if authorJobTitle != "">
57 <p class="blog-author-title" id="author-job-title" >${authorJobTitle}</p>
58 <hr />
59 </#if>
60 </#if>
61 <#if authorDescription??>
62 <#if authorDescription != "" && authorDescription != "null" >
63 <p class="blog-author-desc" id="author-job-desc">${authorDescription}</p>
64 <hr />
65 </#if>
66 </#if>
67</div>