Forecasting
The Story is in the Residuals
December 23, 2019
In any field that involves heaps of data and information, details are everything. Load forecasting is no exception. If you’ve spent any amount of time building load forecast models, trying to improve existing ones, conducting out-of-sample tests, etc., then you know these kinds of processes and assessments require paying close attention to the details. “Does that coefficient make sense? What happened in October that made the residual so big? Why does my forecast for Tuesday look a little funky? What is my model missing?” These are some of the questions I find myself asking (or being asked) a lot, and invariably, it forces you to get your hands dirty.
One could argue that the art of building good models is in the details, or as I like to sometimes call them, the residuals. Sure, in-sample fit statistics are useful, but they don’t tell the whole story. Oftentimes overlooked, the residuals can tell a powerful story to those modelers willing to listen. They can reveal outliers and patterns or trends in the data that otherwise might not be easily identified by just looking at the data. And they can give the modeler a sense for what is and isn’t working in their model.
To view a residual chart in MetrixND, you simply go to the Err tab of the model window (press the eyeglasses button) and press the button that looks like a little residual chart on the toolbar. Whether you’re building a model from scratch or polishing off an existing one, this step is a must. What I like to do is take it a step further and plot the residuals against a key variable in my model and see if I’m capturing the right pattern or relationship in my data.
For example, whenever I get asked the question, “Why can’t we just drop average temperature into our linear regression model?” I always show a scatterplot like the one below that illustrates the nonlinear relationship between loads and temperatures.
A simple regression line isn’t going to capture this nonlinearity, but I think this point is really driven home when we put average temperature on the right-hand side of our linear regression equation and then plot the residuals against temperatures.
The pattern seen in this scatterplot tells us that our model is mis-specified. The horizontal line at zero is essentially our regression line, which is showing that we’re underpredicting loads at low and high temperatures and overpredicting at mid-range temperatures. If we specify our model correctly, then we would hope to see our residuals reduced to white noise with no discernible pattern (i.e., most of the data hovering around the zero line). Fortunately, when it comes to temperature, we can leverage a polynomial functional form or heating and cooling degree variables to accomplish this.
As the saying goes, “the devil is in the details.” Or as I like to say, “the story is in the residuals.” Perhaps it’s not as catchy, but I think it rings true when it comes to building good load forecast models.
Happy Holidays everyone! See you in the new year!
One could argue that the art of building good models is in the details, or as I like to sometimes call them, the residuals. Sure, in-sample fit statistics are useful, but they don’t tell the whole story. Oftentimes overlooked, the residuals can tell a powerful story to those modelers willing to listen. They can reveal outliers and patterns or trends in the data that otherwise might not be easily identified by just looking at the data. And they can give the modeler a sense for what is and isn’t working in their model.
To view a residual chart in MetrixND, you simply go to the Err tab of the model window (press the eyeglasses button) and press the button that looks like a little residual chart on the toolbar. Whether you’re building a model from scratch or polishing off an existing one, this step is a must. What I like to do is take it a step further and plot the residuals against a key variable in my model and see if I’m capturing the right pattern or relationship in my data.
For example, whenever I get asked the question, “Why can’t we just drop average temperature into our linear regression model?” I always show a scatterplot like the one below that illustrates the nonlinear relationship between loads and temperatures.
A simple regression line isn’t going to capture this nonlinearity, but I think this point is really driven home when we put average temperature on the right-hand side of our linear regression equation and then plot the residuals against temperatures.
The pattern seen in this scatterplot tells us that our model is mis-specified. The horizontal line at zero is essentially our regression line, which is showing that we’re underpredicting loads at low and high temperatures and overpredicting at mid-range temperatures. If we specify our model correctly, then we would hope to see our residuals reduced to white noise with no discernible pattern (i.e., most of the data hovering around the zero line). Fortunately, when it comes to temperature, we can leverage a polynomial functional form or heating and cooling degree variables to accomplish this.
As the saying goes, “the devil is in the details.” Or as I like to say, “the story is in the residuals.” Perhaps it’s not as catchy, but I think it rings true when it comes to building good load forecast models.
Happy Holidays everyone! See you in the new year!
Wystąpił błąd podczas przetwarzania szablonu.
The following has evaluated to null or missing:
==> authorContent.contentFields [in template "44616#44647#114455" at line 9, column 17]
----
Tip: It's the step after the last dot that caused this error, not those before it.
----
Tip: If the failing expression is known to legally refer to something that's sometimes null or missing, either specify a default value like myOptionalVar!myDefault, or use <#if myOptionalVar??>when-present<#else>when-missing</#if>. (These only cover the last step of the expression; to cover the whole expression, use parenthesis: (myOptionalVar.foo)!myDefault, (myOptionalVar.foo)??
----
----
FTL stack trace ("~" means nesting-related):
- Failed at: contentFields = authorContent.content... [in template "44616#44647#114455" at line 9, column 1]
----
1<#assign
2 webContentData = jsonFactoryUtil.createJSONObject(author.getData())
3 classPK = webContentData.classPK
4/>
5
6<#assign
7authorContent = restClient.get("/headless-delivery/v1.0/structured-contents/" + classPK + "?fields=contentFields%2CfriendlyUrlPath%2CtaxonomyCategoryBriefs")
8contentFields = authorContent.contentFields
9categories=authorContent.taxonomyCategoryBriefs
10authorContentData = jsonFactoryUtil.createJSONObject(authorContent)
11friendlyURL = authorContentData.friendlyUrlPath
12authorCategoryId = "0"
13/>
14
15<#list contentFields as contentField >
16 <#assign
17 contentFieldData = jsonFactoryUtil.createJSONObject(contentField)
18 name = contentField.name
19 />
20 <#if name == 'authorImage'>
21 <#if (contentField.contentFieldValue.image)??>
22 <#assign authorImageURL = contentField.contentFieldValue.image.contentUrl />
23 </#if>
24 </#if>
25 <#if name == 'authorName'>
26 <#assign authorName = contentField.contentFieldValue.data />
27 <#list categories as category >
28 <#if authorName == category.taxonomyCategoryName>
29 <#assign authorCategoryId = category.taxonomyCategoryId />
30 </#if>
31 </#list>
32 </#if>
33 <#if name == 'authorDescription'>
34 <#assign authorDescription = contentField.contentFieldValue.data />
35
36 </#if>
37
38 <#if name == 'authorJobTitle'>
39 <#assign authorJobTitle = contentField.contentFieldValue.data />
40
41 </#if>
42
43</#list>
44
45<div class="blog-author-info">
46 <#if authorImageURL??>
47 <img class="blog-author-img" id="author-image" src="${authorImageURL}" alt="" />
48 </#if>
49 <#if authorName??>
50 <#if authorName != "">
51 <p class="blog-author-name">By <a id="author-detail-page" href="/w/${friendlyURL}?filter_category_552298=${authorCategoryId}"><span id="author-full-name">${authorName}</span></a></p>
52 <hr />
53 </#if>
54 </#if>
55 <#if authorJobTitle??>
56 <#if authorJobTitle != "">
57 <p class="blog-author-title" id="author-job-title" >${authorJobTitle}</p>
58 <hr />
59 </#if>
60 </#if>
61 <#if authorDescription??>
62 <#if authorDescription != "" && authorDescription != "null" >
63 <p class="blog-author-desc" id="author-job-desc">${authorDescription}</p>
64 <hr />
65 </#if>
66 </#if>
67</div>
The following has evaluated to null or missing:
==> authorContent.contentFields [in template "44616#44647#114455" at line 9, column 17]
----
Tip: It's the step after the last dot that caused this error, not those before it.
----
Tip: If the failing expression is known to legally refer to something that's sometimes null or missing, either specify a default value like myOptionalVar!myDefault, or use <#if myOptionalVar??>when-present<#else>when-missing</#if>. (These only cover the last step of the expression; to cover the whole expression, use parenthesis: (myOptionalVar.foo)!myDefault, (myOptionalVar.foo)??
----
----
FTL stack trace ("~" means nesting-related):
- Failed at: contentFields = authorContent.content... [in template "44616#44647#114455" at line 9, column 1]
----
1<#assign
2 webContentData = jsonFactoryUtil.createJSONObject(author.getData())
3 classPK = webContentData.classPK
4/>
5
6<#assign
7authorContent = restClient.get("/headless-delivery/v1.0/structured-contents/" + classPK + "?fields=contentFields%2CfriendlyUrlPath%2CtaxonomyCategoryBriefs")
8contentFields = authorContent.contentFields
9categories=authorContent.taxonomyCategoryBriefs
10authorContentData = jsonFactoryUtil.createJSONObject(authorContent)
11friendlyURL = authorContentData.friendlyUrlPath
12authorCategoryId = "0"
13/>
14
15<#list contentFields as contentField >
16 <#assign
17 contentFieldData = jsonFactoryUtil.createJSONObject(contentField)
18 name = contentField.name
19 />
20 <#if name == 'authorImage'>
21 <#if (contentField.contentFieldValue.image)??>
22 <#assign authorImageURL = contentField.contentFieldValue.image.contentUrl />
23 </#if>
24 </#if>
25 <#if name == 'authorName'>
26 <#assign authorName = contentField.contentFieldValue.data />
27 <#list categories as category >
28 <#if authorName == category.taxonomyCategoryName>
29 <#assign authorCategoryId = category.taxonomyCategoryId />
30 </#if>
31 </#list>
32 </#if>
33 <#if name == 'authorDescription'>
34 <#assign authorDescription = contentField.contentFieldValue.data />
35
36 </#if>
37
38 <#if name == 'authorJobTitle'>
39 <#assign authorJobTitle = contentField.contentFieldValue.data />
40
41 </#if>
42
43</#list>
44
45<div class="blog-author-info">
46 <#if authorImageURL??>
47 <img class="blog-author-img" id="author-image" src="${authorImageURL}" alt="" />
48 </#if>
49 <#if authorName??>
50 <#if authorName != "">
51 <p class="blog-author-name">By <a id="author-detail-page" href="/w/${friendlyURL}?filter_category_552298=${authorCategoryId}"><span id="author-full-name">${authorName}</span></a></p>
52 <hr />
53 </#if>
54 </#if>
55 <#if authorJobTitle??>
56 <#if authorJobTitle != "">
57 <p class="blog-author-title" id="author-job-title" >${authorJobTitle}</p>
58 <hr />
59 </#if>
60 </#if>
61 <#if authorDescription??>
62 <#if authorDescription != "" && authorDescription != "null" >
63 <p class="blog-author-desc" id="author-job-desc">${authorDescription}</p>
64 <hr />
65 </#if>
66 </#if>
67</div>