Forecasting
I Get by With a Little Help from My Friends, and Google
It almost goes without saying that most utilities have seen a noticeable deviation in their electricity sales in 2020 due to the pandemic. But the question remains, how long will the deviation persist and what does the path ahead look like? Load forecasters the world over are peering into their crystal balls to try to figure this out.
A big part of the challenge is finding a driver that forecasters can use in their models to help capture the variation in sales due to behavioral changes from various COVID-19 mitigation policies. And if you do find a driver, with any luck you can forecast it without too much heartburn. Coincidentally, Google has been publishing daily anonymized COVID-19 Community Mobility Data that shows deviations from baseline location data by state, county and country from users who have turned on their Location History setting on their mobile devices. A few of my colleagues and I have leveraged this data in our forecast models, and the results have proven quite favorable.
The comprehensive dataset is available for download in CSV format on Google’s mobility data website, free of charge. Just scroll down to “Community Mobility Reports” and click on the “Global CSV” option to download the CSV file. The file is quite large (about 240 MB and counting) with too many records to fully load into Excel, and so I recommend opening it in something like Notepad or Notepad++ and copying and pasting the relevant data into a spreadsheet. You’ll have to do a little wrangling to get the data in a useable form, but surprisingly not much.
This data represents the percentage change in people’s visits to – or time spent – in six categories of places relative to the defined “baseline day,” or median value for that day-of-the-week from the period of Jan. 3 – Feb. 6, 2020. The categories are retail and recreation, grocery and pharmacy, parks, transit stations, workplaces, and residential. To give you a visual, here is what the data for the state of California looks like:
From this visual, we can see there’s a positive percentage change in the residential category and a negative one in workplaces as people shift to spending more time at home and little to no time in their place of work. Retail and transit are also down as people are shopping less and not taking public transportation. Grocery and pharmacy is down as well, but not as much as other categories because people obviously still need to buy food and medications. These percentage deviations appear to have stabilized since June, which makes forecasting this data a little less intimidating.
One thing that pops out from looking at the data is there’s a well-defined day-type pattern (i.e., weekend vs. weekday) for the residential and workplace categories. That is, there’s less of a change on weekends because people were already home and not at work before the pandemic took off. The large spikes are for holidays, as those days reflect a significant change relative to Google’s baseline. Retail also has a day-type pattern, albeit a little less well-defined. For this reason, I found the retail, workplace and residential categories to be the most applicable and useful for predicting loads in this COVID-19 world. And since the data are of daily frequency, you can leverage them in a daily model, or run them through billing cycles and incorporate them into a monthly SAE model.
Going with the latter approach, I started with a “business as usual” Residential SAE model (i.e., one that’s estimated with data through February 2020 so the COVID-19 period data does not influence the model coefficients). What the model shows is that residential use per customer has been higher since April relative to where it should have been subject to the actual weather that occurred.
But incorporating Google’s mobility data into the model helps to close this gap. Moreover, forecasting what we think the percentage changes in the relevant categories will be gives us a better projection for how the rest of the year might shake out.
Undoubtedly, this data is not perfect. For example, the baseline days probably aren’t representative of the true baseline, and Google is aware of this too. And using them certainly won’t remove all of the wrenches this pandemic has thrown into our forecast models. But they just might help to tighten things up and yield a more reasonable load forecast.
Google states that the data will be available for as long as public health officials find them useful, but who knows how long that may be. I don’t think I will try and forecast that. But with any luck, that will be just long enough.
Shout out to the folks in the Operational Forecasting Team at AEMO for calling this data to our attention!
Si è verificato un errore nell'elaborarazione del modello.
The following has evaluated to null or missing:
==> authorContent.contentFields [in template "44616#44647#114455" at line 9, column 17]
----
Tip: It's the step after the last dot that caused this error, not those before it.
----
Tip: If the failing expression is known to legally refer to something that's sometimes null or missing, either specify a default value like myOptionalVar!myDefault, or use <#if myOptionalVar??>when-present<#else>when-missing</#if>. (These only cover the last step of the expression; to cover the whole expression, use parenthesis: (myOptionalVar.foo)!myDefault, (myOptionalVar.foo)??
----
----
FTL stack trace ("~" means nesting-related):
- Failed at: contentFields = authorContent.content... [in template "44616#44647#114455" at line 9, column 1]
----
1<#assign
2 webContentData = jsonFactoryUtil.createJSONObject(author.getData())
3 classPK = webContentData.classPK
4/>
5
6<#assign
7authorContent = restClient.get("/headless-delivery/v1.0/structured-contents/" + classPK + "?fields=contentFields%2CfriendlyUrlPath%2CtaxonomyCategoryBriefs")
8contentFields = authorContent.contentFields
9categories=authorContent.taxonomyCategoryBriefs
10authorContentData = jsonFactoryUtil.createJSONObject(authorContent)
11friendlyURL = authorContentData.friendlyUrlPath
12authorCategoryId = "0"
13/>
14
15<#list contentFields as contentField >
16 <#assign
17 contentFieldData = jsonFactoryUtil.createJSONObject(contentField)
18 name = contentField.name
19 />
20 <#if name == 'authorImage'>
21 <#if (contentField.contentFieldValue.image)??>
22 <#assign authorImageURL = contentField.contentFieldValue.image.contentUrl />
23 </#if>
24 </#if>
25 <#if name == 'authorName'>
26 <#assign authorName = contentField.contentFieldValue.data />
27 <#list categories as category >
28 <#if authorName == category.taxonomyCategoryName>
29 <#assign authorCategoryId = category.taxonomyCategoryId />
30 </#if>
31 </#list>
32 </#if>
33 <#if name == 'authorDescription'>
34 <#assign authorDescription = contentField.contentFieldValue.data />
35
36 </#if>
37
38 <#if name == 'authorJobTitle'>
39 <#assign authorJobTitle = contentField.contentFieldValue.data />
40
41 </#if>
42
43</#list>
44
45<div class="blog-author-info">
46 <#if authorImageURL??>
47 <img class="blog-author-img" id="author-image" src="${authorImageURL}" alt="" />
48 </#if>
49 <#if authorName??>
50 <#if authorName != "">
51 <p class="blog-author-name">By <a id="author-detail-page" href="/w/${friendlyURL}?filter_category_552298=${authorCategoryId}"><span id="author-full-name">${authorName}</span></a></p>
52 <hr />
53 </#if>
54 </#if>
55 <#if authorJobTitle??>
56 <#if authorJobTitle != "">
57 <p class="blog-author-title" id="author-job-title" >${authorJobTitle}</p>
58 <hr />
59 </#if>
60 </#if>
61 <#if authorDescription??>
62 <#if authorDescription != "" && authorDescription != "null" >
63 <p class="blog-author-desc" id="author-job-desc">${authorDescription}</p>
64 <hr />
65 </#if>
66 </#if>
67</div>
The following has evaluated to null or missing: ==> authorContent.contentFields [in template "44616#44647#114455" at line 9, column 17] ---- Tip: It's the step after the last dot that caused this error, not those before it. ---- Tip: If the failing expression is known to legally refer to something that's sometimes null or missing, either specify a default value like myOptionalVar!myDefault, or use <#if myOptionalVar??>when-present<#else>when-missing</#if>. (These only cover the last step of the expression; to cover the whole expression, use parenthesis: (myOptionalVar.foo)!myDefault, (myOptionalVar.foo)?? ---- ---- FTL stack trace ("~" means nesting-related): - Failed at: contentFields = authorContent.content... [in template "44616#44647#114455" at line 9, column 1] ----
1<#assign
2 webContentData = jsonFactoryUtil.createJSONObject(author.getData())
3 classPK = webContentData.classPK
4/>
5
6<#assign
7authorContent = restClient.get("/headless-delivery/v1.0/structured-contents/" + classPK + "?fields=contentFields%2CfriendlyUrlPath%2CtaxonomyCategoryBriefs")
8contentFields = authorContent.contentFields
9categories=authorContent.taxonomyCategoryBriefs
10authorContentData = jsonFactoryUtil.createJSONObject(authorContent)
11friendlyURL = authorContentData.friendlyUrlPath
12authorCategoryId = "0"
13/>
14
15<#list contentFields as contentField >
16 <#assign
17 contentFieldData = jsonFactoryUtil.createJSONObject(contentField)
18 name = contentField.name
19 />
20 <#if name == 'authorImage'>
21 <#if (contentField.contentFieldValue.image)??>
22 <#assign authorImageURL = contentField.contentFieldValue.image.contentUrl />
23 </#if>
24 </#if>
25 <#if name == 'authorName'>
26 <#assign authorName = contentField.contentFieldValue.data />
27 <#list categories as category >
28 <#if authorName == category.taxonomyCategoryName>
29 <#assign authorCategoryId = category.taxonomyCategoryId />
30 </#if>
31 </#list>
32 </#if>
33 <#if name == 'authorDescription'>
34 <#assign authorDescription = contentField.contentFieldValue.data />
35
36 </#if>
37
38 <#if name == 'authorJobTitle'>
39 <#assign authorJobTitle = contentField.contentFieldValue.data />
40
41 </#if>
42
43</#list>
44
45<div class="blog-author-info">
46 <#if authorImageURL??>
47 <img class="blog-author-img" id="author-image" src="${authorImageURL}" alt="" />
48 </#if>
49 <#if authorName??>
50 <#if authorName != "">
51 <p class="blog-author-name">By <a id="author-detail-page" href="/w/${friendlyURL}?filter_category_552298=${authorCategoryId}"><span id="author-full-name">${authorName}</span></a></p>
52 <hr />
53 </#if>
54 </#if>
55 <#if authorJobTitle??>
56 <#if authorJobTitle != "">
57 <p class="blog-author-title" id="author-job-title" >${authorJobTitle}</p>
58 <hr />
59 </#if>
60 </#if>
61 <#if authorDescription??>
62 <#if authorDescription != "" && authorDescription != "null" >
63 <p class="blog-author-desc" id="author-job-desc">${authorDescription}</p>
64 <hr />
65 </#if>
66 </#if>
67</div>