- About Us
- Infosys Knowledge Institute
Utilities are embracing new technologies to modernize their electric grid and cope with the changing load profiles created by distributed energy resources such as solar, electric vehicles and battery storage. The path forward for trailblazers is often the most challenging. In the spirit of the early pioneers, this paper attempts to share some of the lessons learned.
In previous papers we discussed the case for grid modernization and grid modernization capabilities. In this paper we will discuss the top 10 lessons learned from pioneering grid modernization efforts in order to assist others on their journey. While such efforts span both grid operations and planning and engineering, this paper focuses primarily on planning and engineering.
Top 10 grid modernization lessons learned:
As utilities try to rationalize data models for managing digital information across various domains, they will likely be haunted by decisions made decades ago. The processing power available today makes use cases possible that couldn’t even be envisioned just a few short years ago. An accurate grid connectivity model requires meter information often stored in customer support systems to be connected to circuit and asset information stored in distribution and transmission systems. Efforts to stitch this information together in a way that accurately represents the electrical network can be stymied by data models and systems that failed to envision the future need.
One of the key grid modernization use cases is the ability to forecast load at a given node over time in order to develop capital and noncapital response solutions. Developing a long-term time series forecast for every node on the network requires massive amounts of historical load data, economic data, load growth data, weather data, geographic information system (GIS) data, distributed energy resources (DER) data and grid connectivity information. When dealing with such large volumes of historical data, it can be easy to overlook significant data gaps that could adversely affect the forecast. Therefore, it is advisable to put a data quality program in place to identify data gaps and develop a plan to fix them before finalizing any release dates.
Logical aggregation of advanced metering infrastructure (AMI) data can be a substitute for missing supervisory control and data acquisition (SCADA) data, provided electrical hierarchies are properly constructed. Incomplete electrical hierarchies are common since they’ve never been used to obtain load and consumption profiles at such specific levels. It can be challenging to disentangle legitimate data quality issues from phantom data quality issues. For example, assets that haven’t been energized might appear exactly the same as assets that have been incorrectly mapped to their consumption data. That’s because they will both appear as electrical hierarchies with no usage data if you haven’t accounted for the asset’s operational state.
Another example of missed data validation is the availability of accurate weather data in the utilities’ territory. This validation must be complete for three dimensions to effectively leverage weather data in forecast generation:
Must complete weather data mapping (longitude and latitude) to nodes in the network.
Create distance limitations between node locations and the weather stations.
Ensure completeness of weather data. (In our experience, up to 20% of hourly data may be missing from weather stations.)
New systems and processes that rely on information created and maintained in legacy data structures run the risk of uncovering data quality issues hidden within existing production systems. Until someone starts using data in a novel way, these data quality issues are not critical and won’t get fixed. A comprehensive data quality program should be put in place to address data quality issues by:
Clearly identifying the systems of record and system of truth for each data entity.
While the types of data quality issues identified by each utility will be unique to their own ecosystem, there are things to look out for:
To deliver grid modernization planning capabilities, multiple platforms, IT and business processes, execution methodologies, and third-party vendors and their tools need to be integrated with a clear line of sight to execution. Without this, it’s easy to miss cross-capability dependency, overlook an architecture inclusion guideline, or simply not understand development and changes on one work stream or platform and their impact on the others. Poor planning is a fundamental cause of failure, especially in such complex transformations.
To ensure successful execution, the following elements of integrated planning must be considered:
Any large program that is delivered with multiple delivery methodologies (Waterfall and Agile) can quickly become exceedingly complex to coordinate if not managed carefully. It is helpful to think of a large transformation in terms of components, applications, tracks and releases.
Alignment milestones are critical for projects using both Agile and Waterfall methodologies
Methodology (Agile vs. Waterfall) is generally determined by the team, and teams are typically organized around applications. It is strongly advised that all members within the same application team share the same delivery methodology. Given that most large enterprises are at some point on their Agile maturity journey, it is highly unlikely that all the application teams across the enterprise are using the same methodology.Therefore, it is highly likely that applications delivered to large enterprises will be either an Agile or Waterfall methodology that are dependent on an application being delivered with the other methodology. For a program that includes projects being delivered through both methodologies, it is critical to have key alignment milestones. This will help avoid the risk of change being introduced by Agile products that phase-gated (otherwise known as Waterfall) projects can’t handle without a change request. To coordinate between these two delivery methodologies, we recommend the following key alignment milestones:
As more and more organizations embrace Agile delivery as a way to speed up velocity and improve quality, we have seen a number of organizations stumble in their initial steps down the Agile delivery path. We advise taking a mindful approach to determining if and when a product should be delivered using an Agile methodology. Methodologies are one tool in the tool kit, and it is important to pick the right tool for the job. We have found the following criteria useful to decide which products should move to an Agile delivery methodology and when to make that move.
Figure 1: How to align different execution methodologies
The data volumes required for time series load profiling and forecasting for a large utility are massive. A utility with five million smart meters that generate four readings an hour would create 175 billion readings per year. Those AMI readings would then be combined with DER and SCADA data. If there are half a million DERs in the network, there could be 17.5 billion readings a year. Assuming there are 5,000 nodes combining A-stations, B-stations and circuits in the network, it would need to create 17.5 billion (96 intervals/day * 365 days * 5,000 nodes * 10 SCADA points) records per year in order to create load profiles for all the nodes in the network. Adding other not so significant entities like projects, network components, etc. can add up to nearly 200 billion records per year. Ten years of historical data and a 10-year forecast would represent four trillion records. If you consider the average record size is 100 bytes, you are looking at 400 terabytes of storage in an environment. It is critical to adequately plan for these data volumes, not just in production, but also in lower level environments such as development, testing and performance testing.
As was discussed at length in a previous article on grid modernization capabilities, the planning and engineering capabilities of a modern grid are highly dependent on a robust data lake. Once the wide variety of information (assets, grid connectivity, usage, weather, economics, load growth, DER, programs, GIS, etc.) required for grid modernization is made available within a data lake, there will likely be a wide variety of consumers across the enterprise interested in this data. Conflicting program requirements on the data lake can wreak havoc on a schedule if not managed properly. That’s why it’s critical to develop a detailed availability and upgrade schedule for any shared environments.
When designing solutions to novel problems, you often don’t know what you don’t know. It is critical to see early prototypes of the solution with real data in order to provide feedback on the solution design.
This iterative process to solution development provides a critical feedback loop for any data visualization or forecasting solution. To accommodate this feedback loop in data intensive solutions, it is imperative to plan for multiple rounds of ad hoc data provisioning. It is equally as important that any business requirements include the data provisioning requirements as early as possible in the solution definition process. This will allow adequate time for data provisioning. With the data volumes involved, data provisioning will often be the long pole in the tent.
Developing predictive models is often an exercise in trial and error as hypotheses are tested with data and then adjusted and tested again. The algorithms are constantly tuned until they best fit the historical data. This tuning process can mean adding more data sources and changing existing and historical data sources. These changes can wreak havoc on a schedule, so it is important to:
While every utility’s grid modernization journey will proceed at a different pace, these lessons learned can help you avoid some of the pitfalls inherent in building solutions to novel problems.