ASE/Kalkitech’s Kalki.io Data Hub for DER data Integration in the cloud

Challenges

The electrical distribution grid is transitioning with the deployment of DERs and Electric Vehicles (EVs) into a highly digitized and bi-directional grid. Growth of DER is accelerating at a record pace due to an increase in government support for the global clean energy goals and increased market adoption of renewable power and EV. This growth poses critical challenges to the ageing grid infrastructure and the stability of the grid if they also don’t keep pace with the DER and EV transition.

When a DER is connected to the grid, if proper care is not taken in terms of analysing the impact and supporting distribution upgrades if needed, it will have serious consequences on the grid. The proliferation of rooftop solar and EVs will require the grid operator to have better visibility on the grid, it would be essential in the future for DERs to support utilities in maintaining grid stability by providing grid services. Even though the impact of an individual DER is minimal, in aggregate they can be a significant strain on the grid. To mitigate this, standards such as IEEE 1547-2018 have brought about many capabilities that can be used by regulators worldwide. A good example is the regulatory mandates in California, namely Rule 21 that require DER above 1 MW to send telemetry data with utility and are further rolling out additional use cases like curtailment services, Connect /disconnect of DER, Curtailment of Active Power and Generation of Reactive Power use cases as well as settings to enable ride through the grid faults. Rule 21 envisages the generation threshold to be lowered to 250 kW once utilities gain experience rolling out 1MW+ telemetry. To improve observability and control, Utilities need to establish a bi-directional communication link with each DER from DERMS or ADMS applications to facilitate these use cases. However, coordinating with each DER becomes an enormous task for the Utilities, unless a scalable infrastructure is implemented.

The IEEE 1547-2018 and IEEE 2030.5 provides a good set of standards and architecture for DER integration. Utilities and regulators need to adopt this more actively and the industry needs to support this quickly to help manage the increased adoption of DERs and EVs IEEE 2030.5 with Smart Energy Profile is designed to communicate securely on the public network and has built-in all the use case requirements for managing DERs.

To manage the scale of DERs and EVs, the infrastructure requirement and the business models need to support Direct Connected DERs to the Utility network for large DERs and a Cloud aggregation model for DER and EV data and control with an aggregated interface to the Utility Control Center. Since the standards-based protocol adoption might take time, the Direct Connect and Aggregator model will require support for DER-specific protocols such as SunSpec, Modbus, DNP3.0, IEEE2030.5, IEC61850, OCPP, OCSP, MQTT and should have the capability to integrate with multiple makes and models of DERs on various protocols as well as with Inverter management system or charge management systems.

Accurate forecasting and scheduling of variable distributed energy generation has assumed significant importance for market operations. At the same time, the costs, of onboarding DERs into markets application must be kept low. Cloud DER aggregators are not only data aggregators, but they can also participate in the Energy Markets as well as provide telemetry and control data to the Utility. Hence regulators are working towards defining aggregators as important stakeholders in the DER and EV management ecosystem. DER aggregators coordinate, manage, facilitate, and control the distributed resource, energy accounting and settlement forecasting, and scheduling of the small-scale and individual renewable energy generators. The aggregators connect with DERs for optimizing and dispatching network/flexibility service, congestion management, and sending real-time signals it receives from the utility DERMS or the ISOs. Since aggregator is an intermediary interfacing with multiple stakeholders and applications, they can take advantage of cloud services with their scalability, security, redundancy, and resiliency in addition to the ability to be deployed in multiple availability zones to address multiple markets. The aggregator providers can also develop additional services in addition to data and control flow intermediary to be a Virtual Power Plant provider and a Demand Response provider. This would require high-speed computations, require scalable compute resources that use the latest CPU technology along with other technologies such as parallel clusters, NoSQL data lakes, and the latest machine learning algorithms which are typically offered by and constantly updated by cloud providers such as AWS.

To help transition the industry faster ASE/Kalkitech has developed a scalable software middleware for DERs and EVs, DER Data Hub on our kalki.io platform hosted on AWS to aggregate monitoring and control capabilities DERs and integrate with Utility DERMS / ADMS / SCADA, ISOs EMS systems, Power system applications and analytics applications.

Solution

ASE/Kalkitech’s Kalki.io Data Hub is a cloud-based software hosted on AWS as a Public Cloud or Private Cloud option, which can connect with DERs, EVs and Utility OT systems over a secure communication link. DER Data Hub can communicate with devices using industry protocols like DNP3, IEC 61850, IEC60870, IEEE2030.5, SunSpec, Modbus and quite a few legacy communication protocols.

Kalki.io Data Hub is a DER Data aggregation middleware which provides remote monitoring and management of DERSs and can connect securely to PV, BESS, and Wind generation and can interface with Meters, Inverters, Site Controllers, Site Energy Management Systems, EV, Inverter management system, and charge management system. DER Data Hub aggregates downstream field data and exchanges it with utility DERMS /ADMS systems over IEEE2030.5. It will accept control commands & settings from DERMS/ADMS system and pass them on to individual DERs. 

Kalki.io DER Data Hub middleware comes with a connection management service and device adaptors which support connection to multi-vendor devices that support different protocols and communication interfaces. Middleware supports 100+ protocol drivers that have been field proven in connecting with 10,000+ types of devices and software out in the field. The Kalki.io connection management layer handles connectivity to any field device regardless of the protocol and network it is connected to. Data Hub supports outbound connection to field devices as a master or client and it also supports inbound connection as a server for certain other types of DER devices and protocols. Remote device management and remote access are inbuilt into the DER middleware for the management of DERs.

Moreover, Data Hub can also exchange data with the market applications directly and to other external applications such as predictive maintenance, degradation monitoring, inventory management and generation forecasting which would help to optimize the ROI for the builders. It can utilize the entire AWS stack for System Integrators and OEMs to build software applications and platforms to enable the Digital Transformation of the Utility Industry and Mobility.

Kalki.io DER Data Hub Component: IEEE 2030.5 Server

IEEE2030.5 server is a scalable component of Kalki.io DER Data Hub to manage connectivity with DER devices in the field over WAN /public internet. IEEE2030.5 Server supports all the functions set as defined in the California Smart Inverter profile (CSIP) standard for DER monitoring and control use cases. Any DER device, gateways or inverter cloud compliant with IEEE2030.5 standard can connect with Kalki.io 2030.5 server. It can support SunSpec certificates as well as a custom trust chain as required by the system design.

Kalki.io DER Data Hub Component:  Multi-Protocol Head End / FEP

Front End Processor (FEP) is used to connect with DER devices in the field directly. It can connect with DER devices over SunSpec, DNP3.0, Modbus, IEC61850, IEC60870, and quite a few legacy communication protocols to establish two-way communication between the cloud and DER for monitoring and control use cases. FEP simplifies the data acquisition architecture significantly by converging all field data from various devices into a canonical model.

Kalki.io DER Data Hub Component: IEEE 2030.5 Aggregator Client

IEEE2030.5 aggregator client is part of the Data Hub product which can be used to connect with utility DERMS/ADMS system compliant with IEEE2030.5 standard. A 2030.5 client can be configured on three modes, fully interpreted, partially interpreted and no-interpretation for handling the DER controls. In no-interpretation mode aggregator shall relay the controls directly to the DER devices or downstream gateways and let the DER device/gateway handle the control prioritization, scheduling, primacy, and default controls. In Partial interpretation mode, the aggregator client passes the scheduling and default control downstream and handles the control primacy and prioritization on the aggregator side. In fully interpretation mode aggregator client does all the handling of DER control including scheduling primacy, and default control handing. The full interpretation would be required for devices which are connected to telemetry protocols like DNP3.0 or SunSpec directly from the field. No interpretation is adopted for when the data hub is getting connected with DER devices /gateways connected over IEEE2030.5 directly.

Kalki.io DER Data Hub Component: Application Programming Interface for VPP, DERMS, DR, EV Sub-Metering MDM service provider

DER Data Hub exposes a robust set of APIs to OEMs and System Integrators to build out various applications like Virtual Power Plants, DERMS, Demand Response, EV Sub-Metering MDM service providers and many more. The APIs expose the entire set of capabilities that Kalki.io DER Data Hub provides to application developers to build full-stack applications on AWS.

Architecture

Multiple types and makes of DERs would need to be integrated with the DER Data Hub. So, what was needed for DER data aggregation is a way to seamlessly connect with DERs through gateways or directly through WAN routers over DNP3.0, SunSpec, IEEE2030.5, OCPP, OCSP etc. Kalki.io Data Hub has IEEE 2030.5 server which can be used for DERs having native IEEE 2030.5 support to connect directly or through a gateway. Kalki.io FEP has DNP3.0, SunSpec, IEC104, WebSocket, Open ADR, OCPP, and OCSP protocols for connecting with the DER devices on the respective device-supported protocol. Data on various protocols as well as IEEE2030.5 data collected by native 2030.5 servers can be shared with utility DERMS/ADMS systems using IEEEE2030.5 Aggregator client service in Data Hub.  Users can send commands to the DER device and receive metering and telemetry data from the field. Telemetry data received from the field can be displayed on the data hub or can be pushed to the utility system over IEEE2030.5 or external applications using Kalki.io interface connectors. DER commands, schedules and settings, and messages send received by DER Data Hub from DERMS or other external systems, can be passed to the field over IEEE2030.5 or through the native protocols supported by the device.

More details on the DER Gateways and Compatible router options are available in

For DER aggregator applications like VPP hosted on AWS, data collected from devices and downstream aggregators can be directly pushed to the AWS Kinesis stream or into an Amazon S3 bucket. Events and alarm data get added to an AWS SQS queue and another SQS queue is used to send control commands to the DER devices. DER Aggregator application support logic to split commands among devices or to aggregate measurements, in DR use cases, where the DERMS want to send an aggregate curtailment command, and the DER Aggregator will determine how to split the aggregate curtailment to individual DERs.

Resiliency & Availability

High availability of the Kalki.io Data Hub is established by deploying the software distributed across the servers in two different availability zones behind an AWS Network Load Balancer (NLB). Servers can be configured on an AWS Autoscaling group to ensure that there will always be a minimum of two servers running for high resilience.  On the backend, the Postgres SQL database is used on a distributed architecture with high availability configuration. 

In addition to redundancy and load balancing clusters across multiple availability zones, it’s also possible to set up a disaster recovery system across additional availability zones in a different AWS region. To support software recovery, the Data Hub can be set up to back up its configuration details to an S3 bucket, encrypted, and using Amazon S3 object locks to make it immutable, and with versioning turned on so a prior version may also be retrieved. In a disaster recovery scenario, a customer can use the backed-up configuration to restore the Data Hub in any AWS region within minutes. AWS global accelerator is used to enable faster failover from one AWS region to another if the primary region is not available. Users can move the IP associated with AWS global accelerator to this new location and be back in operation soon. The global accelerator also helps in avoiding network configuration at the DER side to switch from one region to another. With this approach, users can establish high levels of availability across multiple geographic regions.  The user could also just choose to use a single region with just two servers deployed thus offering flexibility in deployment options to meet their specific risk tolerance. All of this can be automated using AWS CloudFormation thereby removing the risk of human error.

Security

Security for the Data Hub does not stop with a highly available architecture.  The Data Hub communicates with the DER devices and meters over an encrypted TLS1.2/TLS 1.3 link using a Secure Gateway or proxy deployed in the field.  Inbound traffic is received by an AWS Network Load Balancer that is configured to receive data on specified ports with a security group that further applies both port and source IP restrictions if required.  The Data Hub servers themselves sit in a private subnet and are inaccessible from the internet and are not even directly accessible (over SSH) for interactive access to ensure system security.  For maintenance of the servers, a customer’s engineer can access the Data Hub servers via a jump box/bastion host or using AWS Session Manager. All disks used by the Data Hub are encrypted. Kalki.io Data Hub includes state of an art Identity and Access management system to provide end-to-end security features to make the end-to-end solution a secure implementation.

Conclusion

The ASE/Kalkitech Kalki.io DER Data Hub is a highly available cloud-based DER data aggregation system that can be used for both telemetry and control applications.  Using a Kalki.io Data hub on AWS supports use cases such as Data aggregation, network isolation, easy onboarding of customer DERs, enabling advanced analytics applications, virtual power plants and applications for the retail/wholesale energy market all of which require the ability to scale compute resources and store large amounts of data.  Using AWS services, the DER Aggregator deployment follows security best practices by controlling access and encrypting all data at rest and in transit.