展开查看详情
1 .Managing Smart Meter
Data with DataStax DSE
Satish Ramjee
Principal Software Engineer
First Utility
June 2017
2 . • Who are First Utility
• Creating a high availability platform
• Enabling customer control and insights through analytics
• Providing a self healing platform with event history
• Monitoring and alerting across the platform
• Conclusion
Contents
4 . ● Largest independent energy supplier
● Supply almost 1 million UK homes
● One of the first to provide Smart meters
● Provide broadband and home services as well
● Technology driven company
First Utility
5 .Creating a high availability platform
6 .● National smart meter rollout
○ Aim is to have all 53 million residential premises in
England, Wales, Scotland with Smart Meters by 2020
● Energy suppliers are required to offer smart meters to all
their customers
● Our immediate need to manage a high volume of meter
reads with high availability
● Meter access devices can potentially provide data every 5-
10 seconds from the total meter estate
Creating a high availability platform
7 .● Smart reads ideal for an event sourced type database
● Saw the need to add resilience, high volume and linear scaling capability into
our key information base
● Opportunity to manage business expectation through service layers
● Providing a meter agnostic business layer
“One of the things DataStax offered to us was the ability to really compress that learning curve. So we had
some of their key experts come in and spend a few weeks with us over a period of time and provide that
technical leadership” Bill Wilkins, CIO, First Utility.
http://diginomica.com/2017/02/02/first-utility-makes-smart-meters-even-smarter-datastax-platform/
Creating a high availability platform
8 . From Monolithic to Microservices Architecture
Application Server
Application Server
Docker Container Docker Container
application application
application application
(instance 1) 1) application
(instance application (instance 1) (instance 2)
(instance 1) 2)
(instance
store
store
multiple node
single node distributed store
rdms
Creating a high availability platform
9 . Data Stores
Customer
Usage View
Insights
Meter Type
1
Billing View Billing
spark
Meter Type Meter Data 1
2
Meter Data 2 spark Industry
Industry
View
Meter Type
n Meter Data 3
spark Data
Warehouse
Creating a high availability platform
10 . ● Multiple DCs
● Appropriate
RF
● DCs split
between
Business
Critical access
and Analytics
Creating a high availability platform
11 .Enabling Customer Control and
Insights
12 .● Provide customers with an insightful view
● Help customers better manage their usage
● My energy comparison of usage
● Allow control of read submission frequency
● In home display reflects current tariff and usage
Enabling Customer Control and Insights
13 . Data Storage
Customer
Insights
read Usage View
Meter
read Meter
Preferences
IHD preference/price Tariff
Meter Data
Enabling Customer Control and Insights
14 .Enabling Customer Control and Insights
15 .Enabling Customer Control and Insights
16 .Enabling Customer Control and Insights
17 .Providing a self healing platform
18 .● Record commands sent to the meter
● Detect failures
● Replay commands (with retry limit)
meter state Event
response
Meter
command
commands
spark
replay
reconcile
Providing a self healing platform
19 .Monitoring and alerting across the
platform
20 .● Eyes and ears
● Dashboards of platform behaviour
● Operational: is the platform working
● Business information: reporting and analytics
Monitoring and alerting across the platform
21 .Monitoring and alerting across the platform
23 .● Gone from read only system to ability control of meter (read/write)
● Self healing
● Real time monitoring and alerting
● Enduring solution can scale both processing and persistence
● Lessons learnt
○ Denormalise - many copies with different PK
○ Know your queries in advance
○ Reporting and aggregation - duplicate, spark or counters
Conclusion
热门城市
