Real world tales of repair - Apache BigData

Distributed databases inevitably have to deal with entropy. Within Apache Cassandra, the Anti-Entropy process initiated via CLI tools is the way of ensuring consistency of data on disk. Over the many years of the Apache Cassandra project it has also been the biggest operator pain points. Without a solid repair process in place, you had no guarantee that deleted data will not come back to life, or that data is fully distributed to replicas when nodes fail. In this talk Alexander Dejanovski, Consultant at The Last Pickle, will explain how Anti-Entropy works and why it should be run on your cluster. He will discuss the different types of repair and their effect on data consistency. He will also introduce tools such as Cassandra Reaper and the range repair script to manage scheduling and running repairs in the most efficient way.
展开查看详情

1.Real world tales of repair

2. APACHE BIGDATA - MAY 2017 Alexander Dejanovski @alexanderdeja Consultant www.thelastpickle.com Datastax MVP for Apache Cassandra Licensed under a Creative Commons Attribution-NonCommercial 3.0 New Zealand License

3.About The Last Pickle
 We help people deliver and improve Apache Cassandra based solutions. With staff in 5 countries : New Zealand, Australia, France, Spain, USA

4. What and why ? Full repair Incremental repair How to make it work www.thelastpickle.com

5.What is repair ? A maintenance operation that restores strong consistency throughout the cluster www.thelastpickle.com

6.Why do we need repair ?
 - Eventual consistency - Downtime / failure recovery - Safe deletes www.thelastpickle.com

7.Tombstones need repair too
 
 Missing tombstones can lead to zombie data (repair within gc_grace_seconds) www.thelastpickle.com

8.Tombstones need repair too
 
 www.thelastpickle.com

9.Tombstones need repair too
 
 www.thelastpickle.com

10.Tombstones need repair too
 
 www.thelastpickle.com

11.Tombstones need repair too
 
 www.thelastpickle.com

12.Tombstones need repair too
 
 www.thelastpickle.com

13.Tombstones need repair too
 
 www.thelastpickle.com

14. What and why ? Full repair Incremental repair How to make it work www.thelastpickle.com

15.How does anti-entropy repair works ? Reads all data www.thelastpickle.com

16.How does anti-entropy repair works ? Reads all data Calculates hashes www.thelastpickle.com

17.How does anti-entropy repair works ? Reads all data Calculates hashes Compares hashes www.thelastpickle.com

18.How does anti-entropy repair works ? Reads all data Calculates hashes Compares hashes Streams mismatching partitions www.thelastpickle.com

19.How does anti-entropy repair works ? www.thelastpickle.com

20.Merkle tree is requested to all replicas www.thelastpickle.com

21.Validation compaction www.thelastpickle.com

22.Merkle tree comparison www.thelastpickle.com

23.Streaming www.thelastpickle.com

24.How do we run repair ? 
 nodetool repair www.thelastpickle.com

25.Improving repair www.thelastpickle.com

26.Improving repair www.thelastpickle.com

27.Improving repair www.thelastpickle.com

28.Improving repair 
 repairing each range once is enough www.thelastpickle.com

29.Improving repair 
 nodetool repair -pr www.thelastpickle.com

user picture
ccone
+关注
我就是我！
已将链接复制至剪贴板
相关文档