4 . Apache Hadoop Meetup 历年回顾
Community Activity
Every 3 – 6 month
@ SF Bay Area
2019年之前 2019.8 北京 2020.9 上海 2021.10 北京 2022.9 上海
…
5 .Is Hadoop Dead? Report -July 2022 - Apache Hadoop
社区讨论的热度有所下降
• * dev@hadoop.apache.org had a 100% decrease in traffic in
the past quarter (0 emails compared to 6)
• * mapreduce-issues@hadoop.apache.org had a 59%
decrease in traffic in the past quarter (87 emails compared to
208)
• * user@hadoop.apache.org had a 50% decrease in traffic in
the past quarter (26 emails compared to 51)
• * user-zh@hadoop.apache.org had a 100% increase in traffic
in the past quarter (6 emails compared to 3)
• * yarn-dev@hadoop.apache.org had a 26% decrease in
traffic in the past quarter (349 emails compared to 466)
• * yarn-issues@hadoop.apache.org had a 39% decrease in
traffic in the past quarter (751 emails compared to 1227)
社区贡献活跃度有所上升
• * 356 issues opened in JIRA, past quarter (21% increase)
• * 267 issues closed in JIRA, past quarter (20% increase)
• * 437 commits in the past quarter (-5% change)
• * 88 code contributors in the past quarter (4% increase)
• * 374 PRs opened on GitHub, past quarter (25% increase)
• * 299 PRs closed on GitHub, past quarter (13% increase)
Hadoop用户群和贡献者群体相对稳定
6 . Hadoop Community Update
• 3.2.X • New PMC • HADOOP-17124 Support LZO using aircompressor
• 3.2.3 -> 2022-03-20; - Sun Chao was added to the PMC on
• HADOOP-18055 Async Profiler endpoint for Hadoop daemons
• 3.2.4 was released on 2022- 2022-03-08
07-22
• HADOOP-17979 Interface EtagSource to allow FileStatus
subclasses to provide etags
• 3.3.X • New Committer
• 3.3.2 was released on 2022- - Gautham Banasandra was added as • MAPREDUCE-7341 Add a task-manifest output committer for
committer on 2021-11-04 Azure and GCS.
03-02
• 3.3.3 was released on 2022- - Benjamin Teke was added as
committer on 2022-03-24
05-17 • YARN-10496 [Umbrella] Support Flexible Auto Queue Creation
• 3.3.4 was released on 2022- - András Győri was added as committer in CapacityScheduler
on 2022-02-15
08-08
- Tao Li was added as committer on • YARN-8849 (DynoYARN: A simulation and testing infrastructure
2022-04-22 for YARN clusters).
• 2.10.X - Mehakmeet Singh was added as • YARN-9698 & YARN-10843 [Umbrella] Tools to help migration
committer on 2022-07-27 from Fair Scheduler to Capacity Scheduler.
• 2.10.2 was released on 2022-
05-31 • YARN-11025 Implement distributed decommissioning
Release Update Community Growing Notable Features
14 .趋势3:多云所带来的新数据孤岛问题
公有云 私有云 / 数据中心
ClickHouse Hive Spark
Data Machine HBase Redis Impala
RDS Data Data Machine
Pipeline Learning Factory DataStore Pub/Sub Cloud ML
Share Learning
Blob Analysis Cosmos Kafka Ozone Kudu
ES Kinesis Athena Bigtable FileStore Spanner
Storage Services DB
S3 Glue RedShift Data Lake AD SQL GCS Catalog BigQuery Hadoop Iceberg Hudi
15 .多云时代对数据Infra提出新需求
1 st generation 2 nd generation 2.1th generation 2.2th generation 3 rdgeneration
local cloud serverless cloud multi-cloud
data warehouse data warehouse query engine native database Analytical + AI
databricks
？？
AWS Athena
Google
Big Query
Unified view to all data
AWS
Amazon
RedShift Unified way to access all data
Unified engine for Analytic & ML
Appliance On Premise Managed Cloud Serverless Elasticity
云下 云上 多云
热门城市
