展开查看详情
1 .堵俊平
-开放原子开源基金会TOC主席
-Apache Member
-Apache Hadoop PMC
2 .Big Data Epic - A brief history of “Hadoop” Ecosystem
Hadoop is found YARN is found， Apache Hudi and
under Lucene AWS provide EMR Hadoop goes to Apache Spark is found, Iceberg get donated,
– first Hadoop 2.0 Big Data goes to new new Data Lake era
cloud service era
2006 2012 2014
2009 2019
2008 2011 2017 2020
2003 2013
Apache Hive is Ozone get
Hadoop graduated as Apache Spark is Hadoop 3.0
GFS, found, independent from
top-level –project found, Big Data Cloud native, AI…
MapReduce(2004), Big Data + SQL Hadoop project
HBase is found, Big Data goes to new era
BigTable(2006)
+ NoSQL
3 .A quick update from Hadoop Community – 2020.Q3 – 2021.Q3
● The Ozone sub project completed split from the Hadoop project. The transition went well.
● we declared the EOL of the 3.1 release line. Currently, we maintain only three release lines: 3.3, 3.2
and 2.10.
● Two releases go live: Hadoop 3.3.1 (2021-06-13), Hadoop 3.2.3 (2021-01-09) , 3.3.2 is ongoing.
8 new committers and 4 PMC members in this project
1500 issues opened in JIRA
New PMC: Xiaoqiao He, Szilard Nemeth, Eric Badger, 1189 issues closed
Takanobu Asanuma
2491 commits merged
New Committer: Bilwa S T, Fengnan Li, Gergely Pollák,
Qi Zhu, Xiaoqiao He, Ahmed Hussein, Lisheng Sun,
Jinglun, Mukund Thakur
5 .• So far, we haven't seen any widely used and dependent projects go into
Attic state.
Apache Big-data (50)
• Apache Accumulo • Apache Helix in the Attic(12)：
• Apache Airavata • Apache Hop (Incubating)
• Apache Ambari • Apache Ignite Apache Apex
• Apache Avro • Apache Kibble Apache Crunch
• Apache Beam • Apache Knox Apache DirectMemory
• Apache Bigtop • Apache Kudu Apache Falcon
• Apache BookKeeper • Apache MXNet (Incubating) Apache Hama
• Apache Calcite • Apache OODT Apache Lens
• Apache Camel • Apache Oozie Apache MetaModel
• Apache CarbonData • Apache ORC Apache PredictionIO
• Apache CouchDB • Apache Parquet Apache Sqoop
• Apache Daffodil • Apache Phoenix Apache Tajo
• Apache DataFu (Incubating) • Apache REEF Apache Trafodion
• Apache Drill • Apache Samza Apache VXQuery
• Apache Flink • Apache Spark
• Apache Flume • Apache Storm
• Apache Fluo • Apache Tez
• Apache Fluo Recipes • Apache Zeppelin
• Apache Fluo YARN
• Apache Giraph
6 .• Apache Hadoop is the cornerstone of all Apache Big Data projects
—Arun C Murthy(The founder of Hadoop yarn): ” Hadoop Is Dead, Long Live Hadoop”
Big Data/Long Running Services With Hadoop 3
BATCH DEEP LEARNING
SERVICES
WORKLOADS APPS
Hadoop
Ozone
HIVE on LLAP
PUBLIC CLOUD
STORAGE
COMPUTE (on-prem/on-cloud) STORAGE
7 .Ecosystem Value of Open Source Software – Architecture
• Software Development Exercise
Field
Infrastructure • Demonstrative capabilities and
Open Source Developers career development
• Avoid vendor lock-in Software community
• Facilitate open source-
based innovation
Free Support Services
Foundation
Legal support
Sponsored Technical service
Customer …
Hiring Developers,
Maintaining Community
Free Services
• Use open source to quickly
Sponsored
seize the market and attract Paid
developers to quickly develop Products/Services
the market.
• Charge for support services
required by enterprise Open Source
customers or monetization Software Vender Investor
through other business models Investment
8 .Ecosystem Value of Open Source Software – 4 dimensions
Open source community operation concept:
Technology
Ecology A series of operation activities, including content operation,
Application/Middlewa
re/Hardware Driver... activity operation, and community operation, are
conducted for users, contributors, technical ecosystems,
and partners based on the community.
Integration and authentication
Operational Objectives:
Build a popular community that people are willing to use,
User ecosystem contribute, and actively promote based on open source
Contribur contribute open Use
IT/Telecommunicat
Ecosystem source ions/E- projects. Cultivate and develop the user ecosystem,
Enterprise/Individual/
Cortion Developer... Project commerce/Scientifi
c research/... contributor ecosystem, technology ecosystem, and partner
ecosystem through the community.
Open Source Community
Operation object:
co-construction
User ecosystem
Contribur Ecosystem
Partner Technology Ecology
ecosystem Partner ecosystem
Enterprise/Equipment
vendor/ISV/...
9 .Ecosystem Value of Open Source Software - community and developers
How does the open source community organize developers
around the world?
1. Stable core developer team fully invests in the project and
promotes open development of the project (competitiveness).
2. Open and transparent R&D process, no threshold for
participation, open meetings, decision-making, and record
(participation mechanism)
3. The up-channel from user to contributor to committer (game
space for commercial firms)
10 .Will the Apache Big-data open-source ecosystem spawned by Apache Hadoop
continue to grow?
May be *Next* in the Attic New Star Project
• Apache Storm • Apache Ozone
• Apache Pig • Apache Submarine
• Apache Flume • Apache YuniKorn
• Apache Ambari • Apache Ratis
• Apache Iceberg/Hudi
11 .Actually, Hadoop’s tomorrow depends on …
热门城市
