Pivotal 大数据套件

1 .The Journey to a Data-Driven Enterprise

2 .Pivotal BigDataSuite 2.0 介绍 以及Open Data Platform 陈威 Email:cwei@pivotal.io Mobile: 139-1100-2375

3 .Agenda Overall BDS 2.0 详解 基于BDS的大数据平台架构 ODP & Open Source 简单案例介绍 为什么Pivotal是您值得信赖的伙伴

4 .Overall

5 .新的数据与应用时代的特征 开放 数据驱动的 应用 云与大数据 的融合

6 .避免技术锁定 依赖生态环境 共同创新 新的互联网时代的准则 开放 敏捷 云亲和 缩短创新的周期 降低 TCO 加速 Time To Market 专注解决业务问题 避免IaaS锁定 安全性

7 .World’s First Open Sourced, Enterprise-Class Data Portfolio + Open Data Platform PIVOTAL BIG DATA SUITE 开放 敏捷 云亲和 Modern Data Infrastructure + Advanced Analytics + Apps at Scale Multiple Cloud Deployment Models + Big Data Suite on Pivotal Cloud Foundry

8 .数据驱动型企业的漫漫征途 自动化部署和横向扩展分析型的 应用并零运维 通过高级深度智能分析 挖掘远见与洞察 现代化的数据基础架构

9 .数据驱动型企业的征途 STORE Structured Unstructured High Volume High Velocity ANALYZE Predictive Analytics Machine Learning Advance Data Science Realtime Analytics DEVELOP Advanced Analytic Pipelines Realtime Analytical Applications Global Scale Data-Driven Applications Enterprise, Consumer, IoT , and Mobile INNOVATE Agile Dev Expertise DevOps Hybrid Cloud Continuous Delivery Closed Loop Applications AGILE DEVELOPMENT BIG DATA PREDICTIVE ANALYTICS ENTERPRISE PAAS

10 .Big Data Suites 的设计原则 Modernize Your Data Infrastructure Leverage Advanced Analytics Deploy Data-driven Apps at Scal e AGILE DATA Flexibility Innovation Performance and Scale FOUNDATION FOR ALL PLAYS

11 .Lambda 大数据架构 1 2 3 4 5 …… …… …… ftp JDBC hdfs API … http REST Socket TCP/UDP MQ JDBC … API JDBC hdfs API JDBC hdfs API JDBC hdfs REST http …

12 .BDS 2.0

13 .Big Data Suite 2015: 更敏捷的数据全栈

14 .敏捷的数据 自动化部署与扩展 深度 智能 分析 现代化架构 Spring XD Spark Pivotal HD & Open Data Platform Pivotal Greenplum Database Pivotal HAWQ Rabbit MQ Redis Pivotal GemFire Pivotal BDS on PCF Pivotal Cloud Foundry

15 .数据驱动型企业征途-with Pivotal BDS STORE Structured Unstructured High Volume High Velocity ANALYZE Predictive Analytics Machine Learning Advance Data Science Realtime Analytics DEVELOP Advanced Analytic Pipelines Realtime Analytical Applications Global Scale Data-Driven Applications Enterprise, Consumer, IoT , and Mobile INNOVATE Agile Dev Expertise DevOps Hybrid Cloud Continuous Delivery Closed Loop Applications AGILE DEVELOPMENT BIG DATA PREDICTIVE ANALYTICS ENTERPRISE PAAS Spring XD Spark Pivotal HD & Open Data Platform Spring XD Pivotal Greenplum Database Pivotal HAWQ Spring XD Pivotal GemFire Redis Rabbit MQ Spring IO Groovy Pivotal BDS on PCF Pivotal Cloud Foundry Pivotal Labs Data Science Data Engineering

16 .真正变成一个数据驱动的企业很难 传统的数据架构 很难扩展和处理大数据挑战 目前有限的技术提供商 不能够提供全面的解决方案 全部自己用开源的 产品来搭建需要极高的技能水平

17 .数据驱动型企业灵活的存储和处理所有的数据 Deploy analytic apps at scale Apply analytic algorithms Discover insights Store any type and size of data

18 .从不灵活的架构迁移到敏捷的数据架构 Based on scaled-up, proprietary systems Siloed and scattered data sets Storage limited with rigid data models Elastic, Scale-out storage and processing Handle any data type or source Cloud friendly and open-source based

19 .Analytics-optimized, ODP–based Hadoop distribution In-memory, distributed processing from Apache Scale-out analytics pipeline management with data ingestion and processing 敏捷，开放的数据存储与处理方式 As we move to combine all the data generated by our activity and to leverage advanced analytics in real time, there’s no better way to do that than through the flexibility and choice provided by Pivotal’s Big Data Suite . – Sylvain LeBorne , EVP Data Platforms “”

20 .Deploy analytic apps at scale Store any type and size of data 数据驱动型企业战略性的运用高级分析带来的远见 Apply analytic algorithms Discover insights

21 .从传统的BI向数据科学家方向转型 Long running queries Stale dashboards Summarized data sets Massive parallel processing of queries Fresh data Machine learning and advanced analytics tools 0101010101010101001010101010101011001010101010101010

22 .高级分析能力-敏捷性与性能 Leading analytic data warehouse Most advanced analytical SQL engine on Hadoop 100X performance improvement analyzing trends among 500 million job postings. “”

23 .数据驱动型企业的基础架构需要高可扩展性 Store any type and size of data Apply analytic algorithms Discover insights Deploy analytic apps at scale

24 .从缺乏横向扩展性的分析型应用转向横向扩展型应用 Service failure or data loss at scale Long innovation cycles Poor experience at scale Resilient, scale-out messaging and processing Agile development with CF-based data services Low-latency, in-memory computing

25 .Distributed, in-memory database for high-scale NoSQL applications In-memory, data structure server for fast read and write applications Robust messaging for high-scale applications Leverage Big Data Suite data services within Pivotal Cloud Foundry applications Deploy and manage Big Data Suite with Pivotal Cloud Foundry Foundation 低延迟高可用的数据存储分析与消息驱动型应用 300% improvement in ticket-serving capacity led to 30% increase in e-ticket sales. “”

26 .当今其他大数据处理产品领域的现状-纷繁复杂且固定的模式 应用的类型 定价的模式 业界对应的产品 数据量/CPU 节点 节点 CPUs和其他约束条件 CPUs和其他约束条件 CPU ? GreenplumOracle /DB2/XXX Database Cloudera / PHDHortonworks /XXXXX Hadoop Distributed File System Impala/Stinger/XXXXX Parallel Query Engine over HDFS In-Memory Data Grid SQLFire /XXXXX In-Memory Data Grid with SQL Layer GemFire /Terracotta/XX In-Memory SQL Over HDFS 1 3 4 2 5 6 Spark/Shark/ GemfireXD /XXXX 企业需要单独采购，用量固定，无法灵活的组装整个数据架构

27 .1 Pivotal化繁为简-整合的统一定价模式 应用的类型 Greenplum DB Database Pivotal HD Hadoop Distributed File System HAWQ Parallel Query Engine over HDFS In-Memory Data Grid SQLFire In-Memory Data Grid with SQL Layer GemFire In-Memory SQL Over HDFS 定价的模式 Pivotal 对应全线产品 SKU 1 3 4 2 5 6 Unit of Measure Price GemFire XD 整合各种需求，统一定价，总量固定，简化企业的选择和流程

28 .ODP & Open Source

29 .Open Data Platform Initiative opendataplatform.org A common core for the Hadoop ecosystem Enterprise-grade software quality Rapidly accelerated certifications Strong ecosystem development