Kyligence公开课(5)—Apache Superset设计与SQL查询

本次公开课邀请了 Kyligence 公司的 Python 全栈工程师赵勇杰为我们讲解Apache Superset。课程通过讲解 Superset 代码,使您可以快速了解 Superset 的系统设计,其中包含:Superset 系统架构、Superset 代码结构讲解、可视化查询到 SQL 生成过程。
展开查看详情

1. Kyligence Apache Superset SQL Yongjie Zhao / Jan 23, 2019 info@kyligence.io

2. , 0 60 2 3 i I ba A BcG S ( • 10 ie • )2 • )2 , H Who uses Apache Superset (incubating)? @Superset h GitHub https://github.com/apache/incubator-superset Manual https://superset.incubator.apache.org/ 1 0, . ,

3.• • • • Time filter • SQLLab • Dashboard • Chart Dashboard • Filter box Chart • SQLLab • 1 0, . ,

4.1. connector 2. SQLAlchemy , , , • Kylin • Hive • Impala/presto • MySQL/PostgreSQL/CSV • …. 3. Druid , data connector, Elasticsearch no-SQL 4. Apache Kylin SQLAlchemy Dialect SQLAlchemy https://www.sqlalchemy.org/ Kylinpy https://pypi.org/project/kylinpy/ 1 0, . ,

5.SQLAlchemy Dialect for apache Kylin $ pip install kylinpy • Query Apache Kylin • Client toolkits 1 0, . ,

6. 2# Kyligence Insight C #1 Kylin 2 Cube • • • • • • K • Calcite 3# Kylin /I / / 8 1 0, . ,

7.Kylin Query Query on Table Query on Cube 1 0, . ,

8.b Sm b u pQ p b f x oa -0. .5 . cF • b , • b L ALe rs • V • h • 5 .Ci • t Ci 1 0, . ,

9.1. 2. 3. 4. 1 0, . ,

10. , (on-the-fly), Superset , • • (AdHoc metrics) • (AdHoc filters) • UDF 1 0, . ,

11.1 0, . ,

12.1 0, . ,

13.Having 1 0, . ,

14.Top 10 1 0, . ,

15.1 0, . ,

16. CSV 1 0, . ,

17.• • SQLLab 2 8 • • • SQL 1 0, . ,

18. Chart Dashboard , FilterBox , , • • Dashboard filterbox • Dashboard • Dashboard • • CSS • / markdown 1 0, . ,

19.1 0, . ,

20. Superset Superset 1. c aK 2 Superset aK d 2. 2Superset 30 2 -filter I 3. 8 2Superset I 2 I Apache Kylin 2 C d 1 0, . ,

21.Technique Python Backend Stack • Flask App Builder for CRUD, auth, role • Pandas for analytics • SQLAlchemy for SQL toolkit Javascript Frontend • React, Redux, Bootstrap, yarn, webpack • D3 • DataTables • Nvd3 (charts libary ) • Deck.gl (WebGL data visualization) • Mapbox GL(WebGL to render interactive maps) 1 0, . ,

22.Superset Data Flow • Table • Pivot Table • Bar Chart • Pie Chart • Bubble Chart Datasource Model SQLAlchemy • Time Series Layer • Tree Map Visualization • Heat Map Pandas Layer • World Map • Country Map Flask App builder Web Layer • Deck.gl • Mapbox • …. Backend Frontend 1 0, . ,

23.Development environment a) Clone GitHub repo git clone https://github.com/apache/incubator-superset.git b) Install python dependence cd incubator-superset pip install -e . pip install -r requirements-dev.txt c) Database initialization superset db upgrade superset init d) flask run –p 8088 1 0, . ,

24.Query 1 0, . ,

25.1. core.superset.explore_json form_data = self.get_form_data()[0] core.py2Iexplore_json Superset C 8, datasource_id, datasource_type = self.datasource_info( datasource_id, datasource_type, form_data) 1. , ( ) return self.generate_json( 2. form_data datasource_type=datasource_type, 3. datasource_id=datasource_id, form_data=form_data, 4. K csv=csv, query=query, results=results, force=force, samples=samples, ) 1 0, . ,

26.Superset code review Coding 1 0, . ,

27.THANK YOU 1 0, . ,

Kyligence (上海跬智信息技术有限公司)由首个来自中国的 Apache 软件基金会顶级开源项目 Apache Kylin 核心团队组建,是专注于大数据分析领域的数据科技公司,通过前沿数据技术的分析认知来加速用户关键商业决策是其使命。
关注他