本次公开课邀请了 Kyligence 公司的 Python 全栈工程师赵勇杰为我们讲解Apache Superset。课程通过讲解 Superset 代码,使您可以快速了解 Superset 的系统设计,其中包含:Superset 系统架构、Superset 代码结构讲解、可视化查询到 SQL 生成过程。
注脚
1. Kyligence Apache Superset SQL Yongjie Zhao / Jan 23, 2019 info@kyligence.io
2. , 0 60 2 3 i I ba A BcG S ( • 10 ie • )2 • )2 , H Who uses Apache Superset (incubating)? @Superset h GitHub https://github.com/apache/incubator-superset Manual https://superset.incubator.apache.org/ 1 0, . ,
3.• • • • Time filter • SQLLab • Dashboard • Chart Dashboard • Filter box Chart • SQLLab • 1 0, . ,
4.1. connector 2. SQLAlchemy , , , • Kylin • Hive • Impala/presto • MySQL/PostgreSQL/CSV • …. 3. Druid , data connector, Elasticsearch no-SQL 4. Apache Kylin SQLAlchemy Dialect SQLAlchemy https://www.sqlalchemy.org/ Kylinpy https://pypi.org/project/kylinpy/ 1 0, . ,
5.SQLAlchemy Dialect for apache Kylin $ pip install kylinpy • Query Apache Kylin • Client toolkits 1 0, . ,
6. 2# Kyligence Insight C #1 Kylin 2 Cube • • • • • • K • Calcite 3# Kylin /I / / 8 1 0, . ,
7.Kylin Query Query on Table Query on Cube 1 0, . ,
8.b Sm b u pQ p b f x oa -0. .5 . cF • b , • b L ALe rs • V • h • 5 .Ci • t Ci 1 0, . ,
9.1. 2. 3. 4. 1 0, . ,
10. , (on-the-fly), Superset , • • (AdHoc metrics) • (AdHoc filters) • UDF 1 0, . ,
11.1 0, . ,
12.1 0, . ,
13.Having 1 0, . ,
14.Top 10 1 0, . ,
15.1 0, . ,
16. CSV 1 0, . ,
17.• • SQLLab 2 8 • • • SQL 1 0, . ,
18. Chart Dashboard , FilterBox , , • • Dashboard filterbox • Dashboard • Dashboard • • CSS • / markdown 1 0, . ,
19.1 0, . ,
20. Superset Superset 1. c aK 2 Superset aK d 2. 2Superset 30 2 -filter I 3. 8 2Superset I 2 I Apache Kylin 2 C d 1 0, . ,
21.Technique Python Backend Stack • Flask App Builder for CRUD, auth, role • Pandas for analytics • SQLAlchemy for SQL toolkit Javascript Frontend • React, Redux, Bootstrap, yarn, webpack • D3 • DataTables • Nvd3 (charts libary ) • Deck.gl (WebGL data visualization) • Mapbox GL(WebGL to render interactive maps) 1 0, . ,
22.Superset Data Flow • Table • Pivot Table • Bar Chart • Pie Chart • Bubble Chart Datasource Model SQLAlchemy • Time Series Layer • Tree Map Visualization • Heat Map Pandas Layer • World Map • Country Map Flask App builder Web Layer • Deck.gl • Mapbox • …. Backend Frontend 1 0, . ,
23.Development environment a) Clone GitHub repo git clone https://github.com/apache/incubator-superset.git b) Install python dependence cd incubator-superset pip install -e . pip install -r requirements-dev.txt c) Database initialization superset db upgrade superset init d) flask run –p 8088 1 0, . ,
24.Query 1 0, . ,
25.1. core.superset.explore_json form_data = self.get_form_data()[0] core.py2Iexplore_json Superset C 8, datasource_id, datasource_type = self.datasource_info( datasource_id, datasource_type, form_data) 1. , ( ) return self.generate_json( 2. form_data datasource_type=datasource_type, 3. datasource_id=datasource_id, form_data=form_data, 4. K csv=csv, query=query, results=results, force=force, samples=samples, ) 1 0, . ,
26.Superset code review Coding 1 0, . ,
27.THANK YOU 1 0, . ,