Kyligence公开课(5)—Apache Superset设计与SQL查询

1 . Kyligence Apache Superset SQL Yongjie Zhao / Jan 23, 2019 info@kyligence.io 

2 . , 0 60 2 3 i I ba A BcG S ( • 10 ie • )2 • )2 , H Who uses Apache Superset (incubating)? @Superset h GitHub https://github.com/apache/incubator-superset Manual https://superset.incubator.apache.org/ 1 0, . , 

3 .• • • • Time filter • SQLLab • Dashboard • Chart Dashboard • Filter box Chart • SQLLab • 1 0, . , 

4 .1. connector 2. SQLAlchemy , , , • Kylin • Hive • Impala/presto • MySQL/PostgreSQL/CSV • …. 3. Druid , data connector, Elasticsearch no-SQL 4. Apache Kylin SQLAlchemy Dialect SQLAlchemy https://www.sqlalchemy.org/ Kylinpy https://pypi.org/project/kylinpy/ 1 0, . , 

5 .SQLAlchemy Dialect for apache Kylin $ pip install kylinpy • Query Apache Kylin • Client toolkits 1 0, . , 

6 . 2# Kyligence Insight C #1 Kylin 2 Cube • • • • • • K • Calcite 3# Kylin /I / / 8 1 0, . , 

7 .Kylin Query Query on Table Query on Cube 1 0, . , 

8 .b Sm b u pQ p b f x oa -0. .5 . cF • b , • b L ALe rs • V • h • 5 .Ci • t Ci 1 0, . , 

9 .1. 2. 3. 4. 1 0, . , 

10 . , (on-the-fly), Superset , • • (AdHoc metrics) • (AdHoc filters) • UDF 1 0, . , 

11 .1 0, . , 

12 .1 0, . , 

13 .Having 1 0, . , 

14 .Top 10 1 0, . , 

15 .1 0, . , 

16 . CSV 1 0, . , 

17 .• • SQLLab 2 8 • • • SQL 1 0, . , 

18 . Chart Dashboard , FilterBox , , • • Dashboard filterbox • Dashboard • Dashboard • • CSS • / markdown 1 0, . , 

19 .1 0, . , 

20 . Superset Superset 1. c aK 2 Superset aK d 2. 2Superset 30 2 -filter I 3. 8 2Superset I 2 I Apache Kylin 2 C d 1 0, . , 

21 .Technique Python Backend Stack • Flask App Builder for CRUD, auth, role • Pandas for analytics • SQLAlchemy for SQL toolkit Javascript Frontend • React, Redux, Bootstrap, yarn, webpack • D3 • DataTables • Nvd3 (charts libary ) • Deck.gl (WebGL data visualization) • Mapbox GL(WebGL to render interactive maps) 1 0, . , 

22 .Superset Data Flow • Table • Pivot Table • Bar Chart • Pie Chart • Bubble Chart Datasource Model SQLAlchemy • Time Series Layer • Tree Map Visualization • Heat Map Pandas Layer • World Map • Country Map Flask App builder Web Layer • Deck.gl • Mapbox • …. Backend Frontend 1 0, . , 

23 .Development environment a) Clone GitHub repo git clone https://github.com/apache/incubator-superset.git b) Install python dependence cd incubator-superset pip install -e . pip install -r requirements-dev.txt c) Database initialization superset db upgrade superset init d) flask run –p 8088 1 0, . , 

24 .Query 1 0, . , 

25 .1. core.superset.explore_json form_data = self.get_form_data()[0] core.py2Iexplore_json Superset C 8, datasource_id, datasource_type = self.datasource_info( datasource_id, datasource_type, form_data) 1. , ( ) return self.generate_json( 2. form_data datasource_type=datasource_type, 3. datasource_id=datasource_id, form_data=form_data, 4. K csv=csv, query=query, results=results, force=force, samples=samples, ) 1 0, . , 

26 .Superset code review Coding 1 0, . , 

27 .THANK YOU 1 0, . ,