Smartsheet’s Transition to Snowflake and Databricks: The Why and Immediate Impac
1.Transition to Snowflake & Databricks Why and Immediate Impact Paivand Jalalian 4/24/19
2.Agenda 1. What is Smartsheet and why is data analytics important to us? 2. How do Snowlake and Databricks help us achieve our purpose? 3. What kind of impact do Snowflake and Databricks make?
3.What is Smartsheet? Why is Data Analytics Important?
4. The Smartsheet Platform for Work Execution Empowering organizations to plan, capture, manage, automate, and report on work at scale. $47M 59% 77K+ (1) (1) Q3 FY19 Revenue YoY Revenue Growth Domain-Based Customers (1),(4) Notes 1. As of October 31st, , 2018. Year-over-year revenue growth from Q3 FY18 and Q3 FY19. 4. We define domain-based customers as organizations with a unique email domain name such as @cisco. All other customers, which we designate as ISP customers, are typically small teams or individuals who register for our services with an email address hosted on a widely 4 used domain such as @gmail, @outlook, or @yahoo.
5. One Platform, Many Uses Project Management It & Operations Sales • Project tracking • Inventory / Assets • Sales pipeline • Resource • System migration • Customer contacts management • Issues triage • Sales training • Executive reporting • Maintenance • Sales rep activities • Gantt charts Marketing Company Product Development • Events Management • Development projects • Campaigns • Company objectives • QA scenarios • Website content • Balanced scorecard • Production process • Product launches • Employee vacations • Feature prioritization • Meeting action tracking Human Resources Finance Specialty Solutions • Candidate tracking • Contract process • Store / branch • New hire • Quarterly reviews communications onboarding • Corporate metrics • Rental property • Exit processing • Budget rollups maintenance • Corporate calendar • Construction projects • Client engagement management 5
6.Data analytics is not important. It’s imperative. Achieve our Purpose Empower everyone to improve how they work. Informed Decisions Targeted Customer Experience Internal Data Analysis Outbound Data Analysis
7.How do Snowlake and Databricks Help Us Achieve Our Purpose?
8.Data Platform Comparison Differences in key features Legacy MySQL Platform Snowflake Platform (On-Prem) (Cloud) Replication & Data Latency Easy & fast direct from app (~1 min) Pipeline to S3 + Airflow (~5min) Availability Replica, constant maintenance Distributed System Easy Scalability No - reaching limits of system Yes Elasticity No - query tuning required Yes (Minutes) Ease of Use MySQL - easy to learn ANSI SQL - easy to learn Occurence of table locks? Frequently Rare Query large tables, ex. Aggregating Slow, Killed after running for 1.5 Quick especially with adjustment of 3B row table hours cluster, ~ 20 Minutes Permissions Simple based on DB and action With views, as complex as needed Syntax Restricted to Mysql ANSI sql, Java, + Connection to Databricks for ML, python, etc
9.Databricks for machine learning, Snowflake for everything else. Data Warehouse Advanced Analytics Analytics (Non-ML)
10. Key Benefits Databricks + Snowflake together provides the unique ability to implement advanced analytics while maintaining structure and integrity of underlying data. Snowflake Databricks Platform ensures data structure and integrity Flexibility • Query speed (scaleable) + query large • Utilize different languages & packages datasets • Create UDFs & procedures (loops) • Conditional Permissions • Schedule jobs • Creation of views + copy DBs, • Easy Visualizations schema’s, tables with in seconds • Intuitive UI/UX • Un-drop tables • Share Notebooks • Departmental usage w/ monitoring • Versioning via Git • Connection to Tableau • Allows self service via “Run” permissions 10
11.Use Cases and Impact
12.Use Cases Solution Impact ● Query 100M+ rows of telemetry + Results and insights derived data in Snowflake quickly ● Pivots, aggregations & Anomaly Detection visualizations in Databricks + Easy/fast distribution of data ● Distribute Databricks dashboard to necessary parties + Increase speed to action ● Raw comment data stored in + Time savings human effort Snowflake minimized Text Analytics of Unstructured ● NLP model in Databricks Customer Comments Notebook (R) + Consistency in categorizations ● Connector for end-to-end solution + Ability to pull out patterns to derive insights
13.The combination of Snowflake & Databricks has not only allowed us to finally keep up with the growing scale of our company but get ahead.