Snowflake announced at its Snowday 2022 event, in San Francisco, that it’s bringing Python-based app development to its data cloud via Streamlit which it acquired in March. Streamlit enables tens of thousands of data scientists and other developers to easily build data applications with Python using its open source framework.
Now Snowflake is further advancing its Streamlit integration so developers can bring their data and machine learning (ML) models to fruition as secure, interactive applications – within Snowflake.
Snowflake’s Streamlit integration will bring together Streamlit’s ease of use and flexibility, with Snowflake’s scalability, governed data coverage, and security, so developers can build powerful applications without the traditional complexity involved with building and deploying web applications. The integration will allow developers to create applications with Python using their data in Snowflake, deploy and run these applications on Snowflake’s secure and governed platform, and share their applications with business teams to further unlock the value of data and ML models.
An official video describes the moves thusly…
Sai Ravuru, GM Data Science & Analytics at JetBlue said, “Streamlit serves as the interaction engine for the vast majority of our Data Science & Machine Learning models today, actively transforming how our teams build, deploy and collaborate on powerful applications with other stakeholders across the business. With Snowflake’s Streamlit integration, we can go from data to ML-insights all within the Snowflake ecosystem, where our data is already present, making it easier and more secure for us to create impactful applications to further mitigate the negative impact of flight disruptions, provide more predictability to our operational planning teams, and more customer personalisation to give our customers the best possible experience.”
With Python being one of the most popular language for data scientists and the third most popular language among all developers, Snowflake is now making its rich ecosystem of open source libraries available to all users and teams with via the general availability of Snowpark for Python. In the months since its public preview announcement and expanded Anaconda integration at Snowflake Summit 2022, Snowpark for Python has, “Seen 6x growth in adoption, with hundreds of customers including Charter Communications, EDF, NerdWallet, Northern Trust, Sophos and more building with their data using Snowpark.”
With Snowpark as Snowflake’s developer framework, developers gain a streamlined architecture that natively supports users’ programming languages of choice including Java, Scala, SQL and now Python. Snowpark for Python is part of the wider Snowpark ecosystem, bringing teams together so that they can collaborate and build on one unified platform with a highly secure Python sandbox, providing developers with the same scalability, elasticity, security, and compliance benefits they’ve come to expect when building with Snowflake. In addition, developers can eliminate data security and compliance roadblocks that have previously prevented projects from going into production. Snowflake is also releasing Snowpark-optimised warehouses (public preview in AWS), so Python developers can run large scale ML training and other memory-intensive operations directly in Snowflake, and Python Worksheets (private preview) to develop applications, data pipelines, and ML models inside Snowflake.
Partners like Anaconda, dbt Labs, and more have been, “Instrumental in accelerating the adoption of Snowpark for Python and allowing developers to build with confidence. These advancements include Anaconda’s integration with Snowpark for Python, which make Anaconda’s open-source Python libraries seamlessly accessible to Snowflake users by eliminating the need for manual installs and package dependency management. In addition, dbt’s new Snowpark for Python support effortlessly combines the power of SQL and Python for modern analytics, enabling customers to further bridge the gap between analytics and data science teams.”
William Wu, Head of Quant Analytics, FOA, Northern Trust, said “With Snowpark for Python, we can build a fully integrated data science ecosystem that empowers financial engineers, data scientists and business analysts to work with the full wealth of their data to build and deliver custom analytics. Snowpark enables us to collaborate on the same data as the rest of the business, so we can uncover and visualise new insights and drive dynamic fact-based and data-driven investment decision-making for our customers.”
Snowflake, “Simplifies streaming pipelines and drives increased automation and observability for developers”
Snowflake is also reimagining how users build data pipelines, “Making it easier to work with streaming data within a single platform and further eliminating silos for customers. To do so, Snowflake is equipping them with the capabilities needed to eliminate complexity while leveraging core software development principles. Users can now improve productivity by onboarding data faster with Schema Inference (private preview) and execute pipelines effortlessly with Serverless Tasks (general availability) natively in Snowflake’s platform.”
Snowflake is also unveiling enhanced tools that further empower developers to build in the Data Cloud including:
- Dynamic Tables (private preview): Formerly introduced as Materialised Tables, Snowflake is removing the boundaries between streaming and batch pipelines by automating incremental processing through declarative data pipelines development for coding efficacy and ease. This also simplifies use cases including change data capture and snapshot isolation, and is native to Snowflake so it can be shared across all Snowflake accounts with full security and governance.
- Observability & Experiences: To further meet the needs of developers, Snowflake is investing in native observability and developer experience features so they can build, test, debug, deploy and monitor data pipelines with increased productivity through alerting (private preview), logging (private preview), event tracing (private preview), task graphs and history (public preview) and more.
Torsten Grabs, Snowflake’s Director of Product Management, said, “As we continue to disrupt application development, we’re giving builders the data access and tools they need to accelerate their pace of innovation securely under Snowflake’s one unified platform. Snowflake’s advancements provide developers with the capabilities to build powerful applications, pipelines, and models with the utmost confidence, and eliminate complexity so they can drive value across their organisations with the Data Cloud.”
Sathish Balakrishnan, Director of Data Engineering, NerdWallet, said, “Snowpark for Python has created new opportunities and use cases for our team to build and deploy secure and compliant data pipelines on Snowflake, so we can more efficiently provide our customers with the tools needed to handle every aspect of their finance journey. Snowflake’s continued investments in Python allow us the flexibility to code in our programming language of choice, and accelerate the speed of innovation for our end users.”
Konstantin Berlin, Head of Artificial Intelligence at Sophos, said, “Our mission is to fundamentally protect the customer as best as we can, as effectively as we can, and Snowpark for Python enables our team to build better detection models so we can do just this. Everything in our industry revolves around Python, and Snowpark enables our data scientists with simple code that’s maintainable, and trackable, so we can significantly increase our pace of innovation.”