EU AI Act Deadlines Approach, Latest on Anthropic and Open AI + Azure/Snowflake Architecture 🧠 #66 w/e 10 Jan 2025
Join the 4,800-strong data herd getting all you need to know about Data for your Friday roundup
Happy New Year and welcome to 2025!
In a world overflowing with data content, we know how much time you spend navigating forums, discourse channels, and yes—even LinkedIn—to stay informed.
We gather everything we learn, discover, and hear, curating it all into this newsletter just for you. From articles, videos, and podcasts to news, thought pieces, and job opportunities, we’ve got it covered.
Make sure you stay subscribed in 2025—you won’t want to miss a thing!
Announcing Orchestra architecture for Azure, ADF and Snowflake
We are excited to share our whitepaper on Azure Reference Architecture with Snowflake 🎉 Read more here or continue below.
The times of needing to rely on painful spark jobs in parquet on ADLS are over since Snowflake has such a powerful and scalable compute engine.
There are also increasingly powerful integrations between Azure infrastruture and Snowflake. This makes using select or choice SAAS tools together with Snowflake and Azure a popular option for building a data stack.
Given the need to integrate additional tools, tighter snowflake integrations and SQL modelling frameworks like dbt and Coalesce, ADF is becoming less and less powerful as an orchestrator for a scalable data platform.
Furthermore, it necessitates the implementation of separate governance, alerting and monitoring systems, which can be time-consuming and costly to build and maintain.
Ready to get started?
Winter Data Conference
Excited to share that anyone using our special code HUGO50 can get a 50% discount to the Winter Data Conference in Zell Am See - check it out here.
Meme Drop
When you ask someone on Azure how they do insights
Also a big shout-out to Mohanad Osnan and anyone else who said they “describe data engineering as like being the plumbers for data pipelines”. We had a lot of folks come up with good explanations for data engineering to non-engineers and this explanation we arbitrarily decided was the best at articulating what we do to the layment, so well done.
Medium 🧠
🧠 Configuring Private Connectivity to Azure External Storage Accounts (link)
🧠 Unlock the Power of Salesforce Data in Snowflake with Cortex Analyst (link)
🧠 Part 3: A Survey of Analytics Engineering Work at Netflix (link)
🧠 Sustainable Business Strategy with Data Analytics (link)
Editor’s Pick
🧠 The Core Tenets of Data Pipelines and how you can implement them (link)
🧠 Charting our own Course: Orchestra’s Year in Review (link)
LinkedIn🕴
🕴 Rethinking the Relational Model: Thoughts and Considerations (link)
🕴 Maximizing Marketing ROI: Campaign Metrics vs. Alliance Cultivation (link)
🕴 Looking Forward to Data Day Texas 2025: Celebrating the Data Community (link)
Editor’s Pick
🕴 Why OM1 Migrated from Databricks to Snowflake for ML Workloads (link)
🕴 How We Accidentally Built a Data Reconciliation Product with Orchestra (link)
News 📰
Lots of news…
📰 Build RAG and Agent-based AI Apps with Anthropic’s Claude 3.5 Sonnet in Snowflake Cortex AI - Today, we are excited to announce the general availability of Claude 3.5 Sonnet as the first Anthropic foundation model available in Snowflake Cortex AI. Customers can now access the most intelligent model in the Claude model family from Anthropic using familiar SQL, Python and REST API interfaces, within the Snowflake security perimeter. This native integration allows data and engineering teams to efficiently tackle some of the most common challenges, per MIT Technology Review Insights poll 2024, in AI deployment including data governance, privacy and integration. Read more.
📰 Announcing egress control for serverless and model serving workloads - We are excited to announce that egress control for Databricks serverless and Mosaic AI Model Serving workloads is available in Public Preview on AWS and Azure! You can now configure policies to centrally control outbound access from serverless workloads across multiple products and workspaces. Read more.
Strategic Developments
📰 OpenAI shifts focus to superintelligence development - Company announces dedicated research division for advancing towards artificial general intelligence. The initiative includes new safety protocols and collaboration with leading academic institutions. Read more
EU AI Act implementation deadlines approach - First phase of comprehensive AI regulation begins February 2025. Companies face strict compliance requirements for high-risk AI systems and prohibited practices, with penalties up to 7% of global revenue. Read more
Anthropic closes in on $60bn valuation - Latest funding round led by major tech investors values the AI safety-focused company at nearly $60 billion. The investment will fund expansion of Claude's enterprise capabilities and continued research into constitutional AI. Read more
📰Microsoft invests $3 billion in India's AI infrastructure - Strategic investment includes three new data centers, AI research facilities, and comprehensive training programs. The initiative aims to establish India as a global AI hub while addressing the growing demand for cloud computing in the region. Read more
📰 Microsoft plans $80 billion AI datacenter investment - Landmark investment focuses on next-generation AI infrastructure across the US. The plan includes quantum-ready facilities and commits to 100% renewable energy usage, creating an estimated 50,000 jobs. Read more
YouTube and Podcast 🎥
🎥 Wait SSO is free? How to set-up Microsoft/Azure/Entra SSO in Orchestra (link)
🎥 The Latest Data Trends and Predictions for 2025 (link)
Special 💫
💫 This incredible podcast series on data and AI innovation in the manufacturing and industrial sectors in Europe - HIGHLY RECOMMENDED (link)
Jobs 💼
💼 Senior Analytics Engineer at Grocery TV
💼 Analytics Engineer at Anduril Industries
💼 Sr. Data Engineer at Kandji
💼 Analytics Engineering Manager at Carta
💼 Senior Analytics Engineer at Delta Defense
💼 DevOps Engineer in Vienna / Austria with ScaleTech Platforms (link)
💼 Data Scientist at Claims Consortium (link)
💼 Tesco are building a next generation cyber security platform and they want YOU (analytics engineers) to be part of it (link)
Want to save on your ingestion bills? You’ll love this
You can leverage Python for lightweight ELT integrations. Here you’re only paying for compute and not being penalised by row-based pricing models. Pretty neat right? Check it out below / head to Orchestra and start today.
The best place to run dbt?
Don’t believe us? Watch the video below.