Скачать с ютуб From ETL to Automation: Leveraging DBT, BigQuery, CI/CD and Airflow - Anima Acharya|GDG Sydney Jun25 в хорошем качестве

From ETL to Automation: Leveraging DBT, BigQuery, CI/CD and Airflow - Anima Acharya|GDG Sydney Jun25

In today’s data-driven world, building efficient, scalable, and automated data pipelines is essential for delivering timely insights. This session will guide you through creating a modern data pipeline using dbt for transformations, BigQuery as the data warehouse, and Apache Airflow for orchestration. We’ll also demonstrate how to integrate CI/CD processes to ensure seamless testing and deployment of your pipeline. By the end of this session, you’ll learn how to: Build and automate a reliable data pipeline.Integrate dbt models with CI/CD workflows for robust testing and deployment. Use Apache Airflow to orchestrate complex data workflows efficiently.Whether you're a beginner or an experienced data professional, This talk will provide actionable insights and tools to take your data engineering skills to the next level. --- About the speaker - Anima She is a passionate advocate for women in tech and a Women Techmakers Ambassador. In her current role, she focuses on developing efficient data solutions that support business analysis and reporting. Anima is an enthusiastic public speaker and enjoys sharing her knowledge on data engineering, as demonstrated in her talk today on "From ETL to Automation: Leveraging DBT, BigQuery, CI/CD and Airflow".