Playing with ETL
I became familiar with Microsoft Azure when researching ETL approaches; specifically, Azure Data Factory, Logic Apps, and Databricks drew my eye. I am learning more about data engineering because I have a strong interest in it. I've completed a few tasks as I've advanced. I'd like to show you a pipeline I recently created. Project Information: This presentation focuses only on the ETL flow, where I make sure that if any entries are added to the sources, the most recent is shown. Data will be stored where it should be stored (SQL Server) after being transformed by the Azure Data Factory using sources such as Github, SharePoint, and SQL Server. This entire automated pipeline will operate at a specific time and complete all the necessary tasks on your behalf. Architecture Diagram link: https://github.com/YM-Codes/Portfolio/blob/main/assets/pdf/Architecture.pdf