Transforming Data, and Businesses, with Azure Data Factory

The ObjectSharp Podcast

In this episode, we chat with Kristie LaPlante, Senior Cloud Consultant with ObjectSharp, and Shane Castle, ObjectSharp's Cloud Practice Lead, all about Azure Data Factory and how teams can leverage its power to transform and relocate data at scale, and what benefits this provides teams whether they are working on big data warehouses or data lakes, or simply looking to move formerly on-prem data into the cloud in a fast and efficient way.

Minutes

• 0:00 - Introduction to today's show with Kristie LaPlante and Shane Castle
• 2:15 - Kristie LaPlante, Senior Consultant ObjectSharp's resident Data Factory expert, introduces herself
• 3:09 - Shane Castle, Cloud Practice Lead at ObjectSharp, introduces himself
• 4:09 - Kristie explains what Azure Data Factory is, what it does, what it can be used for
• 5:40 - Kristie and Jeff talk about how Azure Data Factory interfaces with traditional on-premise solutions
• 6:17 - Kristie talks about SSIS (SQL Integration Services), and how investments in SSIS can be leveraged with newer solutions and technologies available in Azure Data Factory
• 8:15 - Kristie and Jeff discuss why Azure Data Factory is becoming so popular today, even for companies who aren't doing big data
• 10:30 - Shane talks about the importance of Azure Data Factory being PaaS; don't have to focus on the management of things like SSIS as you did historically, and the pricing is based on active data flows, allowing you to better manage cost by only paying for when things are being done
• 12:14 - Kristie and Shane discuss the monitoring capabilities and built-in logging in Azure Data Factory
• 13:40 - Shane talks about pipeline runs, and the set of telemetry that comes with Azure Data Factory in Azure Monitor, and the in-built dashboards that make it very easy to see what's going in with your data
• 15:40 - Kristie discusses examples of how she and the teams she works with are using Azure Data Factory in the real world, such as transformations into and out of big data lakes, and moving data from Oracle on-prem to Azure SQL storage
• 20:25 - Reality check: Kristie talks about the challenges she's seen with teams who are starting their journey with Azure Data Factory, including their use of Azure Data Bricks
• 23:20 - Shane talks about his practical tips for teams using Azure Data Factory, including when to use it as the right tool for the problem you are trying to solve vs other tools, matching the right Azure services to the requirements of your project
• 24:50 - Conclusion and outro