Coursera
2 weeks to complete at 10 hours a week
This course is designed to equip data engineers with the skills to build scalable and efficient data pipelines using Scala and Spark, with a focus on optimizing performance and ensuring data quality.
This course is ideal for data engineers with some experience in data processing. While it assumes familiarity with data engineering concepts and cloud technologies, anyone eager to improve their skills in Scala and Spark will benefit from the practical, step-by-step approach.
The course covers a range of topics, including setting up a development environment for building data pipelines in Scala, using Spark DataFrames, Datasets, and SQL with Scala for data processing, profiling and cleaning data using Deequ for improved data quality, implementing test-driven development and code maintainability best practices, automating workflows with CI/CD and GitHub, optimizing data processing performance, and building both batch and streaming data pipelines using Spark and Scala.
Coursera is a leading online learning platform that partners with top universities and organizations to offer high-quality courses and programs. This course is developed and taught by experienced instructors from Packt, a global technical learning content provider with over 20 years of experience delivering premium content from industry experts.
By completing this course, data engineers will be equipped with the skills to build scalable and efficient data pipelines using Scala and Spark, optimizing performance and ensuring data quality. This will enable them to take on more advanced data engineering roles, working with large-scale data processing and real-time data pipelines in cloud environments.
Course fee
Start date
You will be redirected to the training provider to enquire directly. Workinitiatives is not the course or training provider. We aggregate and list courses only.