We’re accepting applications for the position of Data Engineer. Duties will include developing scalable and efficient data pipelines according to business requirements; collaborating with functional data owners to resolve data-related queries and providing necessary support for data-driven initiatives; managing data integrity, security, quality, and accessibility; assisting with the development of reporting solutions; measuring the reliability and performance of Extract, Transform, Load or Extract, Load, Transform workflows and ensuring minimal downtime and efficient data processing; among other functions.
Duties and Responsibilities:
- Developing scalable and efficient data pipelines according to business requirements
- Collaborating with functional data owners to resolve data-related queries and providing necessary support for data-driven initiatives
- Managing data integrity, security, quality, and accessibility
- Assisting with the development of reporting solutions
- Measuring the reliability and performance of Extract, Transform, Load or Extract, Load, Transform workflows, ensuring minimal downtime and efficient data processing
- Tracking metrics including data completeness, consistency, and error rates to maintain high-quality, reliable datasets for analytics and decision-making
- Evaluating improvements in data infrastructure, including query performance, storage optimisation, and cost efficiency of data processing
Minimum Requirements:
- At least 3 to 5 years experience in data engineering
- Proficiency in SQL and Python
- Experience with data pipeline tools such as Apache Airflow
- Hands-on experience with cloud platforms such as AWS and Azure
- Strong knowledge of data warehousing solutions (Snowflake, Redshift, BigQuery)
- Familiarity with containerization tools (Docker, Kubernetes) is a plus
- Experience in developing reporting solutions using visualization tools
- Experience with real-time data processing frameworks (Kafka, Spark Streaming)
- Experience with Smartsheet is a plus
- In-depth knowledge and practical experience of data lifecycle management