Data Platform Engineer (Pyspark, 40K)

In this role, you will play a key part in developing and enhancing a state-of-the-art AI platform and data ecosystem, supporting a strategic digital transformation initiative that spans multiple years.

• Handle routine and ad-hoc data requests from various business units; design and implement end-to-end data pipelines encompassing data collection, transformation (ETL), analysis, reporting, and visualization, including dashboard creation and data modeling
• Support, maintain, and enhance data platform applications utilizing technologies such as Python, Bash, PowerShell, SQL, Excel VBA, and others
• Troubleshoot data issues, enforce best practices in data governance and management
• Assist business users with the daily operation and utilization of data warehouse, data lake, or lakehouse environments
• Participate in special projects and ad-hoc tasks to meet evolving business needs

Requirements:

• Degree in Computer Science, Software Engineering, or a related discipline.
• 2-4 years’ experience in data engineering using Airflow and Pyspark.
• Proven expertise in designing and managing large-scale data architectures like data lakes or data warehouses is a PLUS.
• Experience in containerization technologies like Docker and Kubernetes is a PLUS.
• Excellent problem-solving and collaboration skills, with the ability to work effectively across cross-functional teams including data scientists, engineers, and business stakeholders.
• Fluent communication skills in Cantonese and English.