Big Data Engineer
Salary HKD40,000 - HKD60,000 per month + 15 ALs
Location Hong Kong IslandCONTRACTOR
Consultant Vivianyw Tsang
Date posted 14 May 20192019-05-14 2019-06-13 financial-services Hong Kong HK HKD 40000 60000 60000 MONTH Robert Walters https://www.robertwalters.com.hk
A large financial institution is now hiring a data engineer to join them. Successful candidates should have extensive experience in software development, experience in engineering (commercial or open source) software platforms and large-scale data infrastructures (preferably on cloud) is a preferred.
- Building distributed and highly parallelised Big Data ingestion and processing pipeline which is able to process massive amounts of data (both structured and unstructured data) in near real-time
- Evaluate and deploy data quality framework to measure and improve data quality in the organisation over time
- Engage with data stewards, producers and consumers to design appropriate data pipelines
- Automate processes where possible and are repeatable and reliable
- Work closely with Site Reliability Engineers and Product Managers to deliver software in a continuous delivery environment
- Extensive experience (5+ years) in data engineering and be a complex problem solver.
- Experience in Apache Spark and Nifi
- Experience in in-memory data processing
- Knowledge in design and implementation of data infrastructure on AWS or GCP
- Excellent communication skills and experience working in cross functional teams