Description:
We’re looking for an experienced Senior Data Engineer to provide technical leadership for how we ingest, store, process and secure data across Distilled. In this role, you’ll collaborate with all engineering teams to improve the quality, reliability and compliance of our data and to set architectural patterns for data across our organisation.
What you will be doing
- Partnering with the engineering, product & commercial teams to build and evolve data models and data flows to accelerate data-driven decision-making at Distilled
- Designing the pipeline and ingestion systems coupled with alerting and testing processes to ensure the accuracy and timeliness of our data
- Designing, building, and maintaining highly available data infrastructure systems for data sharing & consumption at all levels of the organisation
- Designing and implementing architectural patterns for getting data into the data lake or data warehouse and processed data out to internal and external customers
- Getting hands on with day-to-day data challenges and delivering real impact for customers and colleagues by improving the architecture of our data systems
We’re looking for someone who has:
- Proficiency in one programming language such as Java or Python (Most of our production systems are Java based, but we have a number of data related Python based services too)
- Experience building/operating highly available, distributed systems for the processing of large data sets
- Experience with relational and non-relational databases - technology such as Redshift, Postgres, MySQL, MongoDB, DynamoDB
- Experience delivering large-scale data warehousing and analytics projects, including using AWS technologies – Redshift, Athena, S3, Kinesis, Glue, Data-pipeline, GA/GTM
- Experience providing technical leadership and mentoring other engineers in best practices in the data engineering space
- Experience with data compliance issues such as GDPR or PCI