Description:
We are seeking a talented Data Engineer with a strong foundation in Big Data systems. The ideal candidate will thrive in a dynamic environment, working across data engineering, event-driven architectures, and infrastructure management within a regulated industry.
Key Responsibilities
- Big Data Systems Development: Design, implement, and optimize scalable data architectures. Build systems leveraging tools like Databricks and Apache Spark to process and analyze large datasets efficiently.
- Database Design & Management: Design and implement robust SQL and NoSQL database schemas. Ensure high performance and scalability in database interactions.
- Event-Based Systems: Develop and manage event-driven architectures using messaging queues or related frameworks.
- Infrastructure as Code (IaC): Manage cloud infrastructure with Terraform, ensuring reliability and scalability.
- Collaboration: Work closely with cross-functional teams, including software engineers and data scientists, to align data systems with business needs.
Key Requirements
- Data Engineering Expertise: Proven experience in designing and implementing big data systems, especially using Databricks
- Event-Driven Architectures: Knowledge of event-based systems and messaging queues (e.g., Kafka, RabbitMQ).
- Infrastructure Knowledge: Proficiency in Terraform for managing cloud infrastructure.
- Database Skills: Strong experience with both SQL and NoSQL databases, including schema design and query optimization.
- Data-Intensive Systems: Experience working with large datasets and solving complex data challenges.