Description
We are seeking for a Data Engineer to join our fast-growing Data Insights organization. As part of the Data Engineering team, you will build and maintain the infrastructure for developing and deploying data analytic products and core data transformation jobs. The team collaborates closely with data scientists and other product engineering teams. The primary goal is to enable Data Scientists to focus on creating data products through discovery and development of transformation jobs and machine-learning models, while ensuring their quick deployment to production.
Requirements
Degree in Computer Science, Information Systems, or equivalent experience Minimum of 5 years of related professional experience Advanced knowledge of data transformations, data pipelines, workflow automation, and scheduling systems Working knowledge of database systems, data warehouses, distributed file storage and compute platforms Good understanding of software engineering, distributed software design and data-centric systems architecture Familiar with using CI/CD tools and processes for software and infrastructure deployment. Experience with several of the technologies: Cloud Providers: Azure, AWS Containerization: Kubernetes, Docker Data Pipelines: Apache Airflow, AWS Step Functions Data Tools: Pyspark, Pandas, Databricks, AWS Glue, SageMaker, Athena Databases: SqlServer, Postgresql, Snowflake Languages: Python, Golang, Bash, SQL SCM: Git, Github, Gitlab CI/CD: Azure DevOps, Gitlab Benefits Above the law.