We're looking for a skilled Data Engineer with strong Databricks administration experience to help build, manage, and improve cloud-based data solutions.
The project involves maintaining Databricks environments, developing scalable data pipelines, integrating data platforms, and improving overall data processing performance and reliability.
We need someone who is comfortable owning technical deliverables from start to finish, making sound engineering decisions, and solving problems independently.
What You'll Be Working On
- Managing and optimizing Databricks workspaces, configurations, permissions, and performance.
- Developing and maintaining reliable ETL/ELT workflows and data processing pipelines.
- Working with Snowflake to support data integration, transformation, and storage.
- Designing scalable data architectures for efficient processing and analytics.
- Handling data migrations between platforms while maintaining data integrity.
- Identifying and resolving pipeline failures, performance issues, and data quality problems.
- Improving automation, monitoring, and operational efficiency.
- Testing solutions, documenting implementations, and delivering production-ready work.
- Strong hands-on Databricks administration experience
- Python and advanced SQL
- Apache Spark / PySpark
- Snowflake
- Data engineering and ETL/ELT development
- Data migration and pipeline optimization
- Experience with AWS, Azure, or GCP
- Troubleshooting and performance tuning
- Apache Airflow or Kafka
- Terraform and CI/CD
- Docker or Kubernetes
- Scala
- Data governance and access control
- Cloud infrastructure automation
- Configured and optimized Databricks environments
- Functional and tested data pipelines
- Reliable integrations with required data platforms
- Completed data migration tasks with validation
- Resolved performance and reliability issues
- Documented architecture, configurations, and workflows
- Source code, test results, and deployment instructions
- Final technical handover
- You should have at least 4 years of relevant experience and a strong background in production data engineering.
- We're especially interested in developers who have worked with complex data environments, can troubleshoot independently, and understand how to build maintainable, high-performance solutions.
- You should be comfortable working remotely, communicating progress clearly, and delivering agreed milestones without constant supervision.