Remote, Contract position
Minimum Requirements:
BSc Engineering/ Computer Science/ relevant IT qualification
4+ years experience in a Data domain role (Data engineering) / Data modelling experience in relevant environment
Data warehouse technical experience definition /implementation/ integration.
Strong programming skills in Python and DBA skills (SQL/PSQL/DynamoDB or other).
Experience with data pipeline and ETL tools and reporting/analytics tools including , but not limited to , any of the following combinations (1) SSIS and SSRS, (2) ETL Frameworks, (3) Data conformance, (4) Caching, (5) Spark (6) AWS data builds.
Experience with data modelling, data governance, and data quality.
Strong problem-solving skills and ability to work in a fast-paced environment.
Strong communication skills and ability to work in a team.
Expertise in Machine Learning (ML) and deep learning frameworks.
Explaining the thinking behind simple ML algorithms.
Proficiency in all aspects of model architecture, data pipeline interaction, and metrics interpretation.
Ideal:
Experience with Big Data technologies such as Hadoop and Spark.
Experience with containerization technologies such as Docker and Kubernetes.
Responsibilities:
Develop and implement portfolio Data modelling, assurance and utilisation strategies and frameworks that align with enterprise approved governance, data and technology strategy and the Data COE.
Lead the implementation of these strategies within the portfolio.
Responsible for reporting activities at a portfolio level that provides insights into the portfolio data assurance landscape, strategy and roadmap and key metrics and indicators.
Serve as though leader and guide in the data domain by sharing knowledge identifying problems, patterns, trends, and support the development of relevant BI and MI solutions.
Design and implement scalable and robust processes for ingesting and transforming complex datasets.
Contribute to the development of architectural frameworks, apply architecture principles, and drive the development of data architecture models within the organisation.
Design and develop data models using dimensional modelling and data vault techniques and ensure stated business requirements are met by these models.
Focused on data stewardship and curation, the data engineer enables the data scientist to run their models and analyses to achieve the desired business outcomes.
Architect, train, validate and test advanced analytics / machine learning models, using enterprise-grade software engineering practices.
Collaborate with data scientists and analysts to understand data requirements and ensure that data models and prompt engines function as expected and data is accessible and usable.
Design, develops and maintain automated scalable data pipelines that improve estate performance, stability and auditability.
These include data pipelines for ETL processing.
Monitor and troubleshoot data pipeline issues.
Define, implement and integrate with enterprise data lake and data warehouse solutions (cloud and on-premises).
Ingest large, complex data sets that meet functional and non-functional requirements.
Enable the business to solve the problem of working with large volumes of data in diverse formats, and in doing so, enable innovative solutions.
Engineer data in the appropriate formats for downstream customers, risk and product analytics or enterprise applications.
Proficiency in managing test data, ensuring data integrity, and maintaining data privacy and security standards.
Provide technical leadership and mentorship to junior, intermediate, and senior data specialists.
Leadership and Team Management
Strong leadership skills to guide and mentor squads, setting clear goals and expectations for team members.
Competence in managing and coordinating efforts, including resource allocation, workload distribution, and task prioritisation.
Ability to foster a collaborative and innovative team culture that promotes excellence in data engineering practices.
Stakeholder Communication
Manage stakeholder communication, providing regular updates on data activities, milestones, and risks.
Excellent communication and presentation skills for effectively conveying data status, data-driven insights, and recommendations to stakeholders at all levels.
Ability to collaborate with cross-functional teams and provide visibility into data/modelling-related matters.
Ethical and Compliance Awareness
Understanding of ethical considerations in data engineering, including data privacy, security, and confidentiality.
Awareness of industry-specific compliance standards, regulations, and best practices, and the ability to ensure adherence in data engineering processes.
Skill in conducting ethical and compliant testing and data assurance activities.
Continuous Learning and Adaptability
Commitment to staying updated with emerging data engineering trends, technologies, and industry developments.
Willingness to pursue certifications, training, and continuous learning opportunities to enhance and adapt data engineering skills.
Ability to quickly adapt to evolving project requirements and data paradigms.
Originally posted on Himalayas