Data Engineering Expert

Company: CGL Department: IT - Data Architechture Employment Type: Regular Full-Time Work Model: Hybrid (2 days in office) Language: English is required, French is an asset. Additional Information: This/these role(s) is/are currently vacant The Opportunity: We are a leading Canadian financial services co-operative committed to being a catalyst for a sustainable and resilient society and our team is essential to deliver on this strategy.  That’s why we prioritize our people, to ensure we provide a strong culture and development opportunities which enables our team to thrive and to live our purpose.  The best part is that you will work with people that care passionately about you, our clients and our communities.

The Data Engineering Expert is a critical role in the data organization, responsible for providing hands on Engineering Leadership by leading the design and build of data products that collect and store large datasets and leading the design and build of data platforms that underpin and accelerate the build of data products. They ensure the data product is resilient and is readily accessible across various teams in a secure manner.

They ensure that data products comply with the data access policies and rules that the governance team implements and leads the design and implementation of   robust security measures to protect sensitive information. They are also responsible for upskilling staff members in data tools and technologies and lead the implementation of new tools. They take end-to-end ownership of building, deploying and maintaining data products.

They are a seasoned professional with deep understanding of Azure cloud infrastructure, automation principles and has a security-first mindset. Data engineering expert will partner with the business, reporting specialists and with specialists from Advanced Analytics to drive the build of a product that satisfies the requirements of multiple stakeholders. They will also partner with networking, security, data governance, data engineering leaders and vendors like Microsoft and Databricks to drive the build of the data platform.

  How you will create impact:  Designing and implementing cloud architecture: Plan capacity, create infrastructure including subscriptions, tools and networking aspects such as private endpoints, subnets, (Network Security Groups) NSGs and create and maintain detailed architectural diagrams to support business requirements. Implement using infrastructure as a Code (IaaC) solutions (Terraform strongly preferred) to automate the provisioning and management of Azure resources such as subscriptions, resource groups, storage, networks and security.

Continuous Integration/Continuous Deployment (CI/CD): Build and Maintain robust CI/CD pipelines that include code quality and security scanning, vulnerability testing to automatically deploy data engineering pipelines to various environments. Security and Compliance: Integrate security principles throughout the development and deployment lifecycles, leveraging tools for vulnerability scanning, threat modeling, and compliance monitoring.

Lead the build of Data Pipelines templates: Lead a team of engineers in building robust data pipeline templates to automate the future development of data pipelines according to medallion architecture including automated reconciliation, data quality, technical metadata capture and monitoring and altering mechanism. Performance Optimization: Define and implement automated process to generate alerts on failures or performance degradation of data products with a view of taking proactive steps in resolving issues before they become evident.

Documentation and Maintenance: Define, lead and enforce a documentation system to capture the state of the existing implementation, and to ensure ongoing maintenance and updates. Process Improvement: Identify and implement engineering best practices and process improvements to enhance the efficiency and effect