G
Senior Data Engineer (Toronto,ON - Hybrid 4 days office)
Salary undisclosed
Apply on
Original
Simplified
Job Description
Job Description
Responsibilities
- Manage timelines/deliverables within the team towards successfully delivering projects.
- Design software solutions by interacting with portfolio managers, traders, operations staff, and peers to understand requirements.
- Develop solutions that align with the client's technology biases, deliver efficiency and scalability, and enable new trading activities.
- Provide knowledge transfer to team members and support staff through application demos, walkthroughs, and documentation.
Skills Must have
- Concepts: Data Mesh, Data Product
- Basic - Data Pipelines: PySpark, Hudi, Iceberg, Airflow, EMR
- Expert - Data Operations: Operational Excellence, monitoring, troubleshooting
- Expert -Data Quality: Great Expectations, AWS Glue DQ, Deequ
- Expert - Entitlements: AWS Lake Formation
- Expert -Infrastructure: AWS Data Eng. Services (Glue, EMR, Athena, etc.), AWS Compute Services (EC2, Lambda), AWS Storage Service (S3), cross-account IAM, RAM, MWAA, Step Functions, etc.
- Knowledge -DevOps: Terraform, Git Actions
- Expert -Metadata: Neptune, OpenLineage, Angular, DataZone
- Knowledge -Consumption: AWS Athena, Trino
- Expert- Other Tools: JupyterLab, Databricks
Good to have
- SQL We are looking for experienced support whereby experience means comfortable working with production-level code and assets with clear understanding of DevOps and DataOps principles.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
Report this job Similar Jobs