DBT PySpark Developer
- ON, Canada
- Ontario, Canada, Canada
Full-time
Applications open in a new tab.
Job details
- Job type
- Full-time
- Apply by
- Posting language
- English
- Posted
Original job description
DBT PySpark Developer
Mississauga, Ontario | Calgary, Alberta
Required Qualifications:
- At least 7 years of Information Technology experience
- At least 7 years of experience in AWS Python technologies.
- Strong expertise in:
- DBT , Python
- AWS
- PySpark
- Hands-on experience with:
- Data Ingestion pipelines
- Shell scripting
- Snowflake , oracle , SQL Server)
- Strong understanding of distributed computing concepts and data processing frameworks.
- Experience in building ETL/data pipelines for large-scale datasets.
- Proficiency in SQL and data modeling.
Preferred Qualifications:
- Hands-on experience with data lakes, data warehouses, and scalable ETL pipeline design, including batch and real-time processing architecture.
- Strong understanding and practical exposure to Agile software development methodologies (Scrum) and SDLC practices.
- Proven experience in Banking domain, supporting use cases such as fraud detection, risk analytics, regulatory reporting, and customer insights.
- Excellent analytical, problem-solving, and communication skills, with the ability to translate business requirements into scalable technical solutions.
- Demonstrated ability to work effectively in cross-functional, multi-stakeholder environments, collaborating with Business, Data Engineering, and Architecture teams.
- Experience with real-time data streaming frameworks such as Kafka and Spark Streaming for low-latency processing.
- Understanding data modeling concepts (dimensional modeling, snowflake schemas) to support analytics workloads.
- Experience and desire to work in a global delivery environment.
More about this job
Responsibilities
Build and support scalable ETL and data ingestion pipelines for large-scale datasets using DBT, Python, AWS, and PySpark. Develop data processing solutions for analytics and business use cases, potentially including batch and real-time workloads.
Requirements
Requires at least seven years of IT experience and at least seven years of experience with AWS and Python technologies, along with strong expertise in DBT, Python, AWS, and PySpark. Candidates should have hands-on pipeline and database experience, strong SQL and data modeling skills, and an understanding of distributed computing; banking, streaming, Agile, and global delivery experience are preferred.
Skills
- DBT
- Python
- AWS
- PySpark
- Data Ingestion Pipelines
- Shell Scripting
- Snowflake
- Oracle
- SQL Server
- Distributed Computing
- ETL Pipelines
- SQL
- Data Modeling
- Kafka
- Spark Streaming
- Agile (Scrum)
Visa sponsorship
Not detected in the job text
Categories
- Technology
- Data & Analytics
- Software
- Finance & Accounting
Keywords
- DBT
- Python
- AWS
- PySpark
- Data Ingestion
- Shell Scripting
- Snowflake
- Oracle
- SQL Server
- Distributed Computing
- Data Processing Frameworks
- ETL
- Data Pipelines
- Large-Scale Datasets
- SQL
- Data Modeling
- Data Lakes
- Data Warehouses
- Batch Processing
- Real-Time Processing
- Agile
- Scrum
- SDLC
- Banking
- Fraud Detection
- Risk Analytics
- Regulatory Reporting
- Customer Insights
- Kafka
- Spark Streaming