Sr Python Developer/ Data Engineer
- Mississauga, ON, Canada
Full-time
Applications open in a new tab.
Job details
- Job type
- Full-time
- Apply by
- Posting language
- English
- Posted
Original job description
Job Title: Data Engineer/ Python Developer
Mississauga, ON, Canada
Fulltime
Mandatory Skills: Python, Spark, PySpark
Responsibilities:
· Design, develop, and maintain software solutions using Python.
· Developed Python and pyspark programs for data analysis.. Good working experience with python to develop Custom Framework for generating of rules (just like rules engine).
· Developed Python code to gather the data from HBase and designs the solution to implement using Pyspark. Apache Spark DataFrames/RDD's were used to apply business transformations and utilized Hive Context objects to perform read/write operations..
· Develop back-end services, APIs, and integrate databases.
· Optimize applications for performance, security, and maintainability.
· Troubleshoot and resolve software defects and issues.
Required Skills:
· Strong proficiency in Python and Spark.
More about this job
Responsibilities
Design, develop, and maintain Python software and data solutions, including custom rule-generation frameworks and PySpark programs that gather and transform data from HBase using Spark and Hive. Develop back-end services and APIs, integrate databases, optimize applications for performance, security, and maintainability, and troubleshoot defects.
Requirements
Strong proficiency in Python and Spark is required, with experience developing Python and PySpark programs for data analysis and data processing. The role also calls for experience with HBase, Spark DataFrames or RDDs, Hive, and custom framework development.
Skills
- Python
- Apache Spark
- PySpark
- HBase
- Hive
- Data Engineering
- Data Analysis
- DataFrames
- RDDs
- Business Transformations
- Custom Framework Development
- Rules Engine Development
- Back-End Services
- API Development
- Database Integration
- Troubleshooting
Visa sponsorship
Not detected in the job text
Categories
- Technology
- Data & Analytics
- Software
Keywords
- Python
- Apache Spark
- PySpark
- HBase
- Hive
- Data Engineering
- Data Analysis
- Spark DataFrames
- RDDs
- Business Transformations
- Custom Framework
- Rules Engine
- Back-End Services
- APIs
- Database Integration
- Performance Optimization
- Application Security
- Maintainability
- Software Defects
- Troubleshooting
- Information Technology