Position: Kafka Engineer/Admin
Location: Hybrid- Mississauga, ON
Job type: Full-Time/Permanent Hiring
We are seeking a Senior Kafka Platform Engineer to administer, maintain, and optimize enterprise Apache Kafka platforms across Development, Test, UAT, and Production environments. The role focuses on Kafka infrastructure, cluster administration, security, performance, HA/DR, monitoring, upgrades, automation, and production support.
Key Responsibilities
- Install, configure, administer, and maintain Apache Kafka clusters across multiple environments.
- Manage Kafka brokers, topics, partitions, replication factors, and consumer groups.
- Monitor Kafka cluster health, availability, capacity, and resource utilization.
- Perform performance tuning, capacity planning, scaling, and optimization for growing workloads.
- Troubleshoot Kafka infrastructure, messaging, connectivity, availability, and performance issues.
- Configure Kafka security including SSL/TLS, SASL, ACLs, authentication, and encryption.
- Administer Kafka Connect, Schema Registry, and related Kafka ecosystem components.
- Design and maintain High Availability (HA) and Disaster Recovery (DR) solutions.
- Perform Kafka upgrades, patching, migrations, and platform maintenance with minimal production impact.
- Configure monitoring, logging, alerting, and incident-management processes.
- Support production deployments, release activities, and environment readiness assessments.
- Perform root cause analysis (RCA) and implement preventive actions for recurring production issues.
- Automate Kafka administration and operational activities using Shell, Python, or similar scripting.
- Develop and maintain operational procedures, runbooks, architecture documentation, and support guides.
- Collaborate with application, data engineering, infrastructure, and DevOps teams to onboard and support Kafka applications.
- Ensure Kafka platforms meet organizational security, governance, audit, and regulatory requirements.
- Participate in on-call support for critical production environments.
- Drive continuous improvements in Kafka performance, scalability, security, availability, and operational efficiency.
Required Skills
- Strong hands-on experience with Apache Kafka administration and platform operations.
- Experience managing Kafka brokers, topics, partitions, replication, and consumer groups.
- Strong knowledge of Kafka performance tuning, monitoring, capacity planning, and cluster scaling.
- Hands-on experience with Kafka security, including SSL/TLS, SASL, ACLs, and authentication.
- Experience with Kafka Connect and Schema Registry.
- Strong experience designing and supporting Kafka HA/DR environments.
- Experience with Kafka upgrades, patching, migration, and production maintenance.
- Strong troubleshooting and production incident-management experience.
- Experience with Shell scripting, Python, or automation tools.
- Experience with enterprise monitoring, logging, alerting, and RCA.
Preferred / Additional Qualifications
- Experience supporting large-scale, multi-environment Kafka platforms.
- Experience with DevOps, CI/CD, infrastructure automation, and cloud platforms.
- Experience with Kafka observability and enterprise monitoring platforms.
- Knowledge of security, governance, audit, and regulatory controls.
- Strong documentation, communication, and cross-functional collaboration skills.
