Kafka Platform Admin/Kafka Developer

Kafka Platform Admin/Kafka Developer

Contract Term: 12 months, renewable

Work Location: Calgary, AB

Work Model: 5 days in office

We are looking for an experienced Kafka Platform Administrator / Kafka Developer with strong expertise in Apache Kafka, Kafka on-premises and cloud environments, and Red Hat OpenShift Service on AWS (ROSA). The ideal candidate should have hands-on experience in Kafka platform administration, cluster management, topic administration, connector development, and ongoing maintenance of existing integrations. The role requires a combination of platform engineering, development, operational support, and monitoring skills to ensure reliable, scalable, and highly available event-driven integration solutions.

Key Responsibilities:

• Administer, configure, and maintain Kafka platforms across on-premises, cloud, and Red Hat OpenShift Service on AWS (ROSA) environments.
• Manage Kafka clusters, brokers, topics, partitions, replication factors, retention policies, consumer groups, and access controls.
• Perform Kafka platform administration, including cluster health monitoring, capacity planning, performance tuning, and availability management.
• Deploy, configure, and support Kafka components and services across multiple environments, including development, testing, staging, and production.
• Develop, configure, deploy, and maintain Kafka connectors using Kafka Connect and related integration frameworks.
• Create new source and sink connectors to integrate Kafka with databases, enterprise applications, cloud services, and other external systems.
• Maintain and enhance existing connectors, troubleshoot connectivity and data ingestion issues, and implement fixes, upgrades, and performance improvements.
• Support connector lifecycle management, including configuration, deployment, validation, error handling, monitoring, and recovery.
• Manage Kafka topics and schemas, including topic creation, configuration changes, schema compatibility, and data retention requirements.
• Work with Kafka Connect, connector plugins, serialization formats, and data transformation mechanisms to support reliable data movement.
• Use Kafka CLI commands and administrative tools to manage topics, consumer groups, configurations, offsets, ACLs, and cluster operations.
• Troubleshoot producer and consumer issues, consumer lag, broker failures, connector task failures, rebalancing, and message delivery problems.
• Configure and maintain monitoring, alerting, and dashboards using Datadog to track Kafka cluster health, throughput, latency, consumer lag, connector status, and infrastructure utilization.
• Develop and maintain Datadog monitors, alerts, dashboards, and incident notifications to proactively identify platform and integration issues.
• Use GitHub for source control, connector configuration management, code reviews, versioning, and collaboration across development and operations teams.
• Support CI/CD pipelines and automated deployment processes for Kafka configurations, connector changes, and application releases.
• Provide L2/L3 production support, participate in incident management, perform root cause analysis (RCA), and implement preventive measures.
• Maintain operational documentation, runbooks, standard operating procedures, and platform administration guidelines.
• Collaborate with application developers, infrastructure teams, architects, security teams, and business stakeholders to deliver secure and reliable event-driven integration solutions.
• Follow best practices for Kafka security, platform governance, high availability, disaster recovery, and operational excellence.

Required Skills:

• Strong experience in middleware, messaging platforms, streaming technologies, platform administration, or integration development, with strong hands-on Kafka experience.
• Strong expertise in Apache Kafka administration and development across on-premises and cloud-based environments.
• Hands-on experience with Kafka on Red Hat OpenShift Service on AWS (ROSA), including deployment, configuration, operations, troubleshooting, and platform support.
• Experience administering Kafka clusters, brokers, topics, partitions, replication, retention settings, consumer groups, and access control lists (ACLs).
• Strong understanding of Kafka architecture, including producers, consumers, brokers, partitions, offsets, replication, and consumer group management.
• Practical experience with Kafka Connect, including connector deployment, configuration, troubleshooting, and lifecycle management.
• Experience developing or customizing Kafka source and sink connectors and maintaining existing connectors in production environments.
• Strong knowledge of connector configuration parameters, plugin management, connector tasks, error handling, retries, dead-letter queues, and offset management.
• Proficiency in Kafka CLI commands and administrative utilities for topic management, consumer group inspection, configuration changes, ACL management, and troubleshooting.
• Experience with command-line tools and scripting for platform administration, automation, and operational support.
• Hands-on experience with GitHub, including repositories, branching strategies, pull requests, code reviews, version control, and configuration management.
• Experience with Datadog monitoring, including dashboards, monitors, alerting, log analysis, metrics, and troubleshooting production incidents.
• Strong understanding of Kafka security, authentication, authorization, encryption, TLS, SASL, and access management.
• Experience troubleshooting Kafka performance issues, broker availability, connector failures, consumer lag, throughput bottlenecks, and message processing errors.
• Knowledge of REST APIs, JSON, Avro, XML, and other data serialization or integration formats.
• Experience working with relational databases, SQL, and enterprise systems for source/sink integrations.
• Understanding of Linux/Unix environments, networking fundamentals, DNS, certificates, and connectivity troubleshooting.
• Experience with CI/CD processes, deployment automation, configuration management, and environment promotion.
• Strong analytical, debugging, problem-solving, documentation, and communication skills.

Nice to Have:

• Experience with Red Hat OpenShift, Kubernetes, containerized workloads, and cloud-native platform operations.
• Exposure to Red Hat Streams for Apache Kafka, Strimzi, or managed Kafka offerings such as Amazon MSK and Confluent Cloud.
• Experience with Schema Registry, schema evolution, and compatibility management.
• Knowledge of Kafka ecosystem tools such as MirrorMaker 2, Kafka REST Proxy, and Kafka UI or equivalent administration tools.
• Experience integrating Kafka with Oracle, SQL Server, PostgreSQL, Salesforce, SAP, or other enterprise applications.
• Experience with Debezium, Change Data Capture (CDC), JDBC connectors, and database replication patterns.
• Knowledge of Python, Java, or shell scripting for connector development, automation, and troubleshooting.
• Experience with Jenkins, GitHub Actions, Azure DevOps, or other CI/CD tools.
• Exposure to AWS services, IAM, networking, secrets management, and cloud observability.
• Experience with disaster recovery, multi-cluster replication, high availability, and business continuity planning.
• Experience in enterprise production support, on-call operations, incident response, and service-level management.
• Kafka-related certifications or relevant Red Hat, cloud, or streaming platform certifications.