Senior Data Lake Consultant (Cloudera)

Our client is seeking a contract based Senior Cloudera Data Lake Platform Consultant to provide hands-on administration, operational support, and technical leadership for an enterprise Cloudera Data Lake environment.

The consultant will ensure the stability, availability, security, and performance of the existing platform while supporting the organization’s transition to Microsoft Azure and Databricks.

Location: Onsite Toronto

Duration: 12 months

Responsibilities

• Provide daily operational support, administration, monitoring, and maintenance of enterprise Cloudera/Hadoop environments.
• Administer and troubleshoot Cloudera services, including HDFS, Hive, Impala, Spark, Kudu, Oozie, Hue, YARN, Ranger, and Atlas.
• Monitor cluster health, resource utilization, storage capacity, workload performance, and platform availability.
• Investigate and resolve production incidents, performance degradation, failed data pipelines, and service interruptions.
• Perform root-cause analysis, log investigation, performance tuning, and capacity optimization across Hadoop services and supporting infrastructure.
• Support cluster configuration, patching, upgrades, backup and recovery, high availability, and disaster recovery.
• Troubleshoot batch-processing workflows, data ingestion, analytics workloads, reporting, and downstream integrations.
• Manage platform security, including Kerberos authentication, LDAP/Active Directory integration, Ranger policies, encryption, and access controls.
• Coordinate technical escalations and incident resolution with internal teams, Cloudera, and external vendors.
• Support the phased migration of workloads from Cloudera to Microsoft Azure and Databricks, ensuring operational continuity throughout the transition.
• Assist with migration dependency analysis, workload validation, data reconciliation, and legacy-platform decommissioning.
• Maintain technical documentation, operational runbooks, troubleshooting procedures, and architecture diagrams.
• Provide ongoing knowledge transfer and technical guidance to internal support and engineering teams.

Requirements

• 8+ years of experience in data platform engineering, infrastructure support, systems administration, or related technical fields.
• 5+ years of hands-on experience administering and supporting enterprise Cloudera or Hadoop environments.
• Strong expertise with Cloudera Data Platform (CDP), Cloudera Distribution of Hadoop (CDH), and Cloudera Manager.
• Advanced hands-on knowledge of HDFS, Hive, Impala, Apache Spark, YARN, Kudu, and Oozie.
• Experience with additional Hadoop ecosystem technologies, including Hue, Ranger, Atlas, ZooKeeper, and Kafka.
• Proven experience supporting complex, business-critical production data platforms, including incident management, troubleshooting, and root-cause analysis.
• Strong Linux administration skills and proficiency in Shell scripting, Python, and SQL.
• Experience with cluster monitoring, performance tuning, capacity planning, patching, upgrades, and platform maintenance.
• Strong understanding of Hadoop security, including Kerberos, LDAP/Active Directory, Ranger policies, encryption, and access management.
• Experience troubleshooting distributed data-processing workloads, batch jobs, data pipelines, and integrations.
• Familiarity with Microsoft Azure, Azure Data Lake Storage, Databricks, and cloud data-platform migration concepts.
• Experience supporting legacy platforms during modernization, migration, or decommissioning initiatives is preferred.
• Strong analytical, problem-solving, communication, documentation, and stakeholder-management skills.
• Ability to independently manage complex production issues and coordinate with cross-functional technical teams.
• Relevant Cloudera, Azure, Databricks, Linux, or ITIL certifications are considered assets.