On LLMOp/DevOps they are looking for people who have setup and operated the Agentcore runtime environment.
DevOps
MUST
- AI-DLC practices
- Agentic code generation (Claude Code SDK, Kiro CLI) with human-in-the-loop checkpoints
- GitHub Actions: reusable workflows, environments and approvals, OIDC federation to AWS
- Container image builds and Amazon ECR (AgentCore Runtime requires ARM64 images)
- AgentCore Runtime deployment (versions, endpoints, rollback) and VPC mode
- VPC endpoints / PrivateLink and security groups
- Event-driven automation: EventBridge, Lambda, Step Functions
- Terraform
- IAM for CI/CD
SHOULD
- AgentCore Gateway rate limits and PortKey admin API (kill switch)
- Game-day practice with audit evidence
- Image signing (Cosign)
- Python
COULD
- Strands Agents SDK
- Backstage templates
- ROSA / OpenShift networking (later phase)
- AI eng
MUST
- AgentCore Evaluations and Bedrock evaluation
- Golden-dataset design and curation with SMEs
- LLM-as-judge design and calibration against human labels
- Integrating evaluations as a CI gate
- Langfuse / OpenTelemetry
- Python
SHOULD
- RAGAS or equivalent
- Trajectory evaluation
- Understanding of agentic red-teaming results (to feed them into the gate)
COULD
- Bedrock Guardrails
- Sampling design for continuous evaluation
- Strands Agents SDK
