AgentOps / DevSecOps Engineer
KPI Partners
- Experience
- 5+ years
- Location
- Remote · Bengaluru, India
- Job type
- Contract (6 months)
- Azure Databricks
- Genie Agents
- Azure DevOps
- GitHub Actions
- MLflow
- Unity Catalog
- Unity Gateway
- Azure security
About the role
Build, maintain, and optimize data and ML operations on Azure Databricks and Genie Agents. Automate deployment of data and ML applications via CI/CD. Ensure robust data governance and security aligned with Azure protocols. Improve operational efficiency of machine learning models. Provide observability into data pipelines and model performance.
What you’ll do
- Design, implement, and manage CI/CD pipelines to automate deployment processes for data and machine learning applications
- Collaborate with data engineers, data scientists, and other stakeholders to enhance the operational efficiency of machine learning models
- Utilize Azure Databricks and Genie Agents for scalable data processing and analytics
- Implement and manage MLflow for tracking experiments, managing models, and facilitating reproducibility in machine learning workflows
- Ensure observability of the data pipeline and model performance through monitoring and logging strategies
- Manage access control and governance practices using Unity Catalog and Unity Gateway
- Implement best practices for Azure security, ensuring the integrity and confidentiality of data and systems
What we’re looking for
Must have
- Azure Databricks
- Genie Agents
- Azure DevOps
- GitHub Actions
- MLflow
- Unity Catalog
What makes this role challenging
- Building reliable CI/CD for ML workloads where models, data, and code all change
- Keeping ML pipelines observable and reproducible across experiments and deployments
- Enforcing data governance and access control without slowing down data scientists
- Hardening Azure security across data and ML systems
What success looks like
- First 30 days
- Ramped on the Azure Databricks, Genie Agents, and MLflow stack and shipped initial CI/CD pipeline improvements
- By 90 days
- Owning CI/CD pipelines and MLflow model lifecycle management end-to-end with monitoring and logging in place
- First year
- Set the governance, observability, and Azure security standards for the data and ML platform
Tech stack
- AWS
- Microsoft Azure
- Google Cloud Platform
- Databricks
- Snowflake
- Microsoft Fabric
- BigQuery
- Oracle
- ServiceNow
- Salesforce
- Spark
- Kafka
About KPI Partners
KPI Partners is a global strategic consulting firm focused on Enterprise AI, Data, Analytics, and Digital Transformation. Founded in 2006, it delivers Data & BI services, platform modernization, and enterprise software solutions to 300+ customers, including Fortune 500 companies across retail, manufacturing, financial services, life sciences, and high-tech. The company is a strategic partner with AWS, Microsoft, Databricks, Snowflake, Google Cloud, and Anthropic.
IT Services and IT Consulting (Data, Analytics & AI) · 600+ consultants (501-1000 employees)
Job information
Description
KPI Partners is seeking an experienced AgentOps / DevSecOps Engineer to join our dynamic team. The successful candidate will play a crucial role in building, maintaining, and optimizing our data and machine learning operations using Azure Databricks and Genie Agents. This position requires proficiency in implementing continuous integration and continuous deployment (CI/CD) practices utilizing Azure DevOps or GitHub Actions, along with expertise in MLflow for machine learning model management and observability. The ideal candidate will also have experience with Unity Catalog and Unity Gateway, ensuring robust data governance and security measures in alignment with Azure's security protocols.
Key responsibilities
- Design, implement, and manage CI/CD pipelines to automate deployment processes for data and machine learning applications.
- Collaborate with data engineers, data scientists, and other stakeholders to enhance the operational efficiency of machine learning models.
- Utilize Azure Databricks and Genie Agents for scalable data processing and analytics.
- Implement and manage MLflow for tracking experiments, managing models, and facilitating reproducibility in machine learning workflows.
- Ensure observability of the data pipeline and model performance through monitoring and logging strategies.
- Manage access control and governance practices using Unity Catalog and Unity Gateway.
- Implement best practices for Azure security, ensuring the integrity and confidentiality of data and systems.
Skills required
- Proficiency with Azure Databricks and Genie Agents.
- Experience with CI/CD tools, specifically Azure DevOps or GitHub Actions.
- Familiarity with MLflow for machine learning operations and model lifecycle management.
- Strong knowledge of observability techniques and monitoring tools.
- Experience with Unity Catalog and Unity Gateway for data governance.
- Understanding of Azure security principles and practices.
- Strong problem-solving skills and ability to work collaboratively in a team environment.
- Excellent communication skills to convey complex technical concepts effectively.
Tools required
- Azure Databricks
- Genie Agents
- Azure DevOps or GitHub Actions
- MLflow
- Monitoring and observability tools
- Unity Catalog and Unity Gateway
- Azure Security tools
5+ Yrs
Salary : 2.5L per month