NVIDIA is seeking an AI Quality & Operations Analyst to join their team, playing a crucial role in improving Sales AI applications. The successful candidate will work on innovative AI solutions as they move from development into production, collaborating with skilled team members to establish consistent evaluation, quality assurance, and operational practices.
Job Responsibilities:
- Collaborate with application teams to implement consistent AI evaluation and quality assurance standards and workflows.
- Establish test scenarios, assessment standards, quality metrics, and acceptance thresholds.
- Evaluate AI outputs for accuracy, relevance, consistency, safety, and alignment with business workflows.
- Analyze evaluation results, service metrics, production data, and user feedback to monitor application quality.
- Investigate failures, contributing factors, recurring patterns, and performance trends, and collaborate with technical teams on prioritized improvements.
- Support production readiness and launches through testing, validation, documentation, monitoring, issue tracking, and quality checks.
- Develop evaluation datasets, test scenarios, quality reports, and operational documentation that support continuous improvement.
- Assess risks from AI-generated outputs and downstream actions, identifying when additional controls, testing, or human review are needed.
Requirements:
- 2+ years of experience in software quality, application operations, data analysis, AI or machine learning, technical support, business analysis, or a related field.
- Experience testing, evaluating, monitoring, or supporting applications in development or production.
- Familiarity with generative AI, machine learning concepts, APIs, data workflows, or enterprise software.
- Experience analyzing test results, application metrics, user feedback, or other data to identify issues and opportunities.
- Solid attention to detail and experience developing structured test cases, evaluation criteria, documentation, or quality reports.
- Ability to collaborate effectively with Engineering, Product, Data, Solution Architecture, and business teams.
- Ability to prioritize multiple assignments and support time-sensitive production readiness and launch activities.
- A bachelor’s degree or equivalent experience in Computer Science, Engineering, Data Science, Information Systems, or a related technical field is encouraged.
Benefits:
- Equity
- Benefits package
