About the role

Aplyr's Quick Take

This role is about evaluating AI agents at FNZ, focusing on performance, safety, and efficiency. You'll design tests, create evaluation criteria, and document findings, working closely with development teams. It's a hands-on position that requires both generalist skills and some specialization in AI evaluation.

Good fit

Ideal candidates have 3-6 years in software testing or AI/ML development, with strong analytical skills and experience in test automation. A background in evaluating AI applications or data science will help you thrive in this role.

Worth noting

The role emphasizes collaboration with development teams and the creation of an automated evaluation platform, which could be a unique opportunity for those interested in both AI and software engineering. The pay range is competitive for the region.

AI Evaluation Engineer

Location: Pune, India

Seniority: Mid-level (3-6 years)

Purpose: Execute comprehensive evaluations of FNZ's AI agents across the six-pillar framework, working as a generalist while developing specialist expertise in 1-2 pillars.

Key Responsibilities:

Design and conduct evaluations covering Task Performance, Safety, Efficiency, Groundedness, Robustness, and Suitability
Create "golden sets" of test examples representing expert judgment on desired agent behaviour
Develop evaluation rubrics and scoring criteria aligned to FNZ Evaluation Framework principles
Build comprehensive test suites covering happy paths, edge cases, and adversarial inputs
Evaluate multi-step agentic workflows: planning, tool selection, execution, error handling
Assess agent groundedness: verify outputs against knowledge bases, detect hallucinations
Document findings with clear evidence; collaborate with development teams on remediation
Contribute to building automated evaluation platform and CI/CD integration

Skills and Experience:

3-6 years in software testing, QA engineering, AI/ML development, or data science
Hands-on test automation skills, experience with ML frameworks highly valuable
Practical experience evaluating LLM applications, RAG systems, or AI agents
Understanding of prompt engineering, retrieval-augmented generation, and agent architectures
Analytical mindset to decompose complex agent behaviours and identify failure modes
Strong documentation and presentation skill

About FNZ

FNZ is committed to opening up wealth so that everyone, everywhere can invest in their future on their terms. We know the foundation to do that already exists in the wealth management industry, but complexity holds firms back.

We created wealth’s growth platform to help. We provide a global, end-to-end wealth management platform that integrates modern technology with business and investment operations. All in a regulated financial institution.

We partner with the world’s leading financial institutions, with over US$2.4 trillion in assets on platform (AoP).

Together with our clients, we empower nearly 30 million people across all wealth segments to invest in their future.

Skills & Tags

Aplyr's read

FNZ is a leading global investment platform transforming wealth management through cutting-edge technology, attracting professionals passionate about finance and innovation.
Synthesized from recent postings & public sources

What's promising

•FNZ's platform is pivotal in modernizing wealth management for financial institutions worldwide.
•The company shows strong growth potential with its expanding global footprint.
•FNZ invests heavily in technology, offering employees opportunities to work on innovative projects.

What to watch

•The financial technology sector is highly competitive, posing challenges for FNZ to maintain its market position.
•Rapid expansion may lead to integration and operational challenges.
•Regulatory changes in financial services could impact FNZ's business model.

Why FNZ

•FNZ provides a comprehensive platform that integrates investment management and wealth solutions.
•The company's focus on end-to-end solutions differentiates it from niche fintech providers.
•FNZ's global reach allows employees to engage with diverse markets and cultures.

Aplyr’s read is generated by AI from public sources. Was it useful?

About FNZ

FNZ

fnz.com

View company

FNZ is a global investment platform that provides technology and services to financial institutions, enabling them to manage investments and deliver wealth management solutions.