AI/ML Scientist

About the Role

• Would you like to shape how an AI agent reasons and talks with customers? We’re looking for an Applied AI/ML Scientist to join a U.S.-based social media management software company. You’ll improve the reasoning and conversational performance of its AI agent by evaluating models, prompts, tool descriptions, and retrieval approaches.
• You’ll use applied science to move real business metrics, such as revenue or adoption, and to resolve customer pain points. You’ll work with technical and non-technical stakeholders, test new AI/ML ideas quickly, and measure the impact of every improvement you ship.

About You

• You have 2 to 3 years of hands-on experience in machine learning and AI, and a track record of quantifiable business impact.
• You can explain the outcome you set out to move, how you moved it, and how applied science helped you get there.
• You think like a product person and tie your work to its impact on the customer experience.
• You enjoy evaluating AI systems, from prompts to retrieval, to find what makes an agent reason and converse better.
• You communicate clearly with technical and non-technical stakeholders.
• You embrace challenges and keep a growth mindset.

What You’ll Be Doing

• Evaluate AI models and measure the quality of the agent’s reasoning and conversational performance.
• Experiment with prompts and tool descriptions to improve agent behavior.
• Evaluate and compare retrieval mechanisms for retrieval-augmented generation (RAG) systems.
• Build evaluation methods, such as LLM-as-a-judge evaluators, in Python.
• Ship AI/ML improvements that resolve customer pain points and enhance the user experience.
• Partner with product, business, and technical stakeholders to define the outcomes each improvement should drive.
• Query and analyze large datasets with SQL and RESTful APIs to inform product decisions.
• Prototype with new AI/ML technologies to explore their potential for business impact.

What We’re Looking For

• Ownership of outcomes: you know which metric your work is tied to.
• Product thinking and a customer-first mindset.
• A growth mindset and comfort with challenging problems.
• Clear communication with technical and non-technical audiences.
• Curiosity and drive to keep learning and improving.

Technical Requirements

Must-Haves

• 2 to 3 years of hands-on experience in machine learning and AI.
• Python proficiency, including the ability to complete a hands-on Python technical exercise.
• Experience evaluating AI models, experimenting with prompts and tool descriptions, and evaluating retrieval approaches for RAG systems.
• A track record of AI/ML solutions with quantifiable business impact.
• SQL and RESTful APIs for querying and analyzing large datasets.
• Experience with transformer architectures and embeddings.
• Advanced English.

Nice-to-Haves

• Experience with PyTorch, TensorFlow, or XGBoost.
• Designing and running A/B testing frameworks and other rigorous experimentation.
• Developing deep learning and NLP models.
• Building LLM-driven features, including evaluation, fine-tuning, optimization, and prompt engineering.
• Creating APIs and using services on AWS, Azure, GCP, OpenAI, and Anthropic.
• Applying AI/ML model security, bias mitigation, and ethical AI frameworks.
• A software engineering background and a history of success at other tech companies.