Red Teaming | Generative AI Analyst - California

Welo Global

Posted 2 months ago

Full Time

, California

Remote OK

Smart Summary

Responsibilities

Interact with generative AI models to create and evaluate prompts that test safety boundaries and identify model failures. Document breakability and align responses with safety taxonomies and project guidelines.

Qualifications

You have native-level English proficiency with excellent written communication and creative writing abilities. You can critically evaluate open-ended AI model responses, follow complex guidelines precisely, and are comfortable working with sensitive content. Prior experience with red teaming, AI safety, or content evaluation is preferred.

Must Have Skills for ATS

generative AI

AI safety

large language models

red teaming

content evaluation

content moderation

QA

AI model evaluation

multimodal AI

prompt engineering

safety taxonomies

policy guidelines

evaluation rubrics

defect categories

RLHF

jailbreak testing

Job Description

About the Role

We are hiring Red Teaming | Generative AI Analyst to support generative AI safety evaluation. In this role, you will interact with AI models, create and evaluate prompts, and identify where model responses fail against defined safety expectations.

Project Details

  • Job Title: Red Teaming | Generative AI Analyst
  • Location: Remote with the option to work onsite in selected cities
  • Hours: 40 hours per week
  • Employment Type: W2 Full-Time Employee
  • Pay Rate: $47.44/hour
\nWhat You’ll Do
  • Interact with generative AI models using project-provided guidelines, safety taxonomies, and attack-vector guidance.
  • Create and evaluate prompts designed to test model behavior across safety-related categories.
  • Identify where model responses become unsafe, noncompliant, inconsistent, or otherwise problematic.
  • Document model breakability, effort level, point of failure, and relevant category alignment.
  • Review text, image, audio, video, or other multimodal content as required by the workflow.
  • Apply detailed guidelines consistently across short, high-volume production sprints.
  • Use sound judgment to evaluate ambiguous, edge-case, or policy-sensitive outputs.
  • Conduct self-review to ensure work is accurate, complete, and aligned with project expectations.
  • Flag unclear guidelines, tooling issues, or recurring model behavior patterns.
  • Participate in calibration, feedback, and quality review sessions to improve consistency.
  • Maintain readiness to pivot quickly between different red teaming runs when active work is launched.
Requirements:
  • Native-level or near-native English proficiency with excellent written communication skills.
  • Work Authorization is required for the role.
  • Strong creative writing ability and comfort constructing varied prompts.
  • Experience with red teaming, safety data annotation, content evaluation, safety review, content moderation, QA, or AI model evaluation preferred.
  • Strong attention to detail and ability to follow complex project guidelines.
  • Ability to think critically and evaluate open-ended model responses.
  • Comfort working with sensitive, adult, NSFW, or policy-relevant content where required.
  • Interest in generative AI, AI safety, large language models, or emerging AI technologies.
  • Ability to work quickly and accurately during short production windows.
  • Bachelor’s degree or equivalent practical experience preferred.
Ways to Stand Out from the Crowd
  • Background in creative writing, English, linguistics, journalism, communications, policy, trust and safety, or content moderation.
  • Experience evaluating generative AI prompts and responses.
  • Familiarity with AI safety, red teaming, jailbreak testing, RLHF, or model evaluation workflows.
  • Experience working with safety taxonomies, policy guidelines, evaluation rubrics, or defect categories.
  • Prior experience reviewing sensitive, adult, NSFW, or policy-relevant content in a professional setting.
  • Experience with multimodal AI workflows involving text, image, audio, or video.
  • QA/testing experience within AI, data operations, content review, or annotation environments.
  • Ability to explain a repeatable approach for staying consistent during high-volume, judgment-based work.
\n

Welo Global

Welocalize, a Welo Global brand, serves localization teams through AI-enabled multilingual content solutions that enable enterprises to operate and scale globally. Welocalize combines AI, automation, and human expertise to support enterprises in more than 300 languages, enabling accurate, culturally aligned, and compliant multilingual content at scale. Welocalize’s Opal Platform comprises patented technology designed to automate multilingual content and improve workflow performance for enterprises. Opal functions as an agentic system that orchestrates AI, automation, and human expertise to coordinate content workflows across the lifecycle, improving speed, scalability, and operational control for global enterprises. These solutions operate within a secure and compliant environment supported by seven ISO certifications. Welocalize is headquartered in New York with offices worldwide.
Runway Icon
Boost Your Interview Chances

With Runway

See Your Fit for This Role

1-5 min

Your Score

?

Top Applicants

90%

Your Job Search Advantage

Key Gaps & Next Steps:

Address these in your resume & Interview

Top Strengths For This Role

Highlight these in your cover letter & interview

Your Interview Guide

A Personalized Interview Strategy

Freshest Opportunities

Never Miss a Good Fit

Get notified when jobs mach your criteria