Introducing Sama Red Team: Elevating AI Integrity & Security

Sama, providing data annotation and model validation for some of the world’s largest artificial intelligence (AI) models, has unveiled Sama Red Team, a new solution aimed at helping developers proactively enhance a model’s safety and reliability.

It ranks among the first solutions specifically designed for generative AI and large language models. Drawing upon the expertise of machine learning (ML) engineers, applied scientists, and human-AI interaction designers, the Sama Red Team aids in evaluating a model’s fairness and safeguards across various modalities such as text, image, and voice search.

Duncan Curtis, SVP of AI products and technology at Sama, emphasized the importance of testing generative AI models for potential vulnerabilities. He stated, “Generative AI models may seem reliable, but there are ways to bypass their safeguards, posing risks to public safety, privacy protection, and compliance with laws. Sama Red Team conducts tests to identify and rectify these vulnerabilities before they are exposed to the public, providing developers with actionable insights to patch these weaknesses.”

Sama Red Team assesses models on 4 key competencies:

• In fairness testing, teams simulate real-world scenarios that may compromise a model’s built-in safeguards and result in offensive or discriminatory content.
• For privacy testing, Sama experts craft prompts designed to make a model reveal sensitive data, such as Personal Identifiable Information (PII), passwords or proprietary information about the model itself.
• In public safety testing, teams act as adversaries and mimic real-world threats to safety, including cyberattacks, security breaches or mass-casualty events.
• For compliance testing, Sama Red Team simulates scenarios to test a model’s ability to detect and prevent unlawful activities such as copyright infringement or unlawful impersonation.

These rigorous tests follow direct consultations with clients to determine a model’s desired behavior in specific use cases. After conducting an initial vulnerability assessment, we test a series of prompts and evaluate the model’s output. Based on the results, we refine prompts or create new ones to further probe vulnerabilities. Sama’s workforce of over 4,000 highly trained annotators can scale up these tests as needed.

Sama GenAI, among the company’s suite of solutions for Generative AI, foundation, and large language models (LLMs), facilitates human feedback loops throughout the model development process. It covers data creation, fine-tuning, and continuous evaluation. Like all Sama services, it utilizes SamaHub™ for collaboration and reporting, supported by SamaAssure™ for quality assurance and SamaIQ™ for extra insights into model vulnerabilities.