Below you can find a detailed outline of how Checkstep, a unified Trust & Safety solution, supports theWell in detecting and preventing CSAE materials from showing up on their social platform.
Classifier Orchestration: Checkstep centralizes all moderation workflows
– automated classification, human review queues, analytics, and policy management
– into a single interface. This reduces complexity and ensures consistent enforcement across content types (text, image, audio, video).
AI Marketplace: Checkstep’s modular AI Marketplace integrates best-in-class third-party providers, enabling dynamic selection of models for each content type andharm category (e.g., hate speech, self-harm, CSAE).
Customizable Workflows: Checkstep supports custom review queues with priority rules, escalation paths, and access controls. Automated actions (webhooks, takedowns) and human-in-the-loop escalation ensure nothing slips through undetected.\
RESTful API: theWell’s platforms submit user-generated content for moderation through Checkstep’s synchronous API endpoints, receiving decisions within hundreds of milliseconds – critical for an environment like theWell’s social space.
Ephemeral Delegated Access: To minimize data liability, Checkstep processes only metadata pointers; raw images/videos remain on theWell’s storage and are pulled for analysis on demand.
Scalable Infrastructure: Behind the API, Checkstep runs on an AWS backend, with a Kafka-driven pipeline that auto-scales to handle theWell’s peak traffic. This ensures consistent SLA adherence and sub-second throughput.
Multiple AI Providers: Checkstep combines specialized vendors: AWS Rekognition, Unitary, SightEngine, OpenAI Moderation, and many others. This maximizes coverage and accuracy across violence, abuse, and CSAE content.
Continuous Benchmarking: Model performance (precision, recall, latency) is monitored and benchmarked; providers are swapped dynamically to maintain high detection rates and low false positives.
Project Arachnid: Checkstep integrates with Project Arachnid’s image matching system – an initiative of the Canadian Centre for Child Protection – to detect previously reported instances of CSAE by comparing user-uploaded images against a global hash database.
Automated Takedown Workflows: Upon a match, Checkstep triggers immediate takedown actions and flags content for human review and law enforcement reporting.
Hash List Updates: Hash databases are automatically refreshed daily, ensuring theWell benefits from the latest reported CSAE signatures.
NCMEC Integration: Checkstep supports direct integration with the National Center for Missing & Exploited Children (NCMEC) CyberTipline. When suspected CSAE is detected, incidents can be packaged and submitted automatically in NCMEC’s required format.
Audit Trails & Compliance: All CSAE detection events and NCMEC submissions are logged in immutable audit trails, supporting regulatory compliance and internal reviews.
Thanks to its partnership with Checkstep, theWell is holistically protected from CSAEmaterials through the means of multiple AI solutions, which work in tandem to detect,remove, and report any sort of violent content – including CSAE content.
