How Social Department built culture-aware video discovery
<p><em>This post is co-written by Mariah Miller and Sirajus Salekin from AWS and Nicholas Chase from Social Department.</em></p> <p>Entertainment marketing teams manage tens of thousands of video assets across studios, networks, and streaming applications. Most campaigns begin with the same video discovery problem: finding the one moment in the footage that will resonate with an audience. When a brief lands, the clock starts, and what they need is almost never something a database can describe.</p> <p>Semantic search, on its own, has a ceiling: It answers the question you can articulate. Ask it for a character holding a coffee mug and it will find every coffee mug in a thousand episodes. The queries that actually drive entertainment campaigns look nothing like that. They are questions of judgment: which moment in a show will travel, which line is quotable, and which single frame will work as a meme.</p> <p>Those questions can’t be typed into a search box. Answering them requires two things the search index doesn’t have: an editorial definition of what “quotable” or “memeable” means for a given library, and a continuously refreshed view of what is moving outside the library. Historically, answering them meant a person with deep knowledge of the show watching footage for a week, and that person doesn’t scale across a catalog.</p> <p>In this post, we describe how Social Department, the AI operating system for entertainment marketing, set out to encode those editorial questions as executable retrieval pipelines. The result is QuickSearch, a set of single-action queries that return the clips a marketer is looking for without requiring them to describe it: Shareable Scenes, Quotable Quotes, Memeable Moments, and Trend Align.</p> <p>The models in this solution run on Amazon Bedrock, so customer data stays within the AWS data boundary. It isn’t shared with model providers and isn’t used to train the underlying models.</p> <p>Social Department’s approach to responsible AI is that AI proposes and people decide. In production, code checks generated output against the source material (for example, a quote is discarded if it isn’t in the episode transcript). Amazon Bedrock Guardrails helps screen customer search text for prompt attacks, and a person on the customer’s team approves every post before it publishes.</p> <p>Social Department owns the pipeline architecture, the retrieval system, and the editorial criteria behind each action. TrackIt, an AWS Premier Tier Services Partner, implemented and delivered against that design. TwelveLabs’ Marengo 3.0 and Pegasus 1.2 models on <a href="https://aws.amazon.com/bedrock/" target="_blank" rel="noopener">Amazon Bedrock</a> provide the scene-level video reasoning the pipeline builds on. AWS supplies the infrastructure, the model access including Anthropic’s Claude models, and <a href="https://aws.amazon.com/opensearch-service/" target="_blank" rel="noopener">Amazon OpenSearch Service</a>, using its serverless option, Amazon OpenSearch Serverless, for retrieval.</p> <p><strong>A note on scope:</strong> This post describes the AWS portions of the system: the ingest and indexing pipeline, the retrieval path, and the services that run them. The components Social Department owns, including its media and data backbone, content analysis and discovery engine, editorial criteria, and evaluation program, are described only to the depth needed to follow the architecture. The named services that follow are the ones this post is about, not an inventory of Social Department’s full system.</p> <h2 id="about-social-department">About Social Department</h2> <p>Social Department is the AI operating system for entertainment marketing, serving major studios, networks, streamers, and FAST applications. It automates campaign strategy, asset creation, publishing, and performance optimization across a client’s full content library. Most of those functions depend on locating the right moment inside the footage first, and semantic clip discovery has been core to the product from the beginning.</p> <h2 id="why-semantic-search-alone-was-insufficient">Why semantic search alone was insufficient</h2> <p>Entertainment marketing operates under two constraints that a general-purpose search layer can’t satisfy.</p> <h3 id="the-valuable-queries-are-the-ones-nobody-can-phrase">The valuable queries are the ones nobody can phrase</h3> <p>A marketer can describe an object, a location, or a line of dialogue, but not what will travel. That judgment, this moment lands and that one doesn’t, is the actual skill in the job. It was never automatable because it depends on knowing how an audience behaves rather than what a video contains. A search layer that only understands content requires a human to supply the intent, and supplying that intent well requires having already watched the show.</p> <h3 id="trend-signal-lives-outside-the-library-and-moves-on-a-different-clock">Trend signal lives outside the library and moves on a different clock</h3> <p>No amount of video understanding tells you what audiences are talking about this morning. That information is created outside the library and changes hourly, while a library changes only when new content is added. A system that answers these queries has to acquire trend signal continuously and reconcile it against the library at request time.</p> <p>That distinction drove the core architectural decision: content understanding and trend acquisition run as independent pipelines on their own schedules and are joined only when a user selects a QuickSearch action.</p> <h2 id="solution-overview">Solution overview</h2> <p>With QuickSearch, teams can interact with a video library in a different way, one where the system proposes rather than waits. It provides four actions, each with its own retrieval strategy, editorial criteria, and ranking model. The walkthrough that follows describes how each is implemented. The list here introduces what they do.</p> <ol type=
Nguyên văn nguồnChi tiết nguồn
- How Social Department built culture-aware video discovery AWS Machine Learning Blog · 19 thg 11, 2026