How to integrate Audible Magic with Amazon IVS to identify copyrighted content
This blog was co-authored with Amazon AWS and was originally published on the AWS for M&E Blog on January 25th, 2024. The original blog can be found here: https://aws.amazon.com/blogs/media/how-to-integrate-audible-magic-with-amazon-ivs-to-identify-copyrighted-content/
Introduction
Amazon Interactive Video Service (Amazon IVS) from Amazon Web Services (AWS) is a managed live streaming service that enables businesses to build interactive experiences and applications to delight their customers with low-latency live video streams.
Customers have leveraged Amazon IVS and built user generated content (UGC) platforms for use cases ranging from live shopping to fitness, education, gaming, and live event experiences over live video. As users engage with these platforms built on top of Amazon IVS, there is the potential for both bad actors and well-intentioned users to stream copyrighted content. This may pose challenges for customers that need to receive and process takedown requests. Even with proper licensing, the use of copyrighted content may require attribution and reporting. In both cases, the ability to properly identify copyrighted content becomes important.
How Audible Magic Technology works
Audible Magic’s Emmy-winning automatic content recognition (ACR) technology presents a possible solution. Leveraging longstanding first-party relationships with thousands of record labels, music publishers, and other rights holders, the integration of Audible Magic’s ACR technology within Amazon Interactive Video Service may help to provide a protective shield for platforms and rights holders, while helping to establish a trustworthy environment for creators and consumers. Audible Magic’s solution enables UGC platforms to actively monitor and filter out unauthorized content while enabling the recognition and reporting of licensed content. It is designed to help reduce DMCA takedown notice disruptions and ensure accurate license administration.
Audible Magic’s ACR technology is tuned to identify copyrighted content embedded in user-generated media. When media is uploaded to a platform, the platform sends digital fingerprints of the media to Audible Magic’s information service, where they are compared to reference fingerprints submitted directly to Audible Magic by content owners. The service then returns results indicating a match or no-match condition. If matching content is found, the content owner’s business rules are returned by Audible Magic’s information service. Business rules express the content owner’s wishes to either block or allow content in a media sharing environment.
The next section of this blog post describes how to integrate Audible Magic’s ACR technology with Amazon IVS to enforce copyrights in live-streamed content.
IVS and Audible Magic end-to-end solution architecture
The simplest of the multiple ways to integrate Audible Magic’s ACR technology service with Amazon IVS.
When a stream begins, it generates a “stream started event” on the event bridge, which triggers an event rule. This trigger pushes a message containing stream information to a SQS queue, where an ECS service reads the message and starts scanning the stream with Audible Magic’s Toolkit. Whenever copyrighted material is identified, a JSON response file is generated with information about the match and is saved to an Amazon S3 bucket. Additionally, an email notification is sent pointing to the location of the JSON response file.
Handling toolkit responses
There are many ways you can use the responses from Audible Magic to take action on the detection of copyrighted content. For example: 1) sending notifications to an AWS Lambda function to display the currently playing song in an overlay on the screen, 2) muting portions of the stream that contain copyrighted content, or 3) automatically changing the song. Ultimately, Audible Magic’s Toolkit responses can extend out through a variety of applications and enable businesses to take actions that align with their content policies.
Email notifications can also help keep track of identifications that occurred during a stream. Further analysis of these identifications may aid with: 1) policies to prevent streaming copyrighted material in the future, 2) reporting on music usage, or 3) making informed edits to a video before VOD publishing.
Integrating Audible Magic – detailed steps
This section provides a walk through the steps to achieve the integration outlined in the end-to-end architecture diagram. If you would like to try these steps in your account, use the step-by-step guide with code samples linked here: AMToolkit identifyStream on AWS ECS
Integration prerequisites
Before any integration can take place, there are a few prerequisites that will allow for the entire system to work as intended. You will need:
- An Amazon IVS channel with streaming software setup. Refer to the Getting Started with Amazon IVS guide to learn how to create a channel and set up a live stream.
- An Audible Magic Toolkit package that contains the software needed to make identifications. You will be provided with an Audible Magic Toolkit package after you have signed up for Audible Magic Identification services.
- An Audible Magic Toolkit configuration file that informs the Audible Magic Toolkit software on how to perform according to your specifications. You will be provided with an Audible Magic Toolkit configuration file after you have signed up for Audible Magic Identification services.
To sign up for Audible Magic Identification services, please contact Audible Magic through its Contact page.
Once you have these prerequisites in place, you can proceed through the rest of this blog post to set up stream identification with Audible Magic. Let’s get started with the integration steps.
1) Create an Amazon S3 Bucket
Create an S3 bucket to hold the responses delivered by the Audible Magic toolkit.
A JSON response file is generated with information about matches found, and is saved to this S3 bucket.
2) Create an Amazon SQS queue
Create an Amazon SQS queue for messages about newly started streams. When a stream begins, it generates a “stream started event” on Amazon EventBridge, which triggers an Event Rule. A message containing stream information gets pushed to an SQS queue, where Amazon Elastic Container Service reads the message and starts scanning the stream with Audible Magic’s Toolkit.
3) Create an EventBridge event rule
Create an EventBridge Event Rule to generate a message when a new stream starts. This message will be sent to the SQS Queue.
4) Attach EventBridge event rule to SQS queue
With the SQS Queue and EventBridge Event Rule created, connect them so the EventBridge can send messages to the SQS Queue whenever an Event Rule is triggered.
5) Create toolkit wrapper application
An application needs to be written that interfaces with Audible Magic’s identifyStream binary, which is embedded within the Audible Magic Toolkit. This application will perform the following while parsing live-streamed content:
- Pull message from SQS queue
- Parse for channel-arn
- Parse for stream-id
- Retrieve stream playback URL using channel-arn
- Start scanning stream with identifyStream
- Monitor for Identification Response files
- Push Identification Response files to the S3 Bucket
Pull message from SQS queue
Messages about new streams will be pushed to the SQS queue created earlier. The Toolkit wrapper application will need to pull, parse, and delete these messages while taking the appropriate action on each.
Retrieve stream playback URL using channel ARN
After pulling the message from the queue, parse the “channel_arn” and use that to obtain the stream playback URL.
Start scanning stream with identifyStream
At this point, all necessary pieces to scan the live stream for copyrighted content using identifyStream are configured. Next, create a function that will run identifyStream using the appropriate parameters, and place the results in an accessible location.
Monitor for identification responses
Since identifyStream responds with matches in real-time during a broadcast, it’s important to monitor for results as they are generated. Identification results are saved locally; next, write a function that will monitor the directory for creation of new files. Files are saved in “/app/{stream_id}” directory.
Push Identification response files to S3 bucket
When new files (identification match results) are detected in the monitored directory, push these files to the S3 Bucket created earlier.
6) Create Docker image of the toolkit wrapper application
With the application now written, create a Docker image using Dockerfile and Docker command.
7) Upload the toolkit wrapper application Docker image to AWS Elastic Container Registry (ECR)
Create the Elastic Container Registry (ECR) repository in which the Docker image will reside.
Prior to pushing the Docker image to ECR, verify that the ECR repository can be connected via Docker using the username “AWS”. Then, tag your local Docker image with the appropriate AWS ECR location and push it to the ECR repository. Ensure the presence of the Docker image in ECR by listing the repositories with the appropriate name Ex. “am-stream-identification”.
8) Create an ECS task
Create an ECS task that defines the criteria needed to run the Docker image. Ensure the task definition contains information about: 1) the SQS Queue from which the application will read messages, 2) the S3 Bucket to which the application will save identification results, and 3) the image that the Docker container will use as its base image.
9) Create ECS service
Create an ECS service that will launch the ECS task. ECS service is named as “am-stream-identification-service”. The ‘desired count’ of this service is set to 1. This will automatically start a single instance of the am-stream-identification-task.
10) Adding email notification integration
Without having to check the S3 bucket for new JSON response files, you can set up email notifications using AWS Simple Notification Service (SNS) to email you whenever an identification match has is made. The email notification feature is optional and can be skipped.
Create a Simple Notification Service topic
The first step in setting up email notifications via SNS is to create an SNS topic. Allow S3 to publish to the SNS Topic. Once the SNS topic can receive notifications from the S3 bucket, add the push notification capability to the S3 bucket that is triggered whenever a new JSON response file is created in the identifications bucket.
Link email address to be sent notifications
In order to receive an email notification via SNS, link the email address that should receive the email notifications. Now, whenever a JSON response file is saved to the S3 bucket, an email message is sent (formatted in JSON) with information about the JSON file, including the location (key) and size.
11) Testing the Integration
With the ECS Service fully set up and running, you can test the Audible Magic identification service against a live stream. Start a stream to your Amazon IVS channel. A message should automatically be sent to the SQS queue that was set up previously. The Audible Magic identification service will scan the stream as long as it is active. Whenever the service finds a match to copyrighted content, a JSON response file with the match results will be produced and saved to the S3 bucket that was created earlier in this guide, within a folder named after the stream-id of the active stream.
Conclusion
In this article, we demonstrate how to set up a fully functioning automatic content recognition solution in just a few simple steps using Audible Magic’s ACR technology integrated with Amazon IVS. If you’re interested in validating this solution using a proof of concept, please visit https://www.audiblemagic.com/contact/.
About Audible Magic
For more than 20 years, Audible Magic has innovated B2B solutions in content identification (both music and TV/Film), music catalog fulfillment, and rights administration. Audible Magic powers billions of monthly transactions to serve customers all around the world. Its music identification services are used to help reduce the number of DMCA takedown notices and administer licenses for user generated content (UGC). Its catalog fulfillment services are a robust set of APIs that enable you to easily integrate and customize music experiences for your app. Audible Magic is a trusted intermediary and approved vendor between major platforms and rights holders (including music labels, distributors, publishers, collectives, and TV/Film studios). For more information, visit https://www.audiblemagic.com/.
Bolo Indya Partners with Audible Magic to Manage Licensed and Unlicensed Use of Music
New Delhi, 21 July, 2021: Bolo Indya, India’s first home-grown and most preferred social live-streaming platform, announced that it has partnered with Audible Magic, the leading provider of rights management and identification solutions. The partnership allows Bolo Indya to proactively manage both occurrences of licensed and unlicensed copyrighted media on their site.
Bolo Indya needed an identification service known for accuracy, ease of use and offered an extensive registry of music from around the world, including native music. Bolo Indya worked with Audible Magic to quickly deploy the scalable identification service to respond to the explosive growth in live streaming and short-form videos on their site.
Commenting on the development Varun Saxena - CEO and Founder, Bolo Indya said: “We are committed to providing industry-leading copyright solutions for user-generated content on our platform. This expanded solution will use Audible Magic’s robust technology to support the compliance and licensing of user-generated content on our platform. We are delighted to partner with Audible Magic to help us achieve our goal.”
Bolo Indya is the first of its kind indigenous live streaming platform and is off to a blazing start with a new redemption method, dual live streaming, and other features. Bolo Live, the live streaming capability by Bolo Indya has been embraced by many renowned organizations and personalities since its launch. In the past few months, the platform has witnessed some special moments on Live, including informational talks about COVID-19 by health experts, Hon’ble Prime Minister Narendra Modi talking to thousands of viewers through MannKiBaat, Ministry of AYUSH hosting International Yoga Day Live on Bolo Indya, interviews with celebrities, and record-breaking viewership.
Audible Magic provides proven solutions for platforms to protect their brand’s reputation while using music in user-generated content, streaming, radio, and downloads. Platforms can speed time to market by avoiding building the extensive systems needed to identify content, manage licenses, and ensure they meet their reporting and payment obligations.
Vance Ikezoye, President and CEO at Audible Magic: “We are pleased to help Bolo Indya to identify licensed and unlicensed use of copyrighted music proactively. The partnership leverages our strength in content identification technology, coupled with the trust we’ve built with rights holders.”
About Audible Magic
For more than 20 years, Audible Magic has innovated solutions to identify content, provide fulfillment solutions, manage rights, and monetize media. Audible Magic’s Emmy-winning automatic content recognition (ACR) technology powers billions of monthly transactions. The Silicon Valley pioneer is the trusted intermediary between major platforms and rightsholders (including labels, studios, distributors, publishers, and collectives). The company works with a wide range of platforms and rights holders, including Facebook, Twitch, SoundCloud, Dailymotion, Vimeo, NBC Universal, Universal Music Group, Sony Music Group, Warner Music Group, The Orchard, CDBaby, and DistroKid. For more information, visit audiblemagic.com .
About Bolo Indya
Bolo Indya is the fastest-growing social live streaming platform. Bolo Indya stands unique as it works towards empowering the next billion internet users of India, by allowing them to transform their social capital and virality into financial independence by unlimited earning every month. It brings to the table the amalgamation of short videos, local languages, UGC as a driver, and Passion economy as the fuel for empowering the internet users of Bharat, taking the penetration of peer-to-peer financial transactions to tier 2 and tier 3 cities of India.
The Bolo-Live streaming feature comes with an integrated real-time gamification in the form of a gifting option where creators can be rewarded by their follower base basis the content quality, density as well as engagement which can be redeemed in cash on the Bolo Indya platform.
Why is Music Such a Challenge for the Fitness App Industry?
The burgeoning Fitness App industry offers incredible workout experiences and a sense of community, but integrating music poses unique technical and licensing challenges.
The health and fitness app industry is growing leaps and bounds, with some predictions putting the market at US$ 18.7 billion by 2030. The growth is driven not only by better real-time data and equipment, but the worldwide pandemic is driving consumers to find a quality workout at home.
Combining high-quality equipment with classes is now a given for companies like Echelon, Hydrow, and CLMBR. These platforms strive to expand their offerings to their fan base with additional classes on and off the equipment - think yoga, strength training, and Pilates.
Instructors with a worldwide reach on these platforms are gaining almost cult followings. Participants accessing either a web streaming or an on-demand class expect an experience that delivers a workout and a sense of community and emotional connection.
Music: Got to have it
The underpinning of the experience is music. It sets the mood, drives the workout, and becomes part of the instructor’s brand. Class reviewers talk equally about the instructor and their choice of music-many reviewing the playlist before choosing a class.
Instructors who understand this build their class around music, whether it is an hour of EDM, 80s greats, or Miley Cyrus. A large catalog of premium music enables continued class creation and engagement. Recognizing this new vertical for music, record labels are partnering with platforms to create remixes and break new artists.
So, accessing music for these platforms is key to their business. The challenge is getting licenses and handling the overhead of administering the licenses.
Licensing: Infrastructure, Infrastructure, and More Infrastructure!
One of the first major hurdles that a platform needs to consider when approaching record labels is proof of technical competency.
Record labels and publishers want assurance that the platform can effectively manage their catalogs while also reporting and paying accurately. Unfortunately, these systems take a lot of licensing expertise, time, and technical resources to build.
The quickest way to move licensing negotiations forward is to partner with an experienced and full-service digital music management vendor before engagement. This service provider will often have the trust of the record labels, access to the masters, sub-licenses, and proven systems for:
- Receiving the music and continuous updates from the labels
- Identifying content to ensure appropriate use
- Linking the sound recording to the musical composition to clear tracks for use, and ensuring proper payment to the publisher(s)
- Reporting and Payments
Very few will provide all those functions, but those who do will make vendor management a lot simpler.
Licenses for Live Streaming and On-Demand
Generally, fitness platforms provide live-streamed classes which may or may not be stored, and they offer on-demand classes. Therefore, in consultation with a music industry expert or lawyer, these platforms will need to obtain appropriate licenses from labels, publishers, and collective management organizations (CMOs) covering the territories they will operate in.
Once they have secured licenses, the platform will have technical and service requirements to ensure only licensed tracks are used and to meet their ongoing licensing obligations as listed below:
- Establish an extensive catalog with matched sound recordings to musical compositions
- Host an approved catalog for use by instructors
- Identify licensed and unlicensed tracks using content identification technology
- Reporting, royalty administration, and payments to rights holders
It is hard enough to administer hundreds of thousands, and in some cases millions of tracks, but the difficulty increases trying to keep up with the continuous changes in catalog ownership. Platforms will need to identify tracks in their current on-demand classes and possibly replace songs when ownership has changed.
One can see that a platform hosting both live-streaming and on-demand classes will have many licensing obligations and will need a partner to help manage the process.
Let Audible Magic Help Manage Music on Your Platform
Audible Magic enables platforms to focus on growing their business by providing world-class proven music management solutions, including:
- Licensing Support - Understand what is cleared and not cleared.
- Fulfillment - Deliver cleared music to your platform now
- Content Identification - ID & track music on your platform
- Royalty Administration and Payment – Royalty reporting and payment to labels, publishers, CMOs and PROs
This blog is issued for informational purposes only and is not intended to be construed or used as general legal advice.
Audible Magic Launches AMLive™ to Protect Against Rebroadcasting Premium Content by Users of Live Streaming Platforms
Los Gatos, CA (April 3, 2018) — Audible Magic Corporation, the leader in audio and video digital content identification solutions, announced today the availability of AMLive — a new service to identify unauthorized user streams of premium content registered by rights holders. The service addresses industry needs to protect live sports, live-broadcasted music and other premium content. AMLive operates on both live streams and video-on-demand.
Using a simple dashboard or via an API to integrate into existing workflows, AMLive makes it easy for broadcasters and programmers to register live broadcast events. Once content is registered with Audible Magic, the social platforms can automatically identify and block unauthorized streams in real-time.
“Content owners can now more easily ensure their valuable content is available only from authorized streaming services.” said Vance Ikezoye, Co-founder and CEO of Audible Magic. “AMLive is well suited for the live sports industry, where piracy of live streams is a major concern. The service also addresses a growing issue where recorded music is streamed live by bedroom DJs.”
“What makes Audible Magic unique in the industry is we provide a service used and paid for by the social platforms to proactively prevent the unauthorized distribution of copyrighted content. The social platforms then make the service available to any content owner who wants to register content they own or control. AMLive enables greater cooperation between the social platforms and the content industry.”
Audible Magic is recognized for its pioneering work in audio and video digital fingerprinting, the foundation behind its content recognition technology, and is a trusted provider among both platforms and rights owners. With the launch of AMLive, Audible Magic expands its portfolio beyond video-on-demand (VOD) services.
Audible Magic will be discussing AMLive and other recent developments with content owners and OVSPs at the National Association of Broadcasters show in Las Vegas, April 9-11. Please contact [email protected] to schedule a meeting.
Additional information on AMLive can be found at http://bit.ly/ampr318