Meta Introduces Movie Gen AI Model

By Katie Paul

NEW YORK (Reuters) – Facebook owner Meta (NASDAQ:META) announced on Friday it created a new AI model named Movie Gen that generates realistic video and audio clips based on user prompts, claiming it competes with leading media generation startups such as OpenAI and ElevenLabs.

Samples of Movie Gen’s output included videos of animals swimming and surfing, as well as videos featuring people’s real photos depicting them engaged in activities like painting a canvas.

Movie Gen can also create background music and sound effects that sync with the video’s content. For example, one video showed the tool adding pom-poms to a runner’s hands in the desert, while another transformed a dry parking lot into one filled with splashing water.

Meta stated that videos generated can be up to 16 seconds long and audio clips can extend to 45 seconds. Blind test results indicated that Movie Gen outperformed offerings from startups like Runway, OpenAI, ElevenLabs, and Kling.

This announcement arrives as Hollywood grapples with the potential of generative AI video technology; earlier this year, Microsoft-backed OpenAI showcased its product Sora, which creates feature film-like videos from text prompts.

While technologists in the entertainment industry are excited about these tools for accelerating the filmmaking process, concerns remain regarding the use of potentially copyrighted content without permission. Additionally, lawmakers have raised alarms over the potential misuse of AI-generated deepfakes in elections across multiple countries, including the U.S., Pakistan, India, and Indonesia.

Meta spokespersons mentioned that the company is unlikely to release Movie Gen for open use by developers like its Llama series of large-language models, considering the risks involved for each model. They chose not to comment specifically on Movie Gen’s assessment.

Instead, Meta aims to collaborate directly with the entertainment industry and content creators to explore applications for Movie Gen, with plans to integrate it into Meta’s products next year.

According to Meta’s blog post and a research paper, the AI model was built using a mix of licensed and publicly available datasets.

OpenAI has also been in talks with Hollywood executives this year regarding potential partnerships with Sora, but no contracts have emerged so far. Concerns rose earlier in May when actress Scarlett Johansson accused OpenAI of mimicking her voice without permission through its chatbot.

Meanwhile, Lions Gate Entertainment, which produced The Hunger Games and Twilight, announced in September access granting to AI startup Runway for its film and TV library to train its models, allowing filmmakers to utilize these tools to enhance their work.