A new patent filed and granted to Blizzard Entertainment establishes a process to generate A.I. art for assets in their video games, with the aim of saving time and effort to meet deadlines.

Current A.I. (artificial intelligence) technology is actually more of an algorithm of averages rather than actually generative, as erroneously being identified as. However, the use of applications such as LLMs (large language models) like ChatGPT and image generators such as Midjourney, has skewed the public and business perception of the primitive A.I., regardless of potential ethical concerns.

And while there are potential and real issues being highlighted with the prevalence of unregulated A.I., the potential for its use in video game development and production have already been explored, such as the recent mod for NPC dialogues in Skyrim.

The patent is for "Machine learning-based 2D structured image generation". According to the abstract of this A.I. image generation Blizzard patent found at Justia.com:

Techniques are described for a multiple-phase process that uses machine learning (ML) models to produce a texturized version of an input image. During a first phase, using a pix2pix-based ML model, an automatically-generated image that depicts structured texture is generated based on an input image that visually identifies a plurality of image areas for the structured texture. During a second phase, a neural style transfer-based ML model is used to apply the style of a style image (e.g., a target image from the training dataset of the pix2pix-based ML model) to the texture image generated at the first phase (the content image) to produce a modified texture image. According to an embodiment, during a third phase, the generated texture image produced at the first phase and the modified texture image produced at the second phase are combined to produce a structured texture image with a moderated amount of detail.

While this is a very technical and complicated description, what it boils down to is using an existing image for the A.I.’s machine learning to be trained on. It will then emulate the style and generate, via the described process, a "new" image. Most A.I. generators today such as Stable Diffusion, Midjourney, Dall-E, and others follow this fundamental process, albeit on a larger scale by using billions of sourced datasets through a diffusion and denoising system.

Going further into the details of Blizzard Entertainment’s patent, the reasoning for the development can be found in the background section:

Art-based products, such as video games, animated movies, and other products, require large quantities of high-quality artwork in order to be marketable. For example, art-based products that are in a hand-painted style include particularly high-quality artwork, which require highly-skilled and trained artists a significant amount of time to produce. As an art-based product becomes more intricate, the scope of the required artwork rapidly increases. This problem of scope is exacerbated when it comes to interactive media, such as video games, where high-quality artwork is required to flesh out interactive digital worlds, and the quality of artwork is directly related to marketability of the interactive media-based product. Many times, these art-based products are subject to release deadlines, and it can be difficult for a team of artists to produce all of the artwork required to meet such deadlines.

The gist of it is that this patented system will theoretically save time and effort in generating common textures for game assets. But there are potential issues regarding this use of A.I. generated imagery.

The U.S. Copyright Office has already made it clear that images generated by A.I. cannot be protected by copyrights without substantive human authorship involved. Another major issue is that the main bulk of datasets used by A.I. image generators are from proprietary content from artists, designers, photographers, and other creative without consent nor compensation. There are already major lawsuits filed against A.I. image generators concerning this very issue.

The second issue mentioned above might be circumvented by Blizzard if the only datasets the company uses in the process are all Blizzard’s own art and graphics. However, the current technology A.I. art generators only function as well as they do due to the benefit of having access to billions of sources. Anything less would potentially generate very low quality output from the diffusion and denoising process.

Will we be seeing the use of this technology in upcoming Blizzard releases, such as World of Warcraft expansions, Hearthstone sets, or this year’s Diablo IV? Possibly, but does Blizzard have enough proprietary art and images to accomplish an ethical output for their image generation of game texture assets with sufficient quality? Furthermore, the Federal Trade Commission (FTC) and other agencies have mandated guidance on A.I. powered products and services. With the recent issues concerning parent company Activision Blizzard (NASDAQ: ATVI) and the Microsoft buyout, the use of potentially controversial A.I. in their video games may potentially be ill-timed.

May 5 update

Blizzard President Mikey Ybarra responded to the news report stating:

Blizzard will always strive to maintain Blizzard quality. You’re trying to associate recent AI advances (generative AI) to something completely unrelated. Our approach at Blizzard is to use machine learning and AI in ways that are additive, empathic, and allow our talented teams to spend more time on the highest quality creative thinking and tasks.

Blizzard will always strive to maintain Blizzard quality. You’re trying to associate recent AI advances (generative AI) to something completely unrelated. @brendensewell says it well here: https://t.co/FXA7WKz2XL. Our approach at Blizzard is to use machine learning and AI in ways… https://t.co/GljUqtkwEb

— Mike Ybarra (@Qwik) May 5, 2023

Furthermore, Ybarra references the response of Brenden Sewell, Blizzard Design Manager:

I suspect a lot of this would be cleared up had the patent the article was about actually been linked in the article. This isn't AI art. The ethics considerations of this are about the same as the ethics of a photoshop filter. It's a tool to save time creating variant textures.

I suspect a lot of this would be cleared up had the patent the article was about actually been linked in the article. This isn't AI art. The ethics considerations of this are about the same as the ethics of a photoshop filter. It's a tool to save time creating variant textures. pic.twitter.com/jXd5lgXvsk

— Brenden Sewell (@BrendenSewell) May 4, 2023

However, it should be noted that the links to the patent were indeed provided and the relevant text quoted directly in the report to avoid such confusion.

Blizzard Entertainment’s Diablo IV is expected to be released on June 6, 2023, with a pre-purchase available right now.