JP7398723B1 - Image generation device, prompt creation support device, program and application program - Google Patents

Abstract

The present invention provides an image generation device, a prompt creation support device, a program, and an application program that support image creation by a user. [Solution] In an illustration background image generation system in which an illustration background image generation server and a plurality of user terminals are connected via a network, the illustration background image generation server generates a UI that includes a plurality of tags on the user terminal. a UI data transmitter that transmits UI data for the user; a tag receiver that receives one or more tags selected by the user from a user terminal; a prompt creator that creates a prompt including elements corresponding to the tag; It has an image AI image generation section that generates an image based on the image, and an image output section that outputs the image. The user can easily create a prompt sentence by selecting a situation tag for a background image that matches the overall composition with the user character, perspective, story, etc. [Selection diagram] Figure 21

Description

Article 30, Paragraph 2 of the Patent Act applies Published on the company's website "REFER" and tweet site on February 10, 2020 https://refer-art. jp https://twitter. com/Refer_official/status/1623985149712175109? s=20

The present invention relates to an image generation device, a prompt generation support device, a program, and an application program.

In recent years, image generation AI systems in which AI automatically generates original images have been known (for example, see Non-Patent Documents 1 and 2). Images created by image generation AI are generated by an algorithm called the "diffusion model" installed in the system. Since users use a system that has the diffusion model installed as a trained model, they can generate various images just by inputting text, without having to understand algorithms or write program code. .

Stable Diffusion Online, Internet <URL: https://stablediffusionweb.com/> NovelAI - The AI Storyteller, Internet <URL: https://novelai.net/>

What these image generation AI systems have in common is that a high-resolution graphic image is generated by inputting text called a prompt. More specifically, the prompt input to the image generation system AI is an instruction sentence that is a combination of English words input to the "text to image" type AI separated by commas.

However, with such "text to image" type AI, the user needs to create an appropriate prompt (text) in order to output an image close to the ideal desired by the user, which is not always an easy task. isn't it. That is, all prompts input to the image generation AI must be written in English text, and there is a problem in that detailed control of the generated images is difficult. In fact, experts called prompt engineers are beginning to appear who research and create prompts to determine how to devise prompts that will allow image generation AI to create high-quality images.

On the other hand, there are illustrators who create their own original characters and post their works on illustration posting sites, etc., without relying on the image generation AI. When creating character illustrations, not only the character illustration but also the illustration background placed on the back of the character are important elements.In order to draw a background that better matches the character, for example, the overall composition, perspective ) or objectively interpreting the story, etc., which requires different skills and more time than drawing characters. For this reason, artists generally tend to devote their skills and time resources to pursuing the originality and quality of the characters, and the reality is that the background of the illustration can be, for example, a solid solid color or a simple background similar to that. there were.

The present invention has been proposed in view of the above points, and one aspect of the present invention is to support image creation by a user.

In order to solve the above problems, an image generation device according to the present invention includes a UI data transmission means for transmitting UI data for generating a UI including a plurality of tags to a user terminal, and a UI data transmission means for transmitting UI data for generating a UI including a plurality of tags to a user terminal; a tag receiving means for receiving one or more of the tags, a prompt generating means for creating a prompt including an element corresponding to the tag, an image generating means for generating an image based on the prompt, and an image generating means for outputting the image. an image output means, and the prompt creation means includes a prompt including a predetermined first prompt element, a prompt including a prompt including an element corresponding to the tag, and a predetermined predetermined prompt. A negative prompt consisting of a second prompt element is created .

Further, in order to solve the above problems, the prompt creation support device according to the present invention includes a UI data transmitting means for transmitting UI data for generating a UI including a plurality of tags to a user terminal, and a user terminal from the user terminal to a user. a tag receiving means for receiving one or more of the tags selected by the tag; and a prompt creating means for creating a prompt including an element corresponding to the tag , and the prompt creating means is configured to receive a predetermined predetermined tag. A prompt including a first prompt element, a prompt including a prompt including an element corresponding to the tag, and a negative prompt including a predetermined second prompt element are created.

According to an embodiment of the present invention, it is possible to support image creation by a user.

FIG. 1 is a diagram showing an example of a network configuration of an illustration background image generation system according to the present embodiment. It is a diagram showing an example of the hardware configuration of an illustration background image generation server according to the present embodiment. It is a diagram showing an example of the software configuration of an illustration background image generation server according to the present embodiment. FIG. 2 is a diagram showing an example of a data structure of a user DB according to the present embodiment. FIG. 2 is a diagram illustrating an example data structure of an image storage DB according to the present embodiment. It is a figure showing an example of tag table information of tag DB concerning this embodiment. It is a figure showing an example of option tag table information of tag DB concerning this embodiment. An example (part 1) of a web UI screen of a user terminal according to the present embodiment is shown. An example (part 2) of the web UI screen of the user terminal according to the present embodiment is shown. An example (part 3) of the web UI screen of the user terminal according to the present embodiment is shown. An example of a web UI screen (No. 4) of a user terminal according to the present embodiment is shown. An example (part 5) of the web UI screen of the user terminal according to the present embodiment is shown. An example of a web UI screen (Part 6) of a user terminal according to the present embodiment is shown. An example of a web UI screen (No. 7) of a user terminal according to the present embodiment is shown. An example of a web UI screen (part 8) of a user terminal according to the present embodiment is shown. An example of a web UI screen (No. 9) of a user terminal according to the present embodiment is shown. An example (No. 10) of the web UI screen of the user terminal according to the present embodiment is shown. An example of a web UI screen (No. 11) of a user terminal according to the present embodiment is shown. An example of a web UI screen (No. 12) of a user terminal according to the present embodiment is shown. An example of an illustration background image after editing according to the present embodiment is shown. FIG. 2 is a flowchart showing image generation processing by the illustration background image generation server according to the present embodiment. An example of an illustration background image with and without a "common prompt" and a "negative prompt" according to the present embodiment is shown.

Embodiments of the present invention will be described in detail with reference to the drawings.
<Network configuration>
FIG. 1 is a diagram showing an example of a network configuration of an illustration background image generation system according to this embodiment. In an illustration background image generation system 100 in FIG. 1, an illustration background image generation server 10 and a user terminal 20 are connected via a network 50.

An illustration background image generation server (hereinafter also simply referred to as an image generation server) 10 provides the user with a web UI (User Interface) screen for generating illustration background images (background content), and also provides the user with a web UI (user interface) screen for generating illustration background images (background content). This is a server device that generates an illustration background image by inputting a prompt created based on the selection operation to an AI image generation unit trained by a Diffusion Model.

The user terminal 20 is an information processing terminal device including a PC (Personal Computer), a smartphone, a tablet terminal, etc. owned by a user such as an illustrator. Through the web UI screen of the image generation server 10, the user creates a prompt for outputting an illustration background image close to the ideal desired by the user while considering the overall composition with the user character, perspective, story, etc. do. Furthermore, the user terminal 20 can generate and output an illustration background image by inputting the created prompt to the AI image generation unit.

The network 50 is a communication network including wired and wireless communication networks. The network 50 includes, for example, the Internet, a public network, WiFi (registered trademark), and the like.

(Hardware configuration)
FIG. 2 is a diagram showing an example of the hardware configuration of the illustration background image generation server according to the present embodiment. The image generation server 10 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, an HDD (Hard Disk Drive) 14, and a communication device 15.

The CPU 11 executes various programs and performs arithmetic processing. The ROM 12 stores programs necessary for startup. The RAM 13 is a work area that temporarily stores processing performed by the CPU 11 and stores data. The HDD 14 stores various data and programs. The communication device 15 communicates with other devices via the network 50.

(Software configuration)
FIG. 3 is a diagram showing an example of the software configuration of the illustration background image generation server according to the present embodiment. The image generation server 10 includes a web UI data acquisition unit 101, a web UI data transmission unit 102, a tag reception unit 103, a prompt creation unit 104, an AI image generation unit 105, an image output unit 106, and a character image acquisition unit as main functional units. 107, an image editing section 108, and a storage section 109.

The web UI data acquisition unit 101 has a function of acquiring web UI data for generating a web UI (for example, a web UI screen) including a plurality of situation tags.

The web UI data transmission unit 102 has a function of transmitting web UI data to the user terminal 20.

The tag receiving unit 103 has a function of receiving one or more tags selected by the user from the user terminal 20.

The prompt creation unit 104 has a function of creating a prompt including an element corresponding to the tag selected by the user.

The AI image generation unit 105 has a function of generating an image based on the created prompt.

The image output unit 106 has a function of outputting the generated image.

The character image acquisition unit 107 has a function of acquiring character images from the user terminal 20.

The image editing unit 108 has a function of arranging a character image on a background image.

The storage unit 109 stores a user DB 109a, an image storage DB 109b, a tag DB 109c, and a learning data DB 109d. The user DB 109a is a DB in which user information of users is registered. The image storage DB 109b is a DB that holds user character images (character content images), illustration background images (background content images), and composite content images. The tag DB 109c is a DB that holds a tag table and an option tag table. The learning data DB 109d is a huge image data set that is trained in advance using, for example, a known diffusion model, in order for the AI image generation unit 105 of the image generation server 10 to generate a content image of an illustration background.

Note that each functional unit is realized by a program executed on hardware resources such as the CPU, ROM, and RAM of the computer that constitutes the image generation server 10. These functional units may be read as "means", "module", "unit", or "circuit". In addition, each functional unit has functions similar to those of a single image generation server 10 by distributing the functions not only to a single image generation server 10 but also to a plurality of server devices and allowing them to communicate with each other via the network 50. may be realized. Further, each DB of the storage unit 109 can also be placed in an external storage device on the network 50. Further, the computer program and the application program may be stored in a computer readable storage medium.

(database)
FIG. 4 is a diagram showing an example of the data structure of the user DB according to this embodiment. The user DB 109a is a DB in which user information of users is registered. The user DB 109a according to this embodiment has data items such as "user ID", "user name", "password", and "email address", for example.

"User ID" is a unique identifier numbered for each user. The “user ID” and “password” are the ID and password for the user to access and log in to the web UI screen of the image generation server 10.

FIG. 5 is a diagram showing an example of the data structure of the image storage DB according to this embodiment. Images stored in the image storage DB 109b include user character images (character content images), illustration background images (background content images), and composite content images. The user character image is a character content image (transparent image) uploaded by the user via the web UI screen. The illustration background image is an illustration background content image generated by the image generation unit of the image generation server 10. The composite content image is a content image in which a user character image is superimposed and composited on an illustration background image.

The image storage DB (user character image) shown in FIG. 5A has data items such as "user ID", "upload date and time", and "image data", for example. "User ID" is a user ID corresponding to the user DB 109a. The "upload date and time" is the date and time when the user character image was uploaded by the user. "Image data" is data of a user character image uploaded by the user.

The image storage DB (illustration background image) shown in FIG. 5B has data items such as "user ID", "creation date and time", "image data", "tag", and "prompt". "User ID" is a user ID corresponding to the user DB 109a. The “creation date and time” is the date and time when the illustration background image was generated by the image generation unit of the image generation server 10. “Image data” is data of an illustration background image generated by the AI image generation unit 105 of the image generation server 10. "Tag" is a tag selected by the user from the web IU. “Prompt” is a prompt text input to the AI image generation unit 105 in order to generate the illustration background image.

The image storage DB (synthesized content image) shown in FIG. 5(c) has data items such as "user ID", "creation date and time", and "image data", for example. "User ID" is a user ID corresponding to the user DB 109a. The “creation date and time” is the date and time when the composite content image was generated in response to a user operation. “Image data” is data of a composite content image generated in response to user operations.

FIG. 6 is a diagram showing an example of tag table information of the tag DB according to the present embodiment. The tag table in the tag DB 109c is a table in which tags associated with situations in the illustration background (also referred to as situation tags) are associated with prompts expressing the tags. The tags are displayed on the web UI screen so that the user can select them. The tag table according to the present embodiment includes data items such as "tag category", "tag", "prompt element", "option tag category", "default prompt element", and "custom base prompt".

The "tag category" is a first layer (top layer) semantic concept tag that is displayed to the user on the web UI screen. A "tag" is a second-layer tag that belongs to any "tag category" and is displayed to the user on the web UI screen. For example, the "tag categories" on the first layer include "school," "building," "ruins," "season," "outdoors," and "nature" category tags. For example, the "school" tag category on the first layer has category tags on the second layer. There are ``pools'', ``schoolyards'', ``corridors'', ``libraries'', ``classrooms'', etc.

A "prompt element" is a constituent element of a prompt to be created, and is a directive word in English for converting a tag selected (specified) by a user operation on a web UI screen into a prompt expression format. To generate an illustration background image, the image generation server 10's AI uses a prompt (instruction text consisting of a combination of English words separated by commas) that includes prompt elements converted in the tag table, rather than the tag itself on the web UI screen. The image is input to the image generation unit 105.

The "option tag category" is an optional tag category that is displayed to the user as an option on the web UI screen. Since it is an option, the presence or absence of an "option tag category" and the content of the tag category vary depending on the tag. For example, tags that mean outdoor situations such as "pool" and "schoolyard" have an optional tag category with the concept of "time of day," whereas tags that mean indoor situations such as "corridor," "library," and "classroom" have an optional tag category with the concept of "time of day." does not have an option tag category with the concept of "time zone".

The "default prompt element" is set as the initial value when there are multiple "option tag categories" and the user does not select (specify) any option tag category from among the multiple option tag categories. A prompt element for the specified option tag.

A "custom base prompt" is a prompt that is specified in place of a common prompt when creating a prompt. For example, if the tag for which a custom base prompt is specified is "corridor", instead of applying "common prompt", replace it with the custom base prompt "(((masterpiece))),(((best quality) )), (((high quality))), (((scenery))), (((no humans))), (((absurdres)))". According to the applicant's research, when a custom base prompt is applied to a tag for which a custom base prompt is defined on the tag table, the image quality of the generated image can be expected to be improved by applying a custom base prompt instead of applying a common prompt.

FIG. 7 is a diagram showing an example of option tag table information of the tag DB according to the present embodiment. The option tag table in the tag DB 109c is a table in which option tags (also referred to as situation option tags) associated with situations in the illustration background are associated with prompts expressing the option tags.

The option tag table according to this embodiment includes data items such as "option tag category," "option tag," and "prompt element." The "option tag category" is an option tag category corresponding to the "option tag category" in the tag table information shown in FIG. The "option tag" is an option tag that belongs to any "option tag category." A "prompt element" is a directive word in English for converting an option tag into a prompt expression format when an option tag is selected (specified) by a user operation from the web UI screen.

<Prompt>
The prompt is an instruction sentence that is a combination of English words to be input into the text-to-image AI, separated by commas. In this embodiment, by converting tags selected (specified) from the web UI screen according to the user's desired background image situation into a prompt expression format based on the tag table and option tag table, the AI image A prompt to be input to the generation unit 105 is created.

Generally, "text to image" type AI receives two inputs: a "prompt" and a "negative prompt." Similarly, the prompt input to the AI image generation unit 105 is a "prompt" and a "negative prompt." be. Furthermore, in this embodiment, the former "prompt" is composed of a "common prompt" and a "user prompt." The common prompt is a fixed prompt that is always included in all "prompts" and is composed of predetermined prompt elements defined in advance. The user prompt is a prompt consisting of prompt elements according to the tag selected by the user. The latter "negative prompt" is a prompt for instructing what is not desired to be displayed in the image, and is composed of predetermined prompt elements defined in advance.

Hereinafter, a "common prompt", a "user prompt", and a "negative prompt" will be exemplified. Brackets () have the indicating effect of emphasizing the prompt element within the brackets.
・Example of common prompt "(((masterpiece))),(((best quality))), (((high quality))), (((scenery))), (((no humans))), ( ((absurdres))),light_particles,finely detail”
・Example of user prompt (for "pool" and "morning" tags)
"pool,school,outdoors,((poolside)),sunlight,blue_sky"
・Example of negative prompt "worst quality, low quality, medium quality, deleted, lowres, comic, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, jpeg artifacts, signature, watermark, username , blurry,cropped,ugly,outside_border"

<Image generation example>
Next, an example of image generation generated by user operation from the web UI screen displayed on the screen of the user terminal 20 will be shown.
(Generation of illustration background image)
FIG. 8 shows an example (part 1) of the web UI screen of the user terminal according to the present embodiment. The user uses a predetermined application program (for example, a general-purpose web browser or a dedicated application program) on the user terminal 20 to access and log in to the illustration background image generation server 10, thereby displaying a web UI screen on the screen of the user terminal 20. indicate.

The web UI screen includes, in a generation mode 201 for generating images, a "Gallery" 202 that shows a list of generated and saved illustration background images, a generated image display area 203 that displays generated images, and a generated image display area 203 that displays images generated by the user. It has a tag selection section 204 for selecting an illustration background situation. The user selects a tag (keyword) that matches the illustration background image that the user wants to generate from among the tags in the tag selection section 204.

FIG. 9 shows an example (part 2) of the web UI screen of the user terminal according to the present embodiment. In the tag selection section 204 of the web UI screen, the user selects, for example, the "school" tag category from among the multiple "tag categories" 205 such as "school," "building," "ruin," "season," "outdoor," and "nature." Then, "tags" 206 belonging to the "school" tag category are displayed. As shown in FIG. 9, the "tags" 206 that belong to the "school" tag category include, for example, "pool," "schoolyard," "corridor," "library," and "classroom" (tags in the tag DB 109c shown in FIG. 6). (see table).

FIG. 10 shows an example (part 3) of the web UI screen of the user terminal according to the present embodiment. For example, when the user selects a "pool" tag belonging to the "school" tag category, the selected tag is displayed in the tag selection field 207.

Further, if an "option tag category" exists in the selected tag, "option tag category" 208 and "option tag" 209 corresponding to the tag are displayed. For example, when a "pool" tag is selected, the "time zone" option tag category, "morning" option tag, and "evening to night" option tag corresponding to the "pool" tag are displayed (as shown in Figure 7). (See option tag table in tag DB 109c). Note that if the user does not select the "option tag" 209, it is assumed that the "default prompt element" (the prompt element corresponding to the "morning" option tag) has been selected.

FIG. 11 shows an example (part 4) of the web UI screen of the user terminal according to the present embodiment. When the user completes tag selection and presses the "background image generation" button 210, the generated illustration background image 203a is displayed on the generated image display section 203. The illustration background image 203a is an image generated by inputting a prompt created based on the tag displayed in the tag selection field 207. An example of a prompt created based on the selected "pool" (and "morning") tags is shown below. Note that the prompt element for "pool" is "pool, school, outdoors, ((poolside))". The default prompt element for "morning" is "sunlight,blue_sky".
Prompts (common prompt + user prompt): "(((masterpiece))),(((best quality))), (((high quality))), (((scenery))), (((no humans) )), (((absurdres))),light_particles,finely detail,pool,school,outdoors,((poolside)),sunlight,blue_sky”
Negative prompt: “worst quality, low quality, medium quality, deleted, lowres, comic, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, jpeg artifacts, signature, watermark, username, blurry ,cropped,ugly,outside_border”

Among these, the "user input" consisting of prompt elements corresponding to the selected "pool" tag is the "pool, school, outdoors, ((poolside)), sunlight, blue_sky" portion.

Note that the created prompt result may be explicitly displayed to the user, for example, on the web UI screen, or may be hidden. The reason why the prompt may be hidden is that the user only needs to select the tag and does not need to be aware of the created prompt itself.

FIG. 12 shows an example (part 5) of the web UI screen of the user terminal according to the present embodiment. FIG. 12 shows a generated illustration background image 203a when the user selects the "evening to night" option tag 209 and presses the "background image generation" button 210 on the web UI screen shown in FIG. show. Comparing the illustration background image 203a in FIG. 12 with the illustration background image 203a in FIG. ” The images are from different situations. Below is an example of a prompt created based on the selected "pool" and "evening to night" tags. Note that the prompt element for "pool" is "pool, school, outdoors, ((poolside))". The default prompt element for "evening-night" is "night, night sky."

Prompts (common prompt + user prompt): "(((masterpiece))),(((best quality))), (((high quality))), (((scenery))), (((no humans) )), (((absurdres))),light_particles,finely detail,pool,school,outdoors,((poolside)),night,night sky”
Negative prompt: “worst quality, low quality, medium quality, deleted, lowres, comic, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, jpeg artifacts, signature, watermark, username, blurry ,cropped,ugly,outside_border”

Among these, the "user input" consisting of prompt elements corresponding to the selected tags "pool" and "evening to night" is "pool, school, outdoors, ((poolside)), night, night sky". be.

When the user presses the "image save" button 211, the image generated in the generated image display section 203 is saved in the image storage DB 109b (FIG. 5(b)). The saved images are also displayed in a “gallery” 202. Images in the "Gallery" 202 can also be downloaded to the user terminal 20.

FIG. 13 shows an example (part 6) of the web UI screen of the user terminal according to the present embodiment. The web UI screen shown in FIG. 13 shows an illustration background image 203a that is generated when the user selects the "nature", "winter", and "morning" tags and presses the "background image generation" button 210. . The generated illustration background images are images of situations such as "nature," "winter," and "morning."

(Editing illustration background image)
FIG. 14 shows an example (No. 7) of the web UI screen of the user terminal according to the present embodiment. In the editing mode 221 for editing an image, the web UI screen serves as an operation unit for editing the background image, "free movement" 222 for moving the illustration background image and the user character image, and enlarging the user character image. "Character enlargement" 223 for scaling down; "Background enlargement" 224 for scaling the illustration background image; "Background blur" 225 for blurring the illustration background image; "Saturation" 226 for increasing/lowering the saturation of the illustration background image. has. The web UI screen also includes an "upload character image" 227 as an operation section for uploading a user character image.

FIG. 15 shows an example (No. 8) of the web UI screen of the user terminal according to the present embodiment. As shown in FIG. 15(a), when the user operates "Upload character image" 227, the user character image is uploaded and superimposed on the illustration background image 203a of the generated image display section 203. The user character image 203b is displayed. Further, the uploaded user character image is stored in the image storage DB 109b (FIG. 5(a)). Further, as shown in FIG. 15(b), when the user operates "enlarge character" 223, the user character image 203b above the illustration background image on the generated image display section 203 can be enlarged or reduced.

Furthermore, when the user operates "delete character" 228, the uploaded user character image 203b can be deleted from the illustration background image of the generated image display section 203. Further, when the user operates the "background save" 229, only the illustration background image 203a of the generated image display section 203 can be saved in the image save DB 109b. On the other hand, when the user operates the "image save" button 211, the entire image displayed on the generated image display section 203, that is, a composite image in which the user character image 203b is superimposed on the illustration background image 203a, is stored in the image storage DB 109b. It can be saved (Figure 5(c)). The saved images are displayed in "Gallery" 202.

FIG. 16 shows an example (No. 9) of the web UI screen of the user terminal according to the present embodiment. As shown in FIG. 16A, when the user operates "background enlargement" 224, the illustration background image 203a on the generated image display section 203 can be enlarged or reduced. Further, as shown in FIG. 16(b), when the user operates "background blur" 225, the illustration background image 203a on the generated image display section 203 can be blurred.

FIG. 17 shows an example (No. 10) of the web UI screen of the user terminal according to the present embodiment. As shown in FIG. 17, when the user operates "saturation" 226, the saturation of the illustration background image 203a in the generated image display section 203 can be increased or decreased.

FIG. 18 shows an example (No. 11) of the web UI screen of the user terminal according to the present embodiment. As shown in FIG. 18A, when the user operates "Character movement" 222a, the user character image 203b on the generated image display section 203 can be moved. Further, as shown in FIG. 18(b), when the user operates "move background" 222b, the illustration background image 203a on the generated image display section 203 can be moved.

FIG. 19 shows an example (No. 12) of the web UI screen of the user terminal according to the present embodiment. As shown in FIG. 19(a), when the user selects one illustration background image from the "Gallery" 202, the illustration background image 203a in the generated image display section 203 is changed to the selected illustration background image 203a. can be changed to .

FIG. 20 shows an example of an illustration background image after editing according to this embodiment. In the editing mode 221 (not shown) for editing images, the web UI screen is more similar to the illustration background image 203a of the generated image display section 203, for example, to the scenery that a fish would see when looking up at the sky from underwater. It is possible to edit the image to create a fisheye image taken from an angle, an exaggerated image taken from an angle where the subject is taken from directly below, or an overhead image taken from an angle taken from directly above the subject.

<Information processing>
FIG. 21 is a flowchart showing image generation processing by the illustration background image generation server according to the present embodiment. When the CPU 11 reads and executes a program that can implement the flowchart shown in FIG. 21, steps (hereinafter referred to as "S") executed by each functional unit can be implemented.

S1: When the web UI data acquisition unit 101 receives an access/login request from the user terminal 20, it acquires web UI screen data provided in advance for the user terminal, and the web UI data transmission unit 102 receives the access/login request from the user terminal 20. Send the acquired web UI screen data to. Upon receiving the web UI screen data, the user terminal 20 displays a web UI screen based on the web UI screen data on the screen via a predetermined application program (for example, a general-purpose web browser or a dedicated application program).

S2: The tag receiving unit 103 receives the tag selected by the user from the user terminal 20 via the web UI screen. The reception timing is, for example, the timing when the user completes tag selection on the web UI screen and presses the "background image generation" button 210.

S3: The prompt creation unit 104 refers to the tag table and option tag table in the tag DB 109c, and creates a prompt based on the tag acquired in S2.

Specifically, the prompt creation unit 104 first refers to the tag table and option tag table in the tag DB 109c, and acquires the prompt element corresponding to the tag acquired in S2. If the tag selected by the user is the "pool" tag, the prompt element "pool,school,outdoors,((poolside))" corresponding to the "pool" tag is obtained. Furthermore, if the option tag category "time zone" is not selected, the default prompt element "sunlight, blue_sky" corresponding to the "morning" tag is acquired. Then, a "user prompt" (A2) consisting of prompt elements corresponding to the tag acquired in S2 is created. pool,school,outdoors,((poolside)),sunlight,blue_sky

Next, the prompt creation unit 104 obtains a predetermined “common prompt” (A2) stored in a predetermined memory area (not shown) of the storage unit 109 in advance.
(((masterpiece))),(((best quality))), (((high quality))), (((scenery))), (((no humans))), (((absurdres))) ,light_particles,finely detail

Then, the prompt creation unit 104 creates a "prompt" (A1+A2) including the acquired "common prompt" (A1) and "user prompt" (A2).

The prompt creation unit 104 also acquires a predetermined “negative prompt” (B) stored in a predetermined memory area (not shown) of the storage unit 109 in advance. Then, a "prompt" (A1+A2) and a "negative prompt" (B) are created.

S4: The prompt creation unit 104 inputs the prompt created in S3 to the AI image generation unit 105. Specifically, the prompt generation unit 104 inputs the above-mentioned “prompt” (A1+A2) and “negative prompt” (B) to the AI image generation unit 105.

S5: The AI image generation unit 105 executes an illustration background image generation process based on the prompt input in S4.

S6: The image output unit 106 outputs the illustration background image generated in S5. The storage unit 109 also stores the illustration background image in the image storage DB (illustration background image) 109b in association with the "user ID" of the user who has accessed and logged in. Further, in the image storage DB (illustration background image) 109b, the tag received in S2 and the prompt input in S4 are stored in the "tag" and "prompt" items, respectively (see FIG. 5(b)).

Note that the image storage DB (illustration background image) 109b shown in FIG. It is possible to read out "image data", "tag", and "prompt" as a set. By selecting a new tag or option tag while referring to the set of "image data", "tag" and "prompt" generated in the past, the newly generated illustration background image and the past You can try generating illustration background images using various tag patterns while comparing the results with the illustration background images generated in . In addition, by referring to "image data" and "prompts", it is possible to learn prompt creation skills and to directly edit the text of "prompts" created in the past.

Note that, in S4, when the prompt creation unit 104 inputs the created prompt to the AI image generation unit 105, a random value called a seed is given to the AI image generation unit 105 as an input along with the prompt. Seeds are used to add randomness to the generated images. The number of digits in the seed random number is very large and changes each time, so even if the input prompt is the same each time, it is highly unlikely that the illustration background image generated and output by S5 will be the same illustration background image. Low (on the contrary, if you input a fixed seed value and a constant prompt, the generated image will be non-random and the same illustration background image will be generated every time). Therefore, even if the user selects the same "tag" 206 or the same "option tag" 209, different illustration background images can be generated and output. Therefore, the user generates a plurality of illustration background images while trying combination variations of the same "tag" or different "tags", and selects the desired generated one from the "gallery" 202. You can select illustration background images.

(“Common Prompt” and “Negative Prompt”)
As described above, the "common prompt" and the "negative prompt" are constant prompts that are composed of predefined predetermined prompt elements and have predetermined directive content. That is, in this embodiment, a "prompt" includes a "common prompt" consisting of a predetermined prompt element, a "user prompt" consisting of a prompt element according to a tag selected by the user, and a "prompt" consisting of a predetermined prompt element. A negative prompt is created.

The "common prompt" and the "negative prompt" are combination instructions of prompt elements that improve the overall image quality of the illustration background found in the applicant's research, and the prompt creation unit 104 according to this embodiment Since prompts are automatically created that incorporate "common prompts" and "negative prompts," users do not have to be aware of tags or prompt elements related to image quality improvement, and can simply select tags according to the situation from the web UI screen. As a result, a high-quality image (for example, an illustration background image) can be obtained.

FIG. 22 shows an example of an illustration background image with and without a "common prompt" and a "negative prompt" according to this embodiment. FIG. 22(a) is an illustration background image generated when the user selects the "grassland" tag. The input prompts include a "common prompt" and a "negative prompt." On the other hand, FIG. 22(b) is an illustration background image generated when the user selects the "grassland" tag. The input prompts have no "common prompt" and no "negative prompt." (a) is a background image with depth, whereas (b) is a flat background image, and the background is too close to make out anything. Usually, a background image with depth provides a composition suitable for arranging characters. If the background image is flat, there is little difference from, for example, a plain background of a single color or something similar. One of the "common prompts" and "negative prompts" includes a prompt element for generating a background with such depth in advance.

Further, FIG. 22(c) is an illustration background image generated when the user selects the "schoolyard" tag. The input prompts include a "common prompt" and a "negative prompt." On the other hand, FIG. 22(d) is an illustration background image generated when the user selects the "schoolyard" tag. The input prompts have no "common prompt" and no "negative prompt." (c) is a background image without a portrait, whereas (d) has a portrait appearing in the background. One of the "common prompts" and "negative prompts" according to this embodiment includes a prompt element that gives an instruction to draw a background. If this is not present, the AI has not received instructions to draw the background, and may draw a portrait that is not suitable as an illustration background image.

<Summary>
As described above, according to the illustration background image generation system 100 according to the present embodiment, the user can easily create a prompt by selecting and operating the tag indicating the situation of the background image from the web UI screen. . That is, the user can easily create a prompt sentence by selecting a situation tag for a background image that matches the overall composition of the user character, perspective, or story.

Furthermore, the user can generate an illustration background image by inputting a created prompt sentence. Thereby, the user can complete his or her own character illustration (composite content image) by placing the user character image on the illustration background image of the situation that the user imagines. Furthermore, by reducing the time and effort required to create prompt sentences, users (illustrators, etc.) can allocate their skills and time resources to pursue character originality and quality.

That is, according to the present embodiment, in one aspect, when the illustration background image is generated by AI, prompt input by the user is supported, and furthermore, the user is supported in creating the illustration background image and completing his/her own character illustration (synthesized content image). It is possible to do so.

Although the present invention has been described with reference to specific examples according to preferred embodiments of the present invention, it is to be understood that these specific examples may be modified without departing from the broad spirit and scope of the present invention as defined in the claims. Obviously, various modifications and changes may be made to the examples. In other words, the invention is not to be construed as limited by the details of the specific examples and the accompanying drawings.

Further, each function of the image generation server 10 according to the present embodiment can be incorporated into an application program installed on the user terminal 20. This allows image generation only on the user terminal 20 side without communicating with the image generation server 10 via the network 50. That is, for example, one aspect of the application program on the user terminal 20 is
to the computer,
A display means (for example, the UI screen in FIG. 8) that displays a UI including a plurality of tags on a display device;
Tag selection means (for example, tag selection unit 204 in FIG. 8) that allows a user to select one or more of the tags;
a prompt creation means for creating a prompt including an element corresponding to the selected tag (corresponding to the function of the prompt creation unit 104 of the image generation server 10);
an image generation unit (corresponding to the function of the AI image generation unit 105 included in the image generation server 10) that generates an image based on the prompt;
an image output means for outputting the image (corresponding to the function of the image output unit 106 of the image generation server 10;
This is an application program for making the system function.

10 Illustration background image generation server 20 User terminal 50 Network 100 Illustration background image generation system 101 Web UI data acquisition unit 102 Web UI data transmission unit 103 Tag reception unit 104 Prompt creation unit 105 AI image generation unit 106 Image output unit 107 Character image acquisition Section 108 Image editing section 109 Storage section

Claims (7)

UI data transmission means for transmitting UI data for generating a UI including a plurality of tags to a user terminal;
tag receiving means for receiving one or more of the tags selected by the user from the user terminal;
prompt creation means for creating a prompt including an element corresponding to the tag;
image generation means for generating an image based on the prompt;
image output means for outputting the image;
has
The prompt creation means includes:
a prompt comprising a predefined predetermined first prompt element; and a prompt comprising an element corresponding to the tag;
a negative prompt consisting of predefined predetermined second prompt elements;
to create,
An image generation device characterized by: The image is a background image,
The tag is a tag indicating a situation of the background image,
Character image acquisition means for acquiring a character image from the user terminal;
image editing means for arranging a character image on the background image;
The image generation device according to claim 1, characterized in that the image generation device has: storage means for storing the image in association with the tag and the prompt;
the image output means outputs the image, the tag and the prompt associated with the image;
The image generation device according to claim 1, characterized in that: to the computer,
UI data transmission means for transmitting UI data for generating a UI including a plurality of tags to a user terminal;
tag receiving means for receiving one or more of the tags selected by the user from the user terminal;
prompt creation means for creating a prompt including an element corresponding to the tag;
image generation means for generating an image based on the prompt;
image output means for outputting the image;
and make it work ,
The prompt creation means includes:
a prompt comprising a predefined predetermined first prompt element; and a prompt comprising an element corresponding to the tag;
a negative prompt consisting of predefined predetermined second prompt elements;
create,
program. UI data transmission means for transmitting UI data for generating a UI including a plurality of tags to a user terminal;
tag acquisition means for acquiring one or more of the tags selected by the user from the user terminal;
prompt creation means for creating a prompt corresponding to the tag;
has
The prompt creation means includes:
a prompt comprising a predefined predetermined first prompt element; and a prompt comprising an element corresponding to the tag;
a negative prompt consisting of predefined predetermined second prompt elements;
to create,
A prompt creation support device characterized by: to the computer,
UI data transmission means for transmitting UI data for generating a UI including a plurality of tags to a user terminal;
tag acquisition means for acquiring one or more of the tags selected by the user from the user terminal;
prompt creation means for creating a prompt corresponding to the tag;
and make it work ,
The prompt creation means includes:
a prompt comprising a predefined predetermined first prompt element; and a prompt comprising an element corresponding to the tag;
a negative prompt consisting of predefined predetermined second prompt elements;
create,
program. to the computer,
Display means for displaying a UI including a plurality of tags on a display device;
tag selection means for allowing a user to select one or more of the tags;
prompt creation means for creating a prompt including an element corresponding to the selected tag;
image generation means for generating an image based on the prompt;
image output means for outputting the image;
and make it work ,
The prompt creation means includes:
a prompt comprising a predefined predetermined first prompt element; and a prompt comprising an element corresponding to the tag;
a negative prompt consisting of predefined predetermined second prompt elements;
create,
application program.