JP7452591B1 - Image generation system and image generation method - Google Patents

Abstract

[Problem] To increase the added value of a photographed image by processing a person other than the photographed subject while maintaining the atmosphere of the photographed environment.
An image generation system includes a receiving unit that receives a photographed image from a photographing device, a discriminating unit that discriminates a plurality of people in the photographed image into a main subject and a sub-person, and a sub-person in the photographed image. and a replacement unit that replaces the image with a fictitious person image or a character image. The image forming apparatus may further include a creation unit that creates the fictitious character image corresponding to the sub-person. Alternatively, the image forming apparatus may further include a storage section that stores model data of a character, and a creation section that creates the character image corresponding to the sub-person using the model data.
[Selection diagram] Figure 1

Description

The present invention relates to an image generation system and an image generation method.

At tourist spots and leisure facilities, services are implemented in which photographers take pictures of customers (tourists, visitors) or automatically take pictures using sensors, and then sell prints and image data of the captured images.

If a third party other than the customer appears in the photographed image, it cannot be sold as is due to issues such as portrait rights, so processing is performed to blur or make the third party transparent (background composition). may be performed.

However, since images that include blurred people can reduce their value as commemorative photos, the shooting locations have been limited to locations such as in front of walls or panels where third parties cannot be seen. In addition, when processing to make a third party transparent, even if there are actually many people around and the surrounding area is lively, the transparency process will result in a situation where there are no people around at all, making it difficult to maintain the atmosphere of the shooting environment. could not.

Patent No. 6419749

An object of the present invention is to provide an image generation system and an image generation method that can enhance the added value of a photographed image by processing a person other than the photographed object while maintaining the atmosphere of the photographing environment.

The image generation system of the present invention includes a receiving unit that receives a photographed image from a photographing device, a discriminating unit that discriminates a plurality of people in the photographed image into a main subject and a sub-person, and a discrimination unit that discriminates a plurality of people in the photographed image into a main subject and a sub-person. , and a replacement section for replacing the image with a fictitious person image or character image.

The image generation method of the present invention includes a step of receiving a photographed image from a photographing device, a step of discriminating a plurality of people in the photographed image into a main subject and a sub-person, and a step of classifying the sub-person in the photographed image into a fictitious one. The method includes a step of replacing the image with a person image or a character image.

The image generation system of the present invention includes a receiving unit that receives a photographed image from a photographing device, a discriminating unit that extracts all the people in the photographed image and determines them as sub-persons, and a determination unit that extracts all the people in the photographed image and distinguishes them as sub-persons, and determines whether the sub-persons are fictitious person images or A replacement section for replacing the character image with the character image is provided.

The image generation method of the present invention includes a step of receiving a photographed image from a photographing device, a step of extracting all the people in the photographed image and determining them as sub-persons, and converting the sub-persons into fictitious person images or character images. and a step of replacing it with.

According to the present invention, it is possible to enhance the added value of a photographed image by processing a person other than the photographed object while maintaining the atmosphere of the photographing environment.

1 is a schematic configuration diagram of an image generation system according to an embodiment of the present invention. FIG. 7 is a diagram illustrating an example of replacing a sub-person within a photographed image. FIG. 2 is a functional block diagram of a server device. It is a flowchart explaining an image generation method. FIG. 7 is a diagram illustrating an example of replacing a sub-person within a photographed image.

Embodiments of the present invention will be described below with reference to the drawings.

As shown in FIG. 1, the image generation system according to the present embodiment includes a server device 1, a control device 2, and a photographing device 3. Prints are sold to customers. The photographing device 3 is, for example, a digital camera.

This image sales system is to be introduced at leisure facilities and the like. A ticket owned by a customer is assigned identification information that uniquely identifies the customer. For example, a ticket includes a two-dimensional code indicating identification information. The ticket may be a paper ticket or an electronic ticket that can be displayed on a smartphone or the like.

A customer registers his or her face image in the server device 1 in association with identification information when entering the park. For example, a customer can register a facial image by accessing a dedicated website using a smartphone or the like that they own and photographing their own face with the smartphone's built-in camera. If you are visiting the park with a group of family members, friends, etc., you may register facial images and identification information of multiple people as a group.

Inside the park, a photographer takes pictures with a photographing device 3 in response to a customer's request. The photographed image data is transmitted to the control device 2. Furthermore, before or after the photograph is taken, the photographer reads the two-dimensional code presented by the customer with the photographing device 3 to obtain identification information. The acquired identification information is transmitted to the control device 2.

The control device 2 is a computer with a communication function. The control device 2 transmits the photographed image and identification information received from the photographing device 3 to the server device 1.

When taking photos inside the park, as shown in Figure 2, not only the customer who requested the photo shoot (main subject M), but also other park visitors in the surrounding area (sub person B) may be included in the shot image. be.

Therefore, in the present embodiment, the server device 1 extracts the facial images linked to the identification information received from the control device 2 from among the facial images registered in advance, and based on the extracted facial images, The person in the photographed image is distinguished into a main subject M and a sub-person B. The server device 1 creates a fictitious person image that matches the physique, age, gender, posture, etc. of the identified sub-person B, and replaces the sub-person B in the photographed image with the fictitious person K, as shown in FIG.

In this way, by replacing the sub-person B other than the person to be photographed (main subject M) with a fictitious person, the problem of portrait rights can be resolved and the photographed image can be sold to customers. Further, since the sub-person B is not erased (made transparent) but replaced with a fictitious person, the atmosphere of the photographing environment, such as the surrounding liveliness and crowding, can be maintained. Since the person appearing in the photographed image is a fictitious person, the customer who purchased the photographed image can freely upload the image to SNS or the like.

Next, the configuration of the server device 1 will be described using FIG. 3. The server device 1 is a computer having a CPU, a storage section 10, a communication section, etc., and the CPU executes a control program stored in the storage section 10, thereby controlling a registration section 11, a captured image receiving section 12, and a discriminating section. 13, the functions of the creation section 14 and the replacement section 15 are realized.

The registration unit 11 receives registration of a customer's facial image from a user terminal such as a smartphone owned by the customer. When the registration unit 11 receives the customer's facial image and identification information from the user terminal, the registration unit 11 stores the facial image in association with the identification information in the storage unit 10 .

Face images of multiple people may be grouped and registered. A group of facial images may be associated with identification information of any one customer, or with a plurality of pieces of identification information.

The photographed image receiving unit 12 receives the photographed image and identification information from the control device 2 .

The determination unit 13 retrieves the face image corresponding to the identification information received from the control device 2 from the storage unit 10 . If the received identification information is linked to a face image group, all face images included in this group are extracted.

The determination unit 13 extracts a person included in the captured image received from the control device 2 using a known method. The determining unit 13 determines, among the people extracted from the photographed images, the person corresponding to the face image retrieved from the storage unit 10 as the main subject. Further, the determining unit 13 determines, among the people extracted from the photographed image, a person who is not the main subject to be a sub-person.

The creation unit 14 creates an image of a fictional character to replace the sub-character. The image of the fictitious character has the same physique, age, gender, facial expression, posture, etc. as the sub-character. The fictional character image may include only the face or the entire body. A known machine learning system such as a generative adversarial network (GAN) can be used to create the fictitious person image.

The creation unit 14 may transmit the image of the sub-person to an external computer, request creation of a fictional character image, and acquire the fictional character image from the external computer.

The replacement unit 15 replaces the sub-person in the photographed image with the created fictitious person. If the created fictitious person image only has a face, only the face part is replaced. Since the fictitious person has the same physique, age, gender, expression, posture, etc. as the sub-person, the photographed image after replacement will have a natural finish.

Next, the image generation method according to this embodiment will be explained along the flowchart shown in FIG.

When a customer enters a leisure facility, he or she registers his or her face image (step S1). For example, the customer uses his or her own smartphone to access the registration site listed on the ticket. When the facial image registration process begins, the smartphone's camera is activated, and the customer takes a photo of his or her face and registers it. At the time of registration, the identification information assigned to the ticket is linked to the facial image.

If you are visiting the park in a group of multiple people, you can register the group by sequentially taking and registering the faces of other people. When photographing another person's face, the two-dimensional code of the ticket owned by that person may be photographed and the identification information may be read. At least one piece of identification information is associated with each group-registered face image.

The customer is photographed in the park using the photographing device 3 (step S2). Identification information is read from the two-dimensional code of the customer's ticket before or after the photo shoot. The photographed image and identification information are transferred to the server device 1.

The determining unit 13 of the server device 1 extracts the pre-registered face image corresponding to the received identification information, and determines the person in the photographed image into a main subject and a sub-person (step S3).

The creation unit 14 of the server device 1 creates a fictitious character image for replacing the sub-character (step S4).

The replacement unit 15 of the server device 1 replaces the sub-person area in the photographed image with the fictitious person image created in step S4 (step S5).

The photographed image in which the sub person is replaced with a fictitious person is stored in the storage unit 10 in association with identification information. When the customer holds the ticket up to a print terminal near the entrance/exit gate to read the identification information, the print terminal communicates with the server device 1 and receives a photographed image corresponding to the identification information. Upon receiving the payment, the print terminal performs printing processing of the photographed image. The customer can obtain a print of the photographed image in which the sub-person in the photo has been replaced with a fictitious person.

Alternatively, the customer may operate a user terminal to download image data of captured images.

When replacing all sub-characters with fictitious characters, the fictitious characters' clothing may be in line with the theme of the facility. For example, in the case of a theme park that recreates an old townscape, the clothes of fictional characters may be the clothes of people from the past. In the case of a theme park that recreates a foreign cityscape, the fictitious characters may be foreigners.

In the above embodiment, an example in which the sub-character is replaced with a fictitious character has been described, but as shown in FIG. 5, the sub-character B may be replaced with the character C. For example, the server device 1 stores 3D model data of a mascot character of a leisure facility, and uses the 3D model data to create a character image whose posture and expression match those of the sub-person in the photographed image. The server device 1 replaces the sub-person in the photographed image with the created character image.

The character image that is composited with the photographed image changes depending on the number, position, posture, expression, etc. of the sub-persons in the photograph, so it is an image with higher added value than a composite image using a predetermined template. can be provided to customers.

In the above embodiment, if the photographing device 3 used by the photographer has a communication function such as a smartphone or a tablet terminal, the control device 2 is omitted and the photographed image and identification are sent from the photographing device 3 to the server device 1. You may also send information.

In the above embodiment, an example in which a photographer photographs a customer has been described, but automatic photographing may be performed using the photographing device 3 installed at a predetermined location. Before or after taking a picture, the customer presents the two-dimensional code of the ticket to the reader to read the identification information.

In the above embodiment, an example has been described in which the user's face image is registered in advance and the server device 1 discriminates between the main subject and the sub person, but the user may also specify the main subject. For example, the server device 1 transmits the photographed image and the person extraction result to a print terminal in the park or a user terminal owned by the user, and displays the captured image and the person extraction result. The user selects a main subject from among the people included in the photographed image. The server device 1 accepts the selection of the main subject, and determines persons who are not selected as sub-persons.

The server device 1 creates a fictitious person image or character image corresponding to the person who is not selected (sub-person) and replaces the image with the sub-person.

In the above embodiment, an example has been described in which the subject to be photographed is a person and the person included in the photographed image is distinguished into the main subject and the sub-person. The main subject may be something other than a person. For example, when the server device 1 receives image processing settings for the non-person mode, it determines all the people extracted from the photographed image as sub-persons, and replaces the sub-persons with fictitious persons or characters.

Although the invention has been described in detail using specific embodiments, it will be apparent to those skilled in the art that various changes can be made without departing from the spirit and scope of the invention.

1 Server device 2 Control device 3 Photographing device

Claims (3)

a receiving unit that receives a photographed image from the photographing device;
a discrimination unit that discriminates a plurality of people in the photographed image into a main subject and a sub-person;
a creation unit that creates a fictitious person image or character image that matches the posture of the sub-person;
a replacement unit that replaces the entire body of the sub-person in the photographed image with the fictitious person image or character image;
a registration unit that associates and registers the facial image and identification information;
Equipped with
The receiving unit receives identification information together with the photographed image,
The determining unit determines, among the plurality of people in the captured image, the person corresponding to the face image linked to the identification information received by the receiving unit as the main subject, and identifies people other than the main subject as the main subject. It is determined that the person is the sub-person,
The image generation system does not replace the main subject in the photographed image with the fictitious person image or character image . It is further equipped with a memory section that stores character model data.
The image generation system according to claim 1, wherein the creation unit creates the character image corresponding to the sub-person using the model data. a step of receiving registration of linked facial images and identification information;
a step of receiving a photographed image and identification information from the photographing device;
A step of determining, among the plurality of people in the photographed image, a person corresponding to a face image linked to the received identification information as a main subject, and determining a person other than the main subject as a sub-person;
creating a fictitious person image or character image that matches the posture of the sub-person;
replacing the entire body of the sub-person in the photographed image with the fictitious person image or character image;
Equipped with
An image generation method in which the main subject in the photographed image is not replaced with the fictitious person image or character image .

Images