JP2024008639A - Information processor, method for processing information, and information processing program - Google Patents

Abstract

To provide an appropriate content regarding a synthesis image.SOLUTION: The information processor according to the present application includes: a determination unit for determining whether a synthesization target image is an appropriate image appropriate for synthesization; and a provision unit for providing a content for synthesizing the synthesization target image and a taken image taken by a user when the determination unit determines that the synthesization target image is an appropriate image.SELECTED DRAWING: Figure 1

Description

The present invention relates to an information processing device, an information processing method, and an information processing program.

In recent years, augmented reality (AR) has become popular as content provided via the Internet. In such AR, for example, a composite image is provided in which three-dimensional data viewed from a virtual viewpoint is displayed superimposed on a captured image (see, for example, Patent Document 1).

Patent No. 7072706

However, in the conventional technology, there is room for improvement in providing appropriate content regarding composite images.

The present invention has been made in view of the above, and an object of the present invention is to provide an information processing device, an information processing method, and an information processing program that can provide appropriate content related to composite images.

In order to solve the above-mentioned problems and achieve the objects, an information processing device according to the present invention includes a determination unit that determines whether a composition target image is an appropriate image suitable for composition, and a determination unit that determines whether the composition target image is an appropriate image suitable for composition. The present invention is characterized by comprising a provision unit that provides content for compositing the compositing target image and a photographed image taken by a user when it is determined that the compositing target image is the appropriate image. .

According to the present invention, appropriate content regarding a composite image can be provided.

FIG. 1 is a diagram illustrating an example of information processing according to an embodiment. FIG. 2 is a diagram (part 1) showing a specific example of information processing according to the embodiment. FIG. 3 is a diagram (part 2) showing a specific example of information processing according to the embodiment. FIG. 4 is a block diagram illustrating a configuration example of an information providing apparatus according to an embodiment. FIG. 5 is a diagram illustrating an example of a posted information database according to the embodiment. FIG. 6 is a block diagram illustrating a configuration example of a terminal device according to an embodiment. FIG. 7 is a diagram showing an overview of estimation processing according to the embodiment. FIG. 8 is a diagram illustrating an example of a composite image according to the embodiment. FIG. 9 is a diagram illustrating an example of a display regarding an appropriate image according to the embodiment. FIG. 10 is a flowchart illustrating an example of a process executed by the information providing apparatus according to the embodiment. FIG. 11 is a flowchart (Part 1) illustrating an example of a process executed by the terminal device according to the embodiment. FIG. 12 is a flowchart (Part 2) illustrating an example of a process executed by the terminal device according to the embodiment. FIG. 13 is a flowchart (part 3) illustrating an example of a process executed by the terminal device according to the embodiment. FIG. 14 is a flowchart (Part 4) illustrating an example of a process executed by the terminal device according to the embodiment. FIG. 15 is a hardware configuration diagram showing an example of a computer that implements the functions of the information processing device according to the embodiment.

DESCRIPTION OF THE PREFERRED EMBODIMENTS An information processing apparatus, an information processing method, and an embodiment of an information processing program (hereinafter referred to as "embodiments") according to the present application will be described in detail below with reference to the drawings. Note that the information processing apparatus, information processing method, and information processing program according to the present application are not limited to this embodiment.

[Embodiment]
[1. Information processing〕
First, an example of information processing according to the embodiment will be described using FIG. 1. FIG. 1 is a diagram illustrating an example of information processing according to an embodiment. In addition, although the case where the information processing apparatus based on embodiment is the terminal device 10 is demonstrated below, the information provision apparatus 100 may be sufficient.

The information providing apparatus 100 according to the embodiment shown in FIG. 1 is, for example, an information processing apparatus that operates an SNS (Social Networking Service). For example, the information providing device 100 distributes posted information posted by posters such as hair salons to the terminal device 10 of the user U who is the viewer.

When providing such an SNS, the information providing device 100 creates a hairstyle that allows the user U to try out various hairstyles by combining a user image taken of the user U with a hairstyle posted on the SNS. Provide simulation-related services.

Further, the terminal device 10 shown in FIG. 1 is a terminal operated by a user U. For example, the terminal device 10 displays an SNS viewing page provided by the information providing device 100 in response to an operation by the user U. Further, the terminal device 10 generates a composite image by combining the hairstyle posted on the SNS with the user image of the user U, and displays the generated composite image.

in particular. As shown in FIG. 1, the information providing device 100 acquires posted information from a poster, such as a beauty salon, hair salon, stylist, etc. (step S1).

Subsequently, the information providing device 100 generates a viewing page for the user U based on the access by the terminal device 10, and then provides the generated viewing page to the terminal device 10 (step S2). The terminal device 10 that received the viewing page displays the viewing page on the screen (step S3). Note that a specific example of the viewing page will be described later using FIG. 2 and the like.

Subsequently, the terminal device 10 determines an image to be combined from among the posted images displayed on the viewing page based on the operation by the user U (step S4). Note that the images to be synthesized here are posted images posted by each poster, and are images of hairstyles.

That is, the user U can select any image to be combined from the viewing page. Furthermore, as will be described later, the terminal device 10 can, for example, determine whether or not the compositing target image is an appropriate image suitable for compositing, and based on the determination result, accept selection of only the appropriate image as the compositing target image. .

Subsequently, the terminal device 10 acquires a user image to be combined with the combination target image (step S5). For example, the terminal device 10 acquires a user image by photographing a new user image with a camera, or by reading a previously photographed user image from memory.

Subsequently, the terminal device 10 generates a composite image by combining the composition target image and the user image. For example, the terminal device 10 generates a composite image by inputting the composition target image and the user image into a composition model. The compositing model here is a model that has been trained to output an image when the user U has a hairstyle that appears in the compositing target image when the user image and the compositing target image are input. Regarding the synthesis model, any model can be adopted, such as a model that outputs an image in which only the hairstyle is changed so that when a face area of the user image is selected, the face area is not changed. be.

Then, the terminal device 10 provides the synthesized image to the user U (step S7). Thereby, the user U can try out the hairstyle of the posted image posted on the SNS, so it is possible to provide a highly convenient service for the user U.

By the way, in providing such a service related to hairstyle simulation, hairstyles photographed by each poster are posted on the SNS. Therefore, the posted images include appropriate images suitable for compositing and inappropriate images not suitable for compositing.

Specifically, since each poster can arbitrarily post an image of a hair model as a subject, the posted images include images of the hair model taken from various angles.

On the other hand, for example, when performing a hairstyle simulation, in order to perform the hairstyle of the user U's face and the posted image, the direction of the user U's face reflected in the user image and the direction of the hair model's face reflected in the compositing target image are determined. It is desired that the error in the image is small and that the face of the user U is easily recognized in the composite image after composition.

Therefore, in the information processing according to the embodiment, it is decided whether or not the appropriate image to be combined is an appropriate image, and content for combining the combined image only for the appropriate image is provided.

Further, for example, when the user U selects an image to be combined and then photographs a user image, it is desired that the user image suitable for composition be easily photographed. Therefore, in the information processing according to the embodiment, it is determined whether or not the posted image posted by each poster is an appropriate image, and based on the appropriate image selected by the user U, the information processing for appropriately photographing the user image is performed. Provide shooting content.

2 and 3 are diagrams showing specific examples of information processing according to the embodiment. As shown in FIG. 2, a viewing page P1 is displayed on the terminal device 10. For example, the viewing page P1 displays a list image area GL in which posted images are displayed as a list, a search window, and the like.

For example, when the user U selects a posted image displayed in the list image area GL, the screen transitions from the viewing page P1 to the details page P2. In the details page P2, a detailed image GR regarding the posted image selected by the user U, an explanatory text TR regarding the detailed image GR, information regarding the poster, the number of "likes", etc. are displayed.

Further, in the detailed image GR, when the detailed image GR is an appropriate image suitable for composition, a composition icon C1 is displayed that moves the composition target image and the photographed image photographed by the user to composition.

For example, the terminal device 10 estimates the orientation of the face of the subject in the detailed image GR using a model that estimates the orientation of the face of the subject in the detailed image GR on three axes. When a predetermined condition is satisfied, such as when the detailed image GR is displayed, a composite icon C1 is displayed for the detailed image GR.

That is, the user U can easily recognize whether or not the detailed image GR is an appropriate image based on whether or not the composite icon C1 is displayed on the detailed page P2. Next, when the user U selects the composite icon C1, the screen transitions from the details page P2 to the user image selection page P3. Note that when the composite icon C1 is selected, it is also possible to move to a shooting screen for shooting a new user image instead of the selection page P3.

On the selection page P3, it is possible to select an image registered in advance in the terminal device 10, and for example, a guide C2 is displayed. For example, the guide C2 suggests the position, size, body orientation, etc. of the user U's face in the user image when composing a composite image. For example, the user U can select an image that matches the guide C2, and the terminal device 10 can synthesize an appropriate composite image.

In this way, the terminal device 10 displays the synthesis icon C1 for transitioning to synthesis for the detailed image GR, which is an appropriate image, and provides synthesis content for synthesizing the detailed image GR and the user image.

Further, as shown in FIG. 3, when the composite icon C1 displayed on the details page P2 is selected, the screen transitions from the details page P2 to the photographing page P4. In this case, the terminal device 10 provides photographic content for photographing a user image.

For example, the terminal device 10 sets shooting conditions for a user image based on the detailed image GR, and then provides shooting content. The terminal device 10 estimates the direction of the face of the subject shown in the detailed image GR, and sets shooting conditions such that the direction of the face of the user U shown in the user image is substantially the same. Then, for example, the terminal device 10 sequentially estimates the direction of the user U's face reflected in the camera during shooting, and activates the shooting button C3, for example, when the direction of the user U's face satisfies the shooting condition. .

In other words, the terminal device 10 makes the shooting button C3 inactive when the shooting conditions are not satisfied. In other words, the terminal device 10 allows the user to take a picture if the shooting conditions are met.

Thereby, the user U can easily photograph an appropriate user image according to the detailed image GR. Note that the shooting content includes, for example, notifying the user by audio output when shooting conditions are met, automatically shooting a user image, and determining the degree of deviation between the shooting conditions and the current user image. It can be changed arbitrarily, such as by displaying it using figures such as , bars, etc.

[2. Configuration example of information providing device]
Next, a configuration example of the information providing apparatus 100 according to the embodiment will be described using FIG. 4. FIG. 4 is a block diagram showing a configuration example of the information providing apparatus 100 according to the embodiment. As shown in FIG. 4, the information providing device 100 includes a communication section 110, a storage section 120, and a control section 130.

(Communication Department 110)
The communication unit 110 is realized by, for example, a NIC (Network Interface Card). Further, the communication unit 110 is connected to a network by wire or wirelessly.

(Storage unit 120)
The storage unit 120 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. As shown in FIG. 2, the storage unit 120 includes a posted information database 121, a score estimation model database 122, and a synthetic model database 123.

(Posted information database 121)
The posted information database 121 is a database that stores posted information posted by each poster. FIG. 5 is a diagram showing an example of the posted information database 121 according to the embodiment.

As shown in FIG. 5, the posted information database 121 includes items such as "posting ID", "poster ID", "image information", "text information", "score", "shooting condition", etc.

"Post ID" is an identifier for identifying posted information, and "poster ID" is an identifier for identifying a poster. “Image information” is information regarding images (posted images) included in posted information. "Text information" is text information included in posted information (for example, an explanatory text regarding a posted image, a hashtag, a comment by a viewer).

The "score" is a score related to a posted image included in posted information. The score here is a score indicating whether or not the posted image is an appropriate image. “Photographing conditions” are shooting conditions required for a user image when the corresponding posted image is used as a composition target image.

(Score estimation model database 122)
Returning to the explanation of FIG. 4, the score estimation model database 122 will be explained. The score estimation model database 122 stores a model for estimating a score indicating whether a posted image is an appropriate image. For example, as will be described later, the score estimation model is a model that outputs a score depending on the orientation of the subject's face in the posted image.

(Synthetic model database 123)
The synthetic model database 123 is a database that stores synthetic models. The compositing model is a model that has been trained to output a composite image in which the hairstyle of the compositing target image is combined with the face of the user U when the posted image, which is the compositing target image, and the user image are input.

(Control unit 130)
The control unit 130 is a controller, and uses, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), an ASIC (Application Specific Integrated Circuit), or an FPGA (Field Programmable Gate Array) to control the information providing apparatus 100. This is achieved by executing various programs (corresponding to an example of an information processing program) stored in an internal storage device using a storage area such as a RAM as a work area. In the example shown in FIG. 4, the control unit 130 includes an acquisition unit 131, a provision unit 132, and a content provision unit 133.

(Acquisition unit 131)
The acquisition unit 131 acquires posted information from the poster's terminal via the communication unit 110. The acquisition unit 131 registers the acquired posted information in the posted information database 121. Further, the acquisition unit 131 acquires an acquisition request regarding the viewed page from the user U's terminal device 10 via the communication unit 110. For example, the acquisition unit 131 passes information regarding the acquired acquisition request to the content providing unit 133.

(Giving unit 132)
The adding unit 132 adds information regarding "score" and "shooting conditions" to posted information registered in the posted information database 121. For example, the assigning unit 132 obtains a score regarding the posted image by inputting it into a score estimation model registered in the score estimation model database 122 that estimates the score of the posted image of posted information, and registers it in the posted information database 121.

Further, the providing unit 132 uses a model to estimate the direction of the face of the subject appearing in the posted information, sets shooting conditions based on the direction of the face, and registers the shooting conditions in the posted information database 121. Note that the series of processing by the adding unit 132 may be performed on each terminal device 10 side.

(Content providing unit 133)
The content providing unit 133 starts the communication unit 110 and provides various types of content to the terminal device 10 of the user U. For example, the content providing unit 133 generates an SNS viewing page based on a request from the terminal device 10 and provides it to the terminal device 10 .

Further, the content providing unit 133 provides various types of content (for example, applications) for compositing the compositing target image and the user image. The content providing unit 133 provides content such as a synthesis target image, a model for estimating the orientation of a subject's face in a user image, and a synthesis model for synthesizing the synthesis target image and the user image.

Note that any type of model can be adopted as the above model. For example, SVM (Support Vector Machine) or DNN (Deep Neural Network) may be adopted as a model. Here, the DNN may be a CNN (Convolutional Neural Network) or an RNN (Recurrent Neural Network). Further, the RNN may be LSTM (Long short-term memory) or the like. That is, any type of model can be adopted as the model. Further, the model may be a model realized by combining a plurality of models, such as a model combining CNN and RNN.

The learning is, for example, deep learning using a deep neural network (DNN). Data mining and other machine learning algorithms may also be used.

[3. Configuration example of terminal device]
Next, a configuration example of the terminal device 10 according to the embodiment will be described using FIG. 6. FIG. 6 is a block diagram showing a configuration example of the terminal device 10 according to the embodiment. As shown in FIG. 6, the terminal device 10 includes a communication section 11, a display section 12, an input section 13, a camera 20, a control section 30, and a storage section 40.

(Communication Department 11)
The communication unit 11 is realized by, for example, a NIC (Network Interface Card). Further, the communication unit 110 is connected to a network by wire or wirelessly.

(Display section 12)
The display unit 12 is, for example, a variety of displays that display images input from the control unit 30.

(Input section 13)
The input unit 13 is an input device for the user U to perform various operations. The input section 13 is a keyboard, a mouse, or a microphone, but when the display section 12 is a touch panel display, the display section 12 and the input section 13 are integrally formed.

(Camera 20)
The camera 20 is an imaging device that includes various imaging elements. Note that although FIG. 6 shows a case where the camera 20 is included in the terminal device 10, the camera 20 may be externally attached to the terminal device 10.

(Storage unit 40)
The storage unit 40 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 40 stores, for example, a user image photographed by the camera 20 and various information transmitted from the information providing device 100.

(Control unit 30)
The control unit 30 is a controller, and uses, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), an ASIC (Application Specific Integrated Circuit), or an FPGA (Field Programmable Gate Array) to control the information providing device 100. This is realized by executing various programs (corresponding to an example of an information processing program) stored in an internal storage device using a storage area such as a RAM as a work area. In the example shown in FIG. 6 , the control unit 130 includes an acquisition unit 31 , a reception unit 32 , an estimation unit 33 , a determination unit 34 , a generation unit 35 , and a provision unit 36 .

(Acquisition unit 31)
The acquisition unit 31 acquires various information from the information providing device 100 via the communication unit 11. For example, the acquisition unit 31 acquires various contents related to SNS and various contents for generating a composite image.

(Reception Department 32)
The reception unit 32 receives various operations by the user U via the input unit 13. For example, the accepting unit 32 accepts, for example, an operation for selecting an image to be combined with respect to the viewing page P1 shown in FIG. 2 .

(Estimation unit 33)
The estimation unit 33 estimates the orientation of the subject's face that appears in the compositing target image selected by the user U, or the orientation of the user's U's face that appears in the captured image that is being captured by the camera 20. For example, the estimation unit 33 estimates the orientation of the subject's face using an estimation model that estimates the orientation of the subject's face in each image. For example, the estimation unit 33 estimates the orientation of the subject's face using on-device AI that has learned the orientation of the subject's face on the terminal device 10.

FIG. 7 is a diagram showing an overview of estimation processing according to the embodiment. For example, as shown in FIG. 7, the estimation unit 33 estimates the angle of the subject's face along three axes (Roll, Pitch, and Yaw) using on-device AI. That is, the estimation unit 33 estimates the direction of the subject's face or the rotation angle (deviation from the front direction) of the face. Then, the estimating unit 33 passes information regarding the estimation result to the determining unit 34.

(Determination unit 34)
The determining unit 34 determines whether the compositing target image selected by the user U is an appropriate image suitable for compositing. For example, the determining unit 34 determines whether or not the image is appropriate based on the orientation of the subject's face appearing in the composition target image estimated by the estimating unit 33.

For example, the determining unit 34 determines that the image is an appropriate image if the orientation of the subject's face estimated by the estimating unit 33 falls within a predetermined range. In other words, if the orientation of the subject's face estimated by the estimation unit 33 deviates from a predetermined range, the determination unit 34 does not determine the image as an appropriate image. For example, the predetermined range here means that the subject's face is facing forward, but any value may be set.

Furthermore, the determination unit 34 may determine whether the compositing target image is an appropriate image based on, for example, the out-of-focus of the subject appearing in the compositing target image and the size of the subject. For example, the determination unit 34 may determine that the compositing target image is an appropriate image when there is little out-of-focus and the subject is an appropriate size. Furthermore, the determination unit 34 may determine whether the compositing target image is an appropriate image based on the background of the subject in the compositing target image. In this case, for example, the determination unit 34 may determine a composition target image in which the background of the subject is a plain color to be an appropriate image.

Note that the above processing by the estimation unit 33 and determination unit 34 regarding the synthesis target image may be performed on the information providing apparatus 100 side. In this case, when requesting a viewing page, the terminal device 10 determines whether the compositing target image is an appropriate image based on the score assigned by the assigning unit 132 of the information providing device 100.

(Generation unit 35)
The generation unit 35 generates a composite image by combining the composition target image and the photographed image photographed by the user U (that is, the user image). For example, the generation unit 35 determines a user image to be combined with the combination target image selected by the user U.

For example, the generation unit 35 determines, based on the operation of the user U, a user image stored in the storage unit 40 of the terminal device 10 or a newly captured user image as the user image to be combined with the compositing target image. .

For example, the generation unit 35 generates a composite image using a composite model. The compositing model is a model that has been trained to output, when a user image and a compositing target image are input, an image when the user U has a hairstyle that appears in the compositing target image.

FIG. 8 is a diagram illustrating an example of a composite image according to the embodiment. As shown in FIG. 8, the generation unit 35 generates the composite image Gn by inputting the composition target image Gm selected by the user U from SNS and the user image Gy to the composition model.

For example, the composite image Gn is an image in which the face of the user U shown in the user image Gy and the hairstyle of the composition target image Gm are combined. For example, by changing the composite image Gn and the user image Gy, the user U can try various hairstyles for the user images Gy, which have different clothes, accessories (such as necklaces and earrings), and facial expressions. Can be done. As an example of usage, user U can take a user image of himself wearing the clothes, decorations, and make-up he would wear to attend a wedding, and try out various hairstyles. image can be made clearer. Note that in such a case, the user image may be an image of the user U's whole body.

(Providing Department 36)
The providing unit 36 provides various contents provided from the information providing device 100 to the user U via the display unit 12. For example, as shown in FIG. 2, the providing unit 36 provides content related to the SNS viewing page P1, details page P2, and user image selection page P3, and as shown in FIG. do.

In addition, when the detailed page P2 is selected before the user image, the providing unit 36 displays the composite icon C1 (Fig. (see 2).

In addition, when the user image is selected before the synthesis target image, the providing unit 36 selects appropriate images that can be appropriately synthesized with the user image on the viewing page P1 where the list image area GL is displayed. Display that synthesis is possible.

FIG. 9 is a diagram illustrating an example of a display regarding an appropriate image according to the embodiment. As shown in FIG. 9, when the user U selects a user image on the selection page P11 for selecting user images to be combined, the screen transitions to a viewing page P12 where a list image area is displayed.

At this time, each posted image has an icon indicating whether or not it is possible to combine, such as "Composition OK" if the image is appropriate for the user image selected by the user, and "Composition NG" if the image is difficult to combine appropriately. C10 is displayed.

More specifically, when the user U selects a user image, the estimation unit 33 estimates the orientation of the user's face in the user image. Subsequently, the determining unit 34 compares each image displayed on the viewing page P12 with the face orientation, and determines an appropriate image for the user image.

In response to these processes, the providing unit 36 displays a synthesis permission icon C10 on the posted image displayed on the viewing page P12. Thereby, the user U can easily recognize posted images that can be combined with the user image selected by the user U.

Note that although the case where a user image is selected from the selection page P11 has been described here, the same process may also be performed for a user image newly captured by the user U with the camera 20.

Further, for example, the providing unit 36 may display the viewing page P12 only with appropriate images that can be combined with the user image. For example, in this case, the providing unit 36 transmits information regarding the orientation of the user U's face in the user image to the information providing device 100 together with an acquisition request regarding the viewing page. The information providing device 100 extracts posted information that is an appropriate image from the posted information database 121 based on information regarding the direction of the user U's face reflected in the user image, generates a viewing page, and provides the page to the terminal device 10. It turns out.

[4. Processing flow]
Next, the processing procedure executed by the information providing apparatus 100 according to the embodiment will be described using FIG. 10. FIG. 10 is a flowchart illustrating an example of a process executed by the information providing apparatus 100 according to the embodiment.

As shown in FIG. 10, the information providing device 100 first obtains posted information from each poster (step S101). Subsequently, the information providing apparatus 100 assigns shooting conditions and compositing conditions to each posted information based on the posted images included in the posted information (step S102). Note that the shooting condition here is the shooting condition required for the user image when the posted image is the image to be synthesized, and the compositing condition is the shooting condition required for the user image when the posted image is the image to be synthesized. These are conditions required for images to be combined.

Subsequently, the information providing device 100 distributes various contents, for example, viewing pages, to the terminal device 10 of the user U (step S103). Then, the information providing device 100 ends the process. Note that the process related to step S102 may be performed on the terminal device 10 side.

Next, the processing procedure executed by the terminal device 10 according to the embodiment will be described using FIGS. 11 to 14. 11 to 14 are flowcharts showing an example of a processing procedure executed by the terminal device 10 according to the embodiment.

First, a series of processing steps until the terminal device 10 provides a composite image will be described using FIG. 11. As shown in FIG. 11, first, the terminal device 10 acquires a viewing page related to a combination target image from the information providing device 100 (step S201).

Subsequently, the terminal device 10 receives selection of an image to be combined by the user U (step S202). Subsequently, the terminal device 10 receives a selection of a user image to be combined with the combination target image (step S203).

Subsequently, the terminal device 10 generates a composite image based on the composition target image and the user image (step S204). Subsequently, the terminal device 10 displays (provides) the composite image (step S205), and ends the process.

Next, a series of processing procedures related to providing a viewing page will be described using FIG. 12. As shown in FIG. 12, first, the terminal device 10 acquires a viewing page related to the compositing target image from the information providing device 100 (step S211).

Subsequently, the terminal device 10 determines whether each posted image displayed on the viewing page is appropriate (step S212). Subsequently, the terminal device 10 provides compositing content (for example, content related to the compositing icon C1 shown in FIG. 2) that enables compositing with the user image to the appropriate image.

Then, the terminal device 10 provides the viewing page to the user U (step S214), and ends the process.

Next, a series of processes related to providing photographed content will be described using FIG. 13. As shown in FIG. 13, first, the terminal device 10 acquires a viewing page related to the compositing target image from the information providing device 100 (step S221).

Subsequently, the terminal device 10 provides the viewing page to the user U (step S222). Subsequently, the terminal device 10 receives an operation for selecting an image to be combined (step S223).

Subsequently, the terminal device 10 sets shooting conditions for the user image based on, for example, the orientation of the face of the subject of the selected compositing target image (step S224). Then, the terminal device 10 provides the photographed content based on the set photographing conditions (step S225), and ends the process.

Next, with reference to FIG. 14, a series of processing steps when determining the user image before the compositing target image will be described. As shown in FIG. 14, the terminal device 10 determines a user image to be combined based on the user's operation (step S231). Note that the user image may be an image registered in advance in the terminal device 10, or may be a newly captured image.

Subsequently, the terminal device 10 determines an appropriate image based on the user image among the posted images displayed on the viewing page (step S232). Subsequently, the terminal device 10 adds compositing content (for example, the compositing permission icon C10 shown in FIG. 9) to the appropriate image among the posted images displayed on the viewing page (step S233).

Then, the terminal device 10 provides the viewing page to the user U (step S234), and ends the process.

[5. Modified example]
In the embodiment described above, a case has been described in which it is determined whether or not the image is an appropriate image based on the direction of the face of the subject appearing in the user image and the synthesis target image, but the present invention is not limited to this. For example, it may be determined whether the image is appropriate based on an evaluation regarding the hairstyle of the subject appearing in the composition target image. More specifically, we created a model that collects ratings using the number of likes on the posting page for each posted image, or crowdwork, etc., and calculates a higher score when a posted image is input. It may also be possible to learn whether the image is appropriate or not based on the output result of the model.

Further, in the above-described embodiment, a case has been described in which a composite image is generated by combining the hairstyle of the composition target image with a user image in which the user's face is photographed, but the present invention is not limited to this. For example, a composite image may be generated in which glasses, hats, accessories, etc. appearing in the composition target image are composited with the user image and the user's face.

Further, in the embodiment described above, a case has been described in which an image posted on an SNS is acquired as a composition target image, but the present invention is not limited to this. For example, an image taken by a user may be used as the synthesis target image.

Furthermore, the user image and the image to be synthesized are not limited to these; for example, the user image and the image to be synthesized are may be changed as appropriate.

[6. effect〕
As described above, the terminal device 10 (an example of an information processing device) according to the present application includes a determination unit 34 that determines whether a composition target image is an appropriate image suitable for composition, and a determination unit 34 that determines whether a composition target image is an appropriate image suitable for composition. The image processing apparatus includes a providing unit 36 that provides content for compositing the compositing target image and the captured image captured by the user when the image is determined to be the appropriate image.

Furthermore, the terminal device 10 according to the present application includes a generation unit 35 that generates a composite image by combining the composition target image and the photographed image. The generation unit 35 generates a composite image in which the hairstyle of the subject appearing in the composition target image is composited with the user's face appearing in the photographed image.

Further, the determining unit 34 determines whether the image to be combined is an appropriate image based on the orientation of the subject's face in the image to be combined. Further, the determination unit 34 determines whether the image is an appropriate image depending on whether the subject's face is facing forward.

Further, the determination unit 34 determines whether the image to be combined is an appropriate image based on the out-of-focus of the subject and the size of the subject. Furthermore, when it is determined that the image to be synthesized is an appropriate image, the providing unit 36 displays a synthesis icon C1 for transitioning to generation of a synthesized image. The terminal device 10 also includes an acquisition unit 31 that acquires images posted on social networking services as images to be combined.

Through any one or a combination of the above-described processes, the information processing apparatus according to the present application can provide appropriate content related to the composite image.

[7. Hardware configuration]
Further, the information processing apparatus according to the embodiments described above is realized by, for example, a computer 1000 having a configuration as shown in FIG. 15. FIG. 15 is a hardware configuration diagram showing an example of a computer that implements the functions of the information processing device according to the embodiment. Computer 1000 has CPU 1100, RAM 1200, ROM 1300, HDD 1400, communication interface (I/F) 1500, input/output interface (I/F) 1600, and media interface (I/F) 1700.

CPU 1100 operates based on a program stored in ROM 1300 or HDD 1400, and controls each section. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 is started, programs depending on the hardware of the computer 1000, and the like.

The HDD 1400 stores programs executed by the CPU 1100, data used by the programs, and the like. Communication interface 1500 receives data from other devices via network (communication network) N and sends it to CPU 1100, and sends data generated by CPU 1100 to other devices via network N.

The CPU 1100 connects output devices such as displays and printers, and input devices such as keyboards and mice (in FIG. 15, the output devices and input devices are collectively referred to as "input/output devices") via the input/output interface 1600. ). CPU 1100 obtains data from an input device via input/output interface 1600. Further, CPU 1100 outputs the generated data to an output device via input/output interface 1600.

Media interface 1700 reads programs or data stored in recording medium 1800 and provides them to CPU 1100 via RAM 1200. CPU 1100 loads this program from recording medium 1800 onto RAM 1200 via media interface 1700, and executes the loaded program. The recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase change rewritable disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory. etc.

For example, when the computer 1000 functions as the information processing device according to the embodiment, the CPU 1100 of the computer 1000 realizes the functions of the control unit 130 by executing a program loaded onto the RAM 1200. The CPU 1100 of the computer 1000 reads these programs from the recording medium 1800 and executes them, but as another example, these programs may be acquired from another device via the network N.

Some of the embodiments of the present application have been described above in detail based on the drawings, but these are merely examples, and various modifications and variations may be made based on the knowledge of those skilled in the art, including the embodiments described in the disclosure section of the invention. It is possible to carry out the invention in other forms with modifications.

[8. others〕
Although the embodiments of the present application have been described above, the present invention is not limited to the contents of these embodiments. Furthermore, the above-mentioned components include those that can be easily assumed by those skilled in the art, those that are substantially the same, and those that are in a so-called equivalent range. Furthermore, the aforementioned components can be combined as appropriate. Furthermore, various omissions, substitutions, or modifications of the constituent elements can be made without departing from the gist of the embodiments described above.

Further, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or the processes described as being performed manually can be performed manually. All or part of this can also be performed automatically using known methods. In addition, information including the processing procedures, specific names, and various data and parameters shown in the above documents and drawings may be changed arbitrarily, unless otherwise specified. For example, the various information shown in each figure is not limited to the illustrated information.

Furthermore, each component of each device shown in the drawings is functionally conceptual, and does not necessarily need to be physically configured as shown in the drawings. In other words, the specific form of distributing and integrating each device is not limited to what is shown in the diagram, and all or part of the devices can be functionally or physically distributed or integrated in arbitrary units depending on various loads and usage conditions. Can be integrated and configured.

For example, the information providing apparatus 100 described above may be realized by a plurality of server computers, and depending on the function, it may be realized by calling an external platform etc. using an API (Application Programming Interface), network computing, etc. can be changed flexibly.

Furthermore, the above-described embodiments and modifications can be combined as appropriate within a range that does not conflict with the processing contents.

Further, the above-mentioned "section, module, unit" can be read as "means", "circuit", etc. For example, the acquisition unit can be read as an acquisition means or an acquisition circuit.

10 Terminal device 11 Communication section 12 Display section 13 Input section 20 Camera 30 Control section 31 Acquisition section 32 Reception section 33 Estimation section 34 Judgment section 35 Generation section 36 Provision section 40 Storage section 100 Information provision device 110 Communication section 120 Storage section 121 Post Information database 122 Score estimation model database 123 Synthetic model database 130 Control section 131 Acquisition section 132 Addition section

Claims (10)

a determination unit that determines whether the compositing target image is an appropriate image suitable for compositing;
a providing unit that provides content for compositing the compositing target image and a captured image taken by a user when the compositing target image is determined to be the appropriate image by the determining unit;
An information processing device comprising: The information processing apparatus according to claim 1, further comprising: a generation unit that generates a composite image by combining the composition target image and the photographed image. The generation unit is
The information processing apparatus according to claim 2, wherein the composite image is generated by combining the hairstyle of the subject appearing in the composition target image with the user's face appearing in the photographed image. The determination unit includes:
The information processing apparatus according to claim 3, wherein the information processing apparatus determines whether or not the image is the appropriate image based on the direction of the face of the subject appearing in the synthesis target image. The determination unit includes:
The information processing device according to claim 4, wherein the information processing device determines whether or not the image is the appropriate image depending on whether or not the face of the subject reflected in the synthesis target image is facing forward. The determination unit includes:
The information processing apparatus according to claim 1, wherein the information processing apparatus determines whether or not the image is an appropriate image based on a focus shift of a subject appearing in the composition target image and a size of the subject. The provision department is
3. The information processing apparatus according to claim 2, further comprising: providing content that displays a synthesis icon that moves to generation of the synthesis image when the synthesis target image is determined to be the appropriate image. The information processing apparatus according to claim 1, further comprising: an acquisition unit that acquires an image posted on a social networking service as the synthesis target image. An information processing method performed by a computer, the method comprising:
a determination step of determining whether the compositing target image is an appropriate image suitable for compositing;
a providing step of providing content for compositing the compositing target image and a photographed image taken by the user when the compositing target image is determined to be the proper image in the determining step;
An information processing method characterized by comprising: a determination procedure for determining whether the compositing target image is an appropriate image suitable for compositing;
a providing step of providing content for compositing the compositing target image and a photographed image taken by a user when the compositing target image is determined to be the proper image by the determining step;
An information processing program that causes a computer to execute.

Images