JP2021140496A - Image processing apparatus, image processing method, and program - Google Patents

Abstract

To provide an image processing apparatus which presents a composition so that a user can reproduce the composition of a picture popular on social network, etc., an image processing method and a program.SOLUTION: A method of an image processing apparatus includes: recognizing an attribute of a subject included in an input image input by a user, and a situation in which the input image was captured; acquiring an image based on a preference of the user, from among images posted on a service on a network, having a predetermined value evaluated in the service; recognizing an attribute of a subject included in the acquired image and the situation where the image was captured; estimating whether the composition of the acquired image can be reproduced or not on the basis of a result of recognizing the input image and a result of recognizing the acquired image; and outputting a method for taking a picture in the composition of the acquired image, to the user, on the basis of a result of estimation.SELECTED DRAWING: Figure 5

Description

The present invention relates to a technique for taking a picture with an optimum composition at the time of taking a picture.

With the spread of digital cameras and smartphones, it has become common to take pictures. There are many compositions that are generally considered to be good as a photographic composition, such as a golden section composition, a vanishing point composition, and a golden spiral composition. However, it was a difficult task for the user to decide which composition to take. Therefore, there is known a technique for presenting to the user what kind of composition is good.

For example, in Patent Document 1, it is possible to determine the shooting situation and inform the user of the recommended composition when a specific subject is the main subject. The same applies to Patent Documents 2 and 3, and a recommended composition can be presented based on the subject.

JP-A-2010-21030 JP-A-2014-192743 JP 2013-102467

However, in recent years, due to the spread of SNS (photo posting system, short text posting system, etc.), images taken with digital cameras and smartphones are uploaded to SNS and used to receive "like" evaluations from other people. Things are getting more and more. Therefore, the composition required by the user is also changing from the correct composition according to the subject and the situation to the currently popular photographs and compositions. However, there was no technology to present the composition of popular SNS photographs that change daily.

The present invention has been made in view of such a problem, and an object of the present invention is to provide a technique for presenting a composition so that a user can easily reproduce a composition of a photograph popular in SNS or the like.

The information processing apparatus according to the present invention is characterized by having the following configurations. That is, based on the user's preference from the recognition means for recognizing the attribute of the subject reflected in the input image input by the user and the situation in which the input image was taken, and the image posted to the service on the network. An acquisition means for acquiring an image whose evaluation is equal to or higher than a predetermined value in the service, a recognition means for recognizing an attribute of a subject reflected in the acquired image, and a situation in which the image is taken, and the above-mentioned. Based on the recognition result of the input image and the recognition result of the acquired image, the estimation means for estimating whether the composition of the acquired image can be reproduced, and the estimation result of the estimation means, the user obtained the acquisition. An output means that outputs a method for taking a picture with the composition of an image.

According to the present invention, a user can easily reproduce the composition of a popular photograph on SNS or the like.

The figure which shows an example which searches a similar image and judges the reproducibility The figure which shows an example which presents the reproducibility method The figure which shows the whole structure of the image processing apparatus of Embodiment 1. The block diagram which shows the internal structure of the image processing apparatus of Embodiment 1. The flowchart which shows the processing of the image processing apparatus of Embodiment 1. The flowchart which shows the operation of S504 of Embodiment 1.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. It should be noted that the following embodiments do not limit the present invention, and not all combinations of features described in the present embodiment are essential for the means for solving the present invention. The same configuration will be described with the same reference numerals.

[Embodiment 1]
First, before explaining the image processing apparatus of this embodiment, the effect of this embodiment will be supplementarily described. 1 and 2 are diagrams showing an example of the effect of the present embodiment.

FIG. 1 is an example of acquiring a reproducible image from an SNS by using an image being photographed by a user. SNS is a service on the network, and photos are posted. FIG. 1A is an image taken by the user. FIG. 1B is an example of extracting a subject from the subject. Here, one person and the xxx building are extracted. 1 (c), 1 (d), and 1 (e) are examples of results of searching popular photographs from SNS based on the extracted subject. "Popular" means that the number of "likes" is high, such as above a predetermined value. From here, reproducibility estimation is further performed. For example, in this case, FIG. 1C estimates that the person is not in the movable range and determines that the reproducibility is NG. In FIG. 1D, the building is slanted, but it is estimated that this can be reproduced by tilting the camera, and it is judged that the reproducibility is OK. Since FIG. 1 (e) is photographed at night, it is presumed that the photograph cannot be taken in the daytime when FIG. 1 (a) is photographed, and it is judged that the reproducibility is NG.

FIG. 2 is an example of means for instructing the user on the reproduction method. As a method for reproducing the image of the photograph of FIG. 1D, the outlines of the positions of people and buildings are superimposed and displayed with a frame line. The user can reproduce the photograph of FIG. 1 (d) by adjusting the position of the camera and the position of the person so as to match the contour.

Then, the image processing apparatus of this embodiment will be described.

FIG. 3 is a diagram showing the overall configuration of the image processing apparatus. The CPU 302 is a system control unit and controls the entire image processing device. The ROM 303 is a read-only memory for storing programs and parameters that do not require changes such as an image processing program and initial data. The RAM 304 is a memory for temporarily storing input information, calculation results in image processing, and the like. The holding unit 305 is a hard disk or memory card fixedly installed in the image processing device, or a memory card, flexible disk, IC card, etc. that can be attached to and detached from the image processing device. The input detection unit 306 receives the required data input. The communication I / F 307 transmits / receives data between the operating device and the cloud or the like. The GPU board 308 is a general-purpose graphic board. The display driver 309 is software for controlling the display unit 311. The video input unit 310 is a center for inputting video from the outside. The GPS acquisition unit 311 is a component that acquires GPS information. It should be noted that this configuration diagram is based on the premise of a camera, and the configuration of a mobile terminal such as a smartphone may be different. When the image processing device is a smartphone, the image processing device has an operation unit 312 and a display unit 313.

The input detection unit 306 is composed of a keyboard, a mouse, and a touch panel, and includes information necessary for creating a natural and dynamic image (for example, information on a person area, information on a type and shape of clothes, information on a material) and various types of information. It is for inputting commands.

The holding unit 305 is a device capable of writing and reading various information. The holding unit 305 contains learning data of various subjects (characteristics of people, buildings, natural objects, etc.), user preference lists, subject attributes (movability, size, color, etc.), and position data (geographically) as learning data in advance. (Position, height, etc.) list is registered.

Further, the display unit 313 is composed of an electronic display device such as a liquid crystal screen of a camera or a display device of a smartphone. The reproducible image acquired from the SNS and the reproduction method are displayed to the user.

Next, the configuration and operation of the image processing device in this embodiment will be described.

The block diagram of the image processing apparatus is shown in FIG. 4, and the operation flowchart is shown in FIG. Hereinafter, it is assumed that the flowchart is realized by the CPU executing the control program.

First, in step S501, the image data acquisition unit 405 holds the input image acquired from the video input unit 310 in the RAM 304. Next, the subject detection unit 406 detects the subject of the acquired image based on the subject learning data list 402.

Next, in step S502, the SNS search condition creation unit creates a search condition for the SNS based on the extracted subject. After setting the search conditions for the extracted subject, as a correction, based on the user preference learning data list 403, based on which subject the user attaches importance to, what is the threshold of popularity, etc. Correct the search conditions. After that, the network communication unit 307 is used to communicate via the network, and an image is acquired from the photograph posted on the SNS.

Next, in step S503, the subject is detected by using the subject detection unit 406 in the same manner as in step S501 with respect to the image acquired from the photograph posted on the SNS.

Next, in step S504, the reproducibility estimation unit 408 is based on the recognition results of the subject recognized in steps S501 and S503, the situation recognition, the subject attribute, the position data list 404, the GPS information acquisition unit 311 as the camera position information, and the like. Estimate the reproducibility. The details of this process will be described later.

Next, in step S505, the user is made to select which image he / she wants to reproduce among the reproducible images displayed on the image display unit 313 through the operation unit 312.

Next, in step S506, the reproduction method creation unit 409 extracts only the outline of the subject from the image to be reproduced, and superimposes and displays it on the screen display unit 313. In addition to displaying on the screen display and instructing the composition, it is also possible to output instructions by voice.

Further, step S507 is executed in parallel, and the user preference analysis unit 401 determines which subject the user has determined to have high priority based on the determination selected by the user in step S505 with the selected image. Compare and estimate the unselected images. Then, the user preference learning data list 403 is updated based on the estimation result.

<Reproducibility estimation unit>
Next, the reproducibility estimation process in step S504 will be described in detail using the flowchart of FIG.

In step S601, the reproducibility for a movable subject is estimated. It is determined whether the number and type of the movable subject match. Also, even if the subject is movable, it cannot be moved anywhere, so the difference in height of the subject from the ground is within the threshold range and the movement in the lateral direction is within the threshold range compared to the image taken by the user. Determine if it is within the threshold range. If it is determined that everything is OK here, the process proceeds to S603. If even one is NG, proceed to S602.

In step S602, even if the determination in S601 is NG, it is determined whether the difference is within the threshold range as a result of weighting each determination. If it is OK, proceed to S603. If it is NG, proceed to S608.

In step S603, the reproducibility for a non-movable subject is estimated. It is determined whether or not the immovable subject can be reproduced by changing the direction and magnification of the camera. First, the geographical position is acquired based on the subject attribute and the position data list 404, and the geographical position of each immovable subject is acquired. Next, the GPS acquisition unit 311 acquires the geographical position of the camera. Without changing the geographical position of the camera position, each immovable subject is turned, and it is calculated and determined whether or not it can be within the shooting range of the camera by changing the magnification. In addition, the angle of the subject is also estimated, compared with the estimated height of the camera, and whether it is within the threshold range is calculated and determined. If everything is OK here, proceed to S605. If even one is NG, proceed to S604.

In step S604, even if the determination in S603 is NG, it is determined whether the difference is within the threshold range as a result of weighting each determination. If it is OK, proceed to S605. If it is NG, proceed to S608.

In step S605, it is determined whether the shooting time zone and the weather match. The difference is calculated and judged based on the brightness and the color of the sky in order to judge whether the shooting time zones are the same. Also, regarding the weather, the weather is estimated and judged from the condition of the sky and the condition of the ground. If everything is OK, proceed to S607. If even one is NG, proceed to S606.

In step S606, even if the determination in S605 is NG, it is determined whether the difference is within the threshold range as a result of weighting each determination. If it is OK, proceed to S607. If it is NG, proceed to S608.

When the process proceeds to step S607, it is determined that the reproducibility is OK, and the image is displayed on the image display unit 313 as a reproducible image by the user.

When the process proceeds to step S608, it is determined that the reproducibility is NG, and the fact that the user does not have a reproducible image is displayed on the image display unit 313.

[Other Embodiments]
The present invention is also realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiment is supplied to the system or device via a network or various storage media, and the computer (or CPU, MPU, etc.) of the system or device reads the program. This is the process to be executed. Further, the present invention can also be realized by performing processing in cooperation with a plurality of processors.

In addition, all of the above-described embodiments are illustrative and not limited to the present invention, and the present invention can be carried out in various other modifications and modifications.

Claims (11)

A recognition means for recognizing the attributes of the subject reflected in the input image input by the user and the situation in which the input image was taken.
An acquisition means for acquiring an image based on the user's preference from an image posted to a service on the network and having an evaluation of a predetermined value or more in the service.
A recognition means for recognizing the attributes of the subject reflected in the acquired image and the situation in which the image was taken.
An estimation means for estimating whether or not the composition of the acquired image can be reproduced based on the recognition result of the input image and the recognition result of the acquired image.
An image processing device including an output means for outputting to the user a method for taking a picture with the composition of the acquired image based on the estimation result of the estimation means. The image processing apparatus according to claim 1, wherein the attribute of the subject is recognized by referring to data in which at least one of the features of a person, a natural object, and a building is registered in advance. The image processing apparatus according to claim 1 or 2, wherein the situation is at least one of a pre-registered position information of a natural object or a building and a time when an image is taken. Any one of claims 1 to 3, wherein the estimation means estimates whether the composition of the acquired image can be reproduced by moving the subject or operating the camera while the user is capturing the input image. The image processing apparatus according to the section. The image processing apparatus according to any one of claims 1 to 4, wherein the estimation means estimates a method of reproducing the composition of the acquired image. The image processing apparatus according to any one of claims 1 to 5, wherein the output means displays information indicating an area of a subject in the acquired image on the display means. The image processing apparatus according to claim 6, wherein the display means shows a region in which the subject exists in the acquired image by superimposing it on the input image with a frame line. The image processing apparatus according to any one of claims 1 to 7, wherein the user's preference includes at least one of information about a subject that the user attaches great importance to and information about a threshold value for evaluation. The user's preference is held by the holding means, and the user's preference held by the holding means is updated based on that the user took a picture according to the instruction by the output means and did not take a picture. The image processing apparatus according to any one of claims 1 to 8, wherein the image processing apparatus is used. A recognition process that recognizes the attributes of the subject in the input image input by the user and the situation in which the input image was taken.
An acquisition process of acquiring an image based on the user's preference from an image posted to a service on the network and having an evaluation of a predetermined value or more in the service.
A recognition process for recognizing the attributes of the subject reflected in the acquired image and the situation in which the image was taken.
An estimation step of estimating whether or not the composition of the acquired image can be reproduced based on the recognition result of the input image and the recognition result of the acquired image.
An image processing method comprising an output step of outputting to the user a method for taking a picture with the composition of the acquired image based on the estimation result of the estimation means. Computer,
A recognition means for recognizing the attributes of the subject reflected in the input image input by the user and the situation in which the input image was taken.
An acquisition means for acquiring an image based on the user's preference from an image posted to a service on the network and having an evaluation of a predetermined value or more in the service.
A recognition means for recognizing the attributes of the subject reflected in the acquired image and the situation in which the image was taken.
An estimation means for estimating whether or not the composition of the acquired image can be reproduced based on the recognition result of the input image and the recognition result of the acquired image.
A program that causes the user to function as an output means for outputting a method for taking a picture with the composition of the acquired image based on the estimation result of the estimation means.

Images