JP7056003B2 - Image processing equipment, image processing methods and programs - Google Patents

Description

The present invention relates to a technique for performing image processing on an image obtained by photographing a person.

A technique is used to generate an image that makes a person look beautiful by performing various correction processes on an image obtained by photographing a person. Patent Document 1 discloses a device that detects a face region in an image and performs skin-beautifying treatment (correction treatment) on the detected face region to generate an image that makes the subject person look beautiful.

Japanese Unexamined Patent Publication No. 2012-3324

However, in a configuration in which processing such as skin beautification is performed on the face region based on the feature amount of the face region in the image, it is appropriate to use the captured image in which the color, brightness, etc. of the entire image or the face region are not good. There is a risk that various processing cannot be performed.

The present invention has been made in view of such circumstances, and an object thereof is to provide an image processing apparatus capable of generating an captured image capable of appropriately performing image-based processing. To do.

The image processing apparatus according to one aspect of the present invention is an image processing apparatus that performs image processing on a captured image of a person, a storage unit that stores a target value related to an image in a predetermined organ of the human face, and a storage unit that stores the target value of the image in a predetermined organ of the human face, and a human image from the captured image. A face detection unit that detects a face region, an organ detection unit that detects a region of the predetermined organ from the face region detected by the face detection unit, and a feature amount in the region of the predetermined organ detected by the organ detection unit are specified. The feature amount specifying unit, the correction function generation unit that generates a correction function for correcting the feature amount specified by the feature amount specifying unit to the target value stored in the storage unit, and the correction function generation unit. It is included in the captured image based on the correction unit that corrects the clothing area of the person included in the captured image by using the generated correction function and the clothing area of the person included in the captured image corrected by the correction unit. It is characterized by including an impression specifying unit for specifying the impression of the person in order to specify the conversation content to be output to the person .

The image processing method according to one aspect of the present invention is an image processing method using an image processing device that performs image processing on a captured image of a person, wherein the image processing device detects a human face region from the captured image and detects the face. The area of the predetermined organ of the face is detected from the area, the feature amount in the detected predetermined organ area is specified, and the specified feature amount is stored in the storage unit. A correction function for correcting the image is generated, and the generated correction function is used to correct the clothing area of the person included in the captured image, and the correction function is used to correct the clothing area of the person included in the captured image. It is characterized in that the impression of the person is specified in order to specify the conversation content to be output to the person included in the captured image .

The program according to one aspect of the present invention is a program for causing a computer having a storage unit for storing a target value related to an image in a predetermined organ of a human face to process a captured image of a person. A human face region is detected from the captured image, a region of the predetermined organ is detected from the detected face region, a feature amount in the detected predetermined organ region is specified, and the specified feature amount is stored in the storage unit. A correction function for correcting to a certain target value is generated, and the generated correction function is used to correct the clothing area of the person included in the captured image, and to the clothing area of the person included in the corrected image. Based on this, it is characterized in that a process for specifying the impression of the person is executed in order to specify the conversation content to be output to the person included in the captured image .

In one aspect of the present invention, it is possible to generate a captured image capable of appropriately performing image-based processing.

It is a block diagram which shows the configuration example of the computer which concerns on Embodiment 1. FIG. It is a block diagram which shows the function realized by the control part. It is explanatory drawing of the process performed by a control unit. It is a flowchart which shows the procedure of a process performed by a computer. It is a block diagram which shows the function realized by the control part of the computer of Embodiment 2. It is explanatory drawing of the process performed by the control unit of the computer of Embodiment 2. It is a flowchart which shows the procedure of the process performed by the computer of Embodiment 2. It is a schematic diagram of the robot which concerns on Embodiment 3. It is a block diagram which shows the structural example of the robot which concerns on Embodiment 3. It is a block diagram which shows the function realized by the control part of the computer of Embodiment 3. FIG. It is a flowchart which shows the procedure of the process which the computer of Embodiment 3 performs.

Hereinafter, embodiments will be specifically described with reference to the drawings.

(Embodiment 1)
FIG. 1 is a block diagram showing a configuration example of a computer according to the first embodiment.
The computer 1 is, for example, a personal computer, a smartphone, a tablet terminal, or the like. The computer 1 includes a control unit 10, a storage unit 11, an operation unit 12, a display unit 13, a camera connection unit 14, and the like, and each of these units is connected to each other via a bus.

The control unit 10 includes a processor such as a CPU (Central Processing Unit) or an MPU (Micro-Processing Unit). The control unit 10 performs various control processes performed by the computer 1 by appropriately executing the control program stored in the storage unit 11.
The storage unit 11 includes a flash memory, a hard disk, an SSD (Solid State Drive), and the like. The storage unit 11 stores in advance a control program (for example, an image processing program 11a) executed by the control unit 10 and various data (for example, target value data 11b). Further, the storage unit 11 temporarily stores data and the like generated when the control unit 10 executes the control program.

The operation unit 12 includes, for example, a keyboard, a mouse, and the like, receives an operation input by the user, and sends a control signal corresponding to the operation content to the control unit 10.
The display unit 13 includes, for example, a liquid crystal display, an organic EL (Electro Luminescence) display, and the like, and displays various information according to instructions from the control unit 10.
The operation unit 12 and the display unit 13 may be a touch panel configured as an integral body.

A camera 14a is connected to the camera connection portion 14 via, for example, a cable. The camera 14a acquires image data such as still image data or moving image data, and outputs the acquired image data to the computer 1. The control unit 10 of the computer 1 acquires the image data (hereinafter referred to as a captured image) output from the camera 14a via the camera connection unit 14 and stores it in the storage unit 11.
The camera 14a and the computer 1 may be configured to be capable of wireless communication, or the camera 14a may be configured to be built in the computer 1.

In addition to the above-described configuration, the computer 1 of the first embodiment may include a communication unit for connecting to a network such as the Internet.
In the first embodiment, the image processing program 11a, which is the program of the present application, is installed in the computer 1, and the control unit 10 executes the image processing program 11a, whereby the computer 1 operates as the image processing device of the present application.

Next, the function realized by the control unit 10 of the computer 1 executing the image processing program 11a will be described. FIG. 2 is a block diagram showing a function realized by the control unit 10, and FIG. 3 is an explanatory diagram of processing performed by the control unit 10. When the image processing program 11a stored in the storage unit 11 is executed, the control unit 10 of the computer 1 has a face detection unit 21, an organ detection unit 22, a feature amount identification unit 23, a correction function generation unit 24, and a correction unit 25. Realize each function of. In the present embodiment, each of the above-mentioned functions is realized by the control unit 10 executing the image processing program 11a, but a part of each of the above-mentioned functions may be realized by a dedicated hardware circuit.

The face detection unit 21 detects a human face region from the captured image (image data) acquired from the camera 14a via the camera connection unit 14. The camera 14a captures, for example, the whole body or the upper body of the subject person, and the face detection unit 21 detects the face region of the subject person included in the captured image. The face detection unit 21 detects a face region in a captured image by, for example, a template matching technique. Specifically, a template showing an image of a general human face is stored in the storage unit 11 in advance, and when the face detection unit 21 detects a region matching the template from the captured image, the detected region is stored. Extract as a face area.
For example, in the captured image shown in FIG. 3A, the region F surrounded by the broken line is detected as the face region.

The organ detection unit 22 detects a region of a predetermined organ from the face region detected by the face detection unit 21. The predetermined organs are, for example, white eyes, teeth, nose, and cheeks, and the organ detection unit 22 detects white eyes, teeth, nose, and cheek areas in the facial area. The organ detection unit 22 detects the region of each organ by, for example, a template matching technique. Specifically, a template showing an image of each organ (white eyes, teeth, nose and cheeks) of a general person is stored in advance in the storage unit 11, and the organ detection unit 22 matches the template from the face area. When a region is detected, the detected region is extracted as the region of each organ.
For example, in the face region shown in FIG. 3B, the region A1 surrounded by a solid line is detected as a white eye region, the region A2 is detected as a tooth region, the region A3 is detected as a cheek region, and the region A4 is detected as a nose region. It has been detected.

The organ detection unit 22 does not need to detect the regions of all the organs for which the template is prepared in advance, but may detect the regions of at least one organ.
Further, information indicating the relative position of each organ with respect to the face region of a general person is stored in advance in the storage unit 11, and the organ detection unit 22 stores the face region detected by the face detection unit 21. The position and region of each organ may be detected based on the information stored in 11.

The feature amount specifying unit 23 specifies the feature amount of the image in the organ region detected by the organ detection unit 22. The feature amount is, for example, the representative color and the average luminance in the organ region, and the feature amount specifying unit 23 calculates the representative color and the average luminance in the organ region based on the image data (pixel value) included in the organ region.
When the organ detection unit 22 detects a plurality of organ regions, the feature amount specifying unit 23 specifies the representative color and the average luminance of each organ region. Then, the feature amount specifying unit 23 performs weighting addition using, for example, a weighting coefficient set for each organ, and calculates (specifies) the representative color and the brightness in a plurality of organ regions (one face region).

The correction function generation unit 24 reads the target value data 11b from the storage unit 11 and corrects the representative color and luminance (feature amount) specified by the feature amount specifying unit 23 to the target value data 11b read from the storage unit 11. Generate a correction function for. The target value data 11b is data of the target value in the organ region to be detected by the organ detection unit 22, and is the color and brightness to be targeted in the image of the organ region. The correction function generation unit 24 generates, for example, a gamma correction function or a linear function, calculates a gamma value when generating a gamma correction function, and calculates a coefficient when generating a linear correction function.

The correction unit 25 performs correction processing based on the correction function generated by the correction function generation unit 24 on the captured image. The correction unit 25 may be configured to perform correction processing on the entire captured image, or may be configured to perform correction processing only on the face region detected (extracted) by the face detection unit 21. For example, FIG. 3C shows a captured image when the correction process is performed on the entire captured image. In the first embodiment, since the correction processing based on the correction function derived based on the image information of the organ in the human face of the subject is performed by each of the above-mentioned parts, the entire captured image is corrected to an appropriate color and brightness. be able to. Therefore, even if the captured image is dark as a whole as shown in FIG. 3 (A), it can be corrected to a bright image as shown in FIG. 3 (C) by the above-mentioned correction process.
Further, when the control unit 10 further performs a process of detecting (extracting) the area of the person of the subject including the face area detected by the face detection unit 21, the correction unit 25 covers the entire area of the person of the subject. On the other hand, it can be configured to perform correction processing.

Next, the processing performed by the computer 1 will be described. FIG. 4 is a flowchart showing a procedure of processing performed by the computer 1. The following processing is executed by the control unit 10 according to the control program stored in the storage unit 11 of the computer 1.
When the computer 1 acquires a captured image of a person as an image processing target, the control unit 10 (face detection unit 21) detects a human face region from the acquired captured image (S1). When a plurality of people are included in the captured image, the control unit 10 detects each of the face regions of the plurality of people.

The control unit 10 extracts one of the detected face areas (S2). Specifically, the control unit 10 extracts image data (pixel values) included in the detected face region.
Next, the control unit 10 (organ detection unit 22) detects a region (organ region) of a predetermined organ from the face region extracted in step S2 (S3). The control unit 10 extracts the image data (pixel value) included in the detected organ region (S4), and specifies the feature amount (representative color and average brightness) of the extracted organ region based on the extracted image data (S4). Calculate) (S5). When a plurality of organ regions are detected in step S3, the control unit 10 extracts the image data included in each organ region in step S4, and specifies the feature amount of each organ region in step S5. , The feature amount in a plurality of organ regions (face region extracted in step S2) is specified (calculated) based on the feature amount of each organ region.

The control unit 10 (correction function generation unit 24) reads out the target value data 11b from the storage unit 11 (S6), and based on the feature amount (representative color and average luminance) specified in step S5 and the target value data 11b. Generate a correction function (S7).
Then, the control unit 10 (correction unit 25) corrects the captured image based on the correction function generated in step S7 (S8). Here, for example, the correction process may be performed on the face region extracted in step S2.

When the control unit 10 performs the above-mentioned processing on the face area extracted in step S2, there is a face area (unprocessed area) in which the above-mentioned processing is not performed among the face areas detected in step S1. It is determined whether or not (S9). When it is determined that there is an unprocessed area (S9: YES), the control unit 10 extracts one unprocessed face area from the face areas detected in step S1 (S2), and with respect to the extracted face area. Then, the processes of steps S3 to S8 are performed.
When the control unit 10 determines that there is no unprocessed area (S9: NO), the control unit 10 ends the processing for this captured image.

In the above-mentioned processing, the correction processing in step S8 may be performed on the entire captured image in addition to being performed on each face region extracted in step S2. In this case, for example, the feature amount of each of the organ regions detected in step S3 for each of the face regions detected in step S1 is weighted and added using, for example, a weighting coefficient set for each organ to obtain this captured image. Identify (calculate) the feature quantities (representative color and brightness) of the organ region. Then, by generating a correction function based on the specified feature amount and the target value data 11b, a correction function for correcting the entire captured image can be obtained.

In the first embodiment described above, the captured image is corrected by using a correction function derived based on the image information of the organ in the human face of the subject. In the captured image, the color and brightness of organs such as white eyes, teeth, nose, and cheeks on the human face are not easily affected by makeup or race, and the corrected target value (color and brightness to be targeted) is set. Easy to do. Therefore, by generating a correction function based on the feature amount in the organ region such as a person's white eyes and teeth in the captured image and the preset target value, the organ region in the captured image is set to the target color and brightness. It is possible to obtain a correction function that can be corrected to approach. Further, when the entire captured image is corrected by using a correction function capable of correcting the organ region closer to the target value, the entire captured image can be corrected to an appropriate color and brightness. That is, it is possible to correct the color and brightness to an appropriate level even in a region other than the organ region.
Therefore, it is possible to perform appropriate processing by using the captured image corrected to an appropriate color and brightness.

(Embodiment 2)
A modification of the computer 1 of the first embodiment described above will be described. The computer 1 of the second embodiment can be realized by the same configuration as the computer 1 of the first embodiment described above. Therefore, the same components are designated by the same reference numerals and the description thereof will be omitted.
In the computer 1 of the second embodiment, the image processing program 11a is slightly different from the first embodiment, so that the function realized by the control unit 10 executing the image processing program 11a is slightly different from the first embodiment. The same functions as those in the first embodiment are designated by the same reference numerals, and the description thereof will be omitted. Hereinafter, the functions different from those of the first embodiment will be described.

FIG. 5 is a block diagram showing a function realized by the control unit 10 of the computer 1 of the second embodiment, and FIG. 6 is an explanatory diagram of a process performed by the control unit 10 of the computer 1 of the second embodiment.
When the image processing program 11a stored in the storage unit 11 is executed, the control unit 10 of the computer 1 of the second embodiment has a face detection unit 21, an organ detection unit 22, a feature amount identification unit 23, and a correction function generation unit 24. In addition to the correction unit 25, the functions of the clothing area extraction unit 26 and the impression identification unit 27 are realized. The face detection unit 21, the organ detection unit 22, the feature amount identification unit 23, and the correction function generation unit 24 perform the same processing as in the first embodiment. Also in the second embodiment, a part of each function realized by the control unit 10 executing the image processing program 11a may be realized by a dedicated hardware circuit.

The clothing area extraction unit 26 extracts an area of clothing (clothing) worn by the subject person from the captured image. For example, in the captured image, the clothing area extraction unit 26 extracts a predetermined region below the face region as a clothing region with reference to the face region detected by the face detection unit 21. For example, in the captured image shown in FIG. 6A, two subjects are included, and when the face detection unit 21 detects the face areas F1 and F2 of the respective subjects, the clothing area extraction unit 26 has each. The clothing areas C1 and C2 of each subject are extracted corresponding to the face areas F1 and F2.
The correction unit 25 of the second embodiment performs correction processing on the clothes area extracted by the clothes area extraction unit 26 based on the correction function generated by the correction function generation unit 24. As a result, in the captured image shown in FIG. 6A, only the clothing areas C1 and C2 of the respective subjects are corrected as shown in FIG. 6B. In the second embodiment, the clothing area is corrected by using the correction function derived based on the image information of the face (organ) of the subject, so that the clothing area can be corrected to an appropriate color and brightness.

The impression specifying unit 27 identifies the impression of the person of this subject based on the image of the clothing area corrected by the correction unit 25. That is, the impression of the subject person is specified based on the clothes of the subject person. The impression specifying unit 27 identifies the impression of the person as the subject based on the impression information database (hereinafter referred to as the impression information DB) 11c stored in the storage unit 11 in advance.
In the impression information DB 11c, information (impression information) of the average color and brightness of the clothes to be impressed is stored in advance in association with the impressions such as formal, casual, chic, and vivid.

The impression specifying unit 27 first specifies the feature amount of the image in the clothing area corrected by the correction unit 25. The feature amount here is, for example, the representative color and the average luminance in the clothing region, and the impression specifying unit 27 calculates the representative color and the average luminance in the clothing region based on the image data (pixel value) included in the clothing region. do. Then, the impression specifying unit 27 compares the calculated representative color and average brightness of the clothing area with the impression information stored in the impression information DB 11c, and identifies the impression information closest to the representative color and average brightness of the clothing area. do. The impression specifying unit 27 identifies the impression in the specified impression information as the impression of the person of the subject including this clothing area.

Next, the processing performed by the computer 1 of the second embodiment will be described. FIG. 7 is a flowchart showing a procedure of processing performed by the computer 1 of the second embodiment. In each process shown in FIG. 7, the same process as the process of the first embodiment shown in FIG. 4 is assigned the same step number and the description thereof will be omitted.

In the computer 1 of the second embodiment, the control unit 10 performs the same processing as in steps S1 to S7 in FIG.
Next, the control unit 10 (clothing area extraction unit 26) extracts the clothing area of the subject including the face area extracted in step S2 from the captured image to be processed (S11). Specifically, the control unit 10 sets a predetermined region below the face region extracted in step S2 as a clothing region in the captured image, and extracts image data (pixel values) included in the clothing region.

Then, the control unit 10 (correction unit 25) performs a correction process based on the correction function generated in step S7 on the clothing area extracted in step S11 (S12).
Further, the control unit 10 (impression specifying unit 27) identifies the impression of the person of the subject including the face area extracted in step S2 based on the clothing area after the correction process (S13).

When the control unit 10 performs the above-mentioned processing on the face area extracted in step S2, there is a face area (unprocessed area) in which the above-mentioned processing is not performed among the face areas detected in step S1. It is determined whether or not (S9). When it is determined that there is an unprocessed area (S9: YES), the control unit 10 extracts one unprocessed face area from the face areas detected in step S1 (S2), and with respect to the extracted face area. Then, the processes of steps S3 to S7 and S11 to S13 are performed.
When the control unit 10 determines that there is no unprocessed area (S9: NO), the control unit 10 ends the processing for this captured image.

In the second embodiment, the processes of steps S3 to S7 and S11 to S13 are performed on the face region extracted in step S2. That is, when a plurality of subjects are included in the captured image, a correction function is generated, correction processing is executed, and an impression is specified for each subject. Therefore, for example, in a captured image including a person in the sun and a person in the shade, processing based on the image information of each subject is executed for each subject. Therefore, the correction process based on the correction function corresponding to each subject is performed without being affected by the shooting conditions of other subjects, and the impression is specified based on the clothing area after the correction process. The impression of the subject can be specified with high accuracy.

In the second embodiment, the same effect as that of the first embodiment can be obtained. That is, since the clothing area is corrected by using the correction function derived based on the image information of the human face (organ) of the subject, the clothing area of the subject can be corrected to an appropriate color and brightness.

(Embodiment 3)
An embodiment of a robot equipped with the computer 1 of the above-described second embodiment will be described. FIG. 8 is a schematic diagram of the robot according to the third embodiment, and FIG. 9 is a block diagram showing a configuration example of the robot according to the third embodiment.
In the robot 40, the head portion 41 is connected on the body portion 42 which is the base, and the head portion 41 is rotated with respect to the body portion 42 with respect to the body portion 42 by the motor 15a, and is in a horizontal plane. On the other hand, it is configured to rotate at an appropriate angle in the vertical direction. The head portion 41 is provided with the camera 14a and the speaker 16a, and the camera 14a and the speaker 16a can be directed in any direction by rotating the head portion 41 by the motor 15a. The robot 40 is configured so that the camera 14a is arranged at a position where the face of an approaching person can be photographed.

The computer 1 of the third embodiment includes a motor driving unit 15 for driving the motor 15a and a speaker connecting unit 16 for connecting the speaker 16a, in addition to the configuration of the computer 1 of the second embodiment. Further, in the computer 1 of the third embodiment, the conversation processing program 11d is stored in the storage unit 11. Other configurations are the same as those in the second embodiment described above, and the same configurations are designated by the same reference numerals and the description thereof will be omitted. In the third embodiment, the image processing program 11a and the conversation processing program 11d are described as different programs, but they may be configured as one program.

In the speaker connection unit 16, for example, the speaker 16a is connected via a cable, and the content instructed by the control unit 10 to output audio is output by the speaker 16a.
The motor drive unit 15 drives the motor 15a according to the motor drive amount instructed by the control unit 10.

FIG. 10 is a block diagram showing a function realized by the control unit 10 of the computer 1 of the third embodiment. In the computer 1 of the third embodiment, the image processing program 11a stored in the storage unit 11 is the same as that of the second embodiment. Therefore, when the control unit 10 of the computer 1 of the third embodiment executes the image processing program 11a, the same functions as those of the second embodiment are realized. Specifically, the control unit 10 of the third embodiment includes a face detection unit 21, an organ detection unit 22, a feature amount identification unit 23, a correction function generation unit 24, a correction unit 25, a clothing area extraction unit 26, and an impression identification unit 27. Realize each function of. The same functions as those in the second embodiment are designated by the same reference numerals, and the description thereof will be omitted.

Next, the function realized by the control unit 10 of the computer 1 of the third embodiment by executing the conversation processing program 11d stored in the storage unit 11 will be described. When the conversation processing program 11d is executed, the control unit 10 of the computer 1 of the third embodiment realizes the functions of the conversation specifying unit 28, the head driving unit 29, and the voice output unit 30.

The conversation specifying unit 28 specifies the content of the conversation with the person based on the impression of the person of the subject specified by the impression specifying unit 27. The conversation specifying unit 28 specifies the conversation content based on the impression information DB 11c stored in the storage unit 11 in advance. The impression information DB 11c of the third embodiment includes impressions of formal, casual, chic, vivid, etc., information on the average color and brightness of clothes to be impressed, and conversation contents according to the person of each impression. Are stored in association with each other. Therefore, the conversation specifying unit 28 reads out the conversation content corresponding to the impression specified by the impression specifying unit 27 from the impression information DB 11c.

For example, in the captured image, the head driving unit 29 directs the camera 14a and the speaker 16a to the person of the subject having this face area based on the position of the face area detected by the face detecting unit 21. The amount of rotation (drive amount of the motor) from the current state in which 41 should be rotated is specified. Specifically, the amount of rotation (motor drive amount) at which the head portion 41 should be rotated so that the person of the subject is located in the center of the imaging region is specified. Then, the head drive unit 29 rotates the head portion 41 by the specified rotation amount by driving the motor 15a to the motor drive unit 15 based on the specified drive amount.
The voice output unit 30 outputs voice based on the conversation content specified by the conversation identification unit 28 from the speaker 16a via the speaker connection unit 16. As a result, the robot 40 can start a conversation with the subject person.

Next, the processing performed by the computer 1 of the third embodiment will be described. FIG. 11 is a flowchart showing a procedure of processing performed by the computer 1 of the third embodiment. In each process shown in FIG. 11, the same process as the process of the second embodiment shown in FIG. 7 is assigned the same step number and the description thereof will be omitted.

In the computer 1 of the third embodiment, the control unit 10 performs the same processing as steps S1 to S7 and S11 to S13 in FIG. 7.
Next, the control unit 10 (conversation specifying unit 28) specifies the conversation content corresponding to the impression specified in step S13 (S21). Specifically, the control unit 10 reads out the conversation content from the impression information DB 11c.
Further, the control unit 10 (head drive unit 29) head portion 41 in order to point the camera 14a and the speaker 16a toward the subject person based on the position of the face region extracted in step S2 in the captured image to be processed. The amount of rotation (driving amount of the motor 15a) to be rotated is specified (S22).

The control unit 10 drives the motor 15a (S23) with the drive amount specified in step S22, and rotates the head portion 41. As a result, the camera 14a and the speaker 16a of the head portion 41 can be aimed at the person as the subject.
Further, the control unit 10 outputs the voice based on the conversation content specified in step S21 from the speaker 16a via the speaker connection unit 16 (S24).

When the control unit 10 performs the above-mentioned processing on the face area extracted in step S2, there is a face area (unprocessed area) in which the above-mentioned processing is not performed among the face areas detected in step S1. It is determined whether or not (S9). When it is determined that there is an unprocessed area (S9: YES), the control unit 10 extracts one unprocessed face area from the face areas detected in step S1 (S2), and with respect to the extracted face area. Then, the processes of steps S3 to S7, S11 to S13, and S21 to S24 are performed.
When the control unit 10 determines that there is no unprocessed area (S9: NO), the control unit 10 ends the processing for this captured image.

In the third embodiment, the processes of steps S3 to S7, S11 to S13, and S21 to S24 are performed on the face region extracted in step S2. That is, when a plurality of subjects are included in the captured image, a correction function is generated, correction processing is executed, impression is specified, and voice output (conversation) is performed for each subject. Therefore, when a plurality of subjects are included in the captured image, the impression of each subject can be specified, and voice output (conversation) can be performed according to the specified impression.

Also in the third embodiment, the same effects as those of the first and second embodiments described above can be obtained. That is, since the clothing area is corrected by using the correction function derived based on the image information of the human face (organ) of the subject, the clothing area of the subject can be corrected to an appropriate color and brightness. In addition, since the impression of the person of the subject is specified based on the appropriately corrected clothing area, the impression of each subject can be specified accurately, and the voice output (conversation) according to the impression of each subject. It can be performed.

A robot that interacts with a person is equipped with a camera so that the impression of the person being the subject can be identified from the image (captured image) obtained by taking a picture of the person approaching the robot, and the conversation can be started from the identified impression. May be configured. Therefore, in such a robot, it is indispensable to be able to accurately identify the impression of the person as the subject. For example, if the impression of a person on the subject is specified based on a captured image in which the area of the subject is dark, such as when shooting in a backlit state, or a captured image in which the entire subject is dark, the impression may be specified as a dark impression. Further, even when the impression of the subject person is specified based on the captured image that is too bright (the brightness is too high), the impression may be specified to be brighter than the original impression of the subject person. In these cases, the impression of the person as the subject may be erroneously specified, and a misguided conversation may be started based on the erroneous impression. Therefore, in the robot 40 of the third embodiment, the captured image can be appropriately corrected by the processing by the computer 1, and the impression of the person as the subject can be appropriately specified based on the corrected captured image. Therefore, it is possible to have a conversation suitable for the person of the subject.

Since the computer 1 of the second and third embodiments identifies the impression of the person of the subject from the clothing area (clothing) of the subject (person) in the captured image, the correction unit 25 of the second and third embodiments is in the captured image. The configuration was such that the correction process was performed only on the clothing area. In addition to this, it is also possible to specify the impression of the subject from the face area of the subject (person). In this case, the correction unit 25 may perform correction processing on the face area detected by the face detection unit 21, and the impression specifying unit 27 may specify the impression of the person as the subject based on the face area after the correction processing.

The computer 1 of the above-described embodiments 1 to 3 can be applied to a surveillance camera system. When the computer 1 of the first to third embodiments is used, for example, even if the image is obtained by taking an image in the dark outdoors at night, the captured image is appropriately captured based on the image information of the human face (organ) of the subject. It can be corrected to various colors and brightness. Therefore, even when the captured image is dark as a whole, the presence or absence of a suspicious person can be detected based on the appropriately corrected captured image.
Further, the computer 1 of the first to third embodiments is mounted on a camera such as a digital still camera or a digital video camera, and the correction process by the computer 1 of the first to third embodiments is performed as one of the correction processes that can be executed by the camera. You may do it.

The correction process by the computer 1 of the above-described first to third embodiments is effective for a captured image in which the whole is dark or a captured image in which the area of the person of the subject is dark. Therefore, for the captured image to be processed, first, it is determined whether or not the average brightness of the captured image is equal to or less than a predetermined value (first threshold value), and for the captured image whose average brightness is equal to or less than the predetermined value. , The configuration may be such that the correction function is generated and the correction process is executed as described above. Further, it is determined whether or not the average brightness of the captured image to be processed is equal to or higher than a predetermined value (second threshold value higher than the first threshold value), and the captured image having an average brightness of equal to or higher than the predetermined value is determined. It is also possible to configure the configuration to generate the correction function and execute the correction process as described above.

The image processing apparatus according to one aspect of the present invention is an image processing apparatus that performs image processing on a captured image of a person, a storage unit that stores a target value related to an image in a predetermined organ of the human face, and a storage unit that stores the target value of the image in a predetermined organ of the human face, and a human image from the captured image. A face detection unit that detects a face region, an organ detection unit that detects a region of the predetermined organ from the face region detected by the face detection unit, and a feature amount in the region of the predetermined organ detected by the organ detection unit are specified. It is characterized by including a feature amount specifying unit and a correction function generation unit that generates a correction function for correcting the feature amount specified by the feature amount specifying unit to a target value stored in the storage unit. ..

According to one aspect of the present invention, based on the region of a predetermined organ of a person's face in a captured image, a correction function capable of appropriately correcting the region of the predetermined organ is generated. By generating the correction function based on the region of the organ that is not easily affected by the shooting condition or the like, it is possible to obtain the correction function for appropriately correcting the captured image.

The image processing apparatus according to one aspect of the present invention is characterized by including a correction unit that corrects a part of the captured image based on the correction function generated by the correction function generation unit.
According to one aspect of the present invention, it is possible to obtain an appropriately corrected captured image by correcting the captured image using a correction function generated based on a region of an organ that is not easily affected by a shooting condition or the like. can.

The image processing apparatus according to one aspect of the present invention is characterized in that a part of the captured image corrected by the correction unit is a clothing region of a person included in the captured image.
According to one aspect of the present invention, it is possible to appropriately correct the clothing area of a person as a subject in a captured image.

The image processing apparatus according to one aspect of the present invention is characterized by including an impression specifying unit that specifies an impression of a person included in the captured image based on a part of the captured image corrected by the correction unit. ..
According to one aspect of the present invention, by using an appropriately corrected captured image, it is possible to accurately identify the impression of a person as a subject in the captured image.

In the image processing apparatus according to one aspect of the present invention, when the predetermined organ includes a plurality of organs and the organ detection unit detects a region of the plurality of organs, the feature amount specifying unit is the organ detection unit. It is characterized in that the feature amount in each of the regions of the plurality of organs detected by is specified, and the feature amount in the region of the plurality of organs is specified based on the identified plurality of feature quantities.
According to one aspect of the present invention, when a plurality of organ regions are detected from a human face region in a captured image, the plurality of organ regions (one face region) are based on the feature amount in each organ region. Features are specified. Therefore, when a plurality of organ regions are detected, it is possible to generate a correction function in consideration of the feature amount of each detected organ region.

In the image processing apparatus according to one aspect of the present invention, when the face detection unit detects a face region of a plurality of people, the organ detection unit may perform the above from each face region for each face region detected by the face detection unit. The region of the predetermined organ is detected, and the feature amount specifying unit specifies the feature amount in the region of the predetermined organ detected by the organ detection unit for each face region detected by the face detection unit, and the correction function generation unit. Is characterized in that, for each face region detected by the face detection unit, a correction function for correcting the feature amount specified by the feature amount specifying unit to a target value stored in the storage unit is generated. do.
According to one aspect of the present invention, when a plurality of face regions are detected from a captured image, each process is performed for each face region (subject person), so that a correction function is generated for each subject person. can do.

The image processing apparatus according to one aspect of the present invention is characterized in that the predetermined organ includes at least one of white eyes, teeth, nose and cheeks.
According to one aspect of the present invention, a correction function is generated based on the organ regions of white eyes, teeth, nose, and cheeks in the facial region in the captured image. Since the regions of the white eyes, teeth, nose, and cheek organs are not easily affected by the imaging conditions and the like, an appropriate correction function can be generated based on each organ region. Therefore, the captured image can be appropriately corrected by using the correction function generated based on the region of each of these organs.

The image processing method according to one aspect of the present invention is an image processing method using an image processing device that performs image processing on a captured image of a person, wherein the image processing device detects a human face region from the captured image and detects the face. The area of the predetermined organ of the face is detected from the area, the feature amount in the detected predetermined organ area is specified, and the specified feature amount is stored in the storage unit as the target value related to the image in the predetermined organ of the human face. It is characterized by generating a correction function for correction to.
According to one aspect of the present invention, it is possible to generate a correction function that can appropriately correct not only the region of the predetermined organ but also the entire captured image based on the region of the predetermined organ of the human face in the captured image. ..

The program according to one aspect of the present invention is a program for causing a computer having a storage unit for storing a target value related to an image in a predetermined organ of a human face to process a captured image of a person. A human face region is detected from the captured image, a region of the predetermined organ is detected from the detected face region, a feature amount in the detected predetermined organ region is specified, and the specified feature amount is stored in the storage unit. It is characterized in that a process of generating a correction function for correcting to a certain target value is executed.
According to one aspect of the present invention, the above-mentioned image processing apparatus can be realized by a computer by installing it in a computer and executing it.

The embodiments disclosed this time should be considered to be exemplary in all respects and not restrictive. The scope of the present invention is not shown above, but is indicated by the scope of claims, and is intended to include all modifications within the meaning and scope equivalent to the scope of claims.

1 Computer (image processing device)
10 Control unit 11 Storage unit 21 Face detection unit 22 Organ detection unit 23 Feature quantity identification unit 24 Correction function generation unit 25 Correction unit 27 Impression identification unit 14a Camera

Claims (6)

In an image processing device that processes images captured by humans,
A storage unit that stores target values related to images in a predetermined organ of a human face,
A face detector that detects the human face area from the captured image,
An organ detection unit that detects a region of the predetermined organ from the face region detected by the face detection unit, and an organ detection unit.
A feature amount specifying unit that specifies a feature amount in a predetermined organ region detected by the organ detection unit, and a feature amount specifying unit.
A correction function generation unit that generates a correction function for correcting the feature amount specified by the feature amount specification unit to a target value stored in the storage unit, and a correction function generation unit.
Using the correction function generated by the correction function generation unit, a correction unit that corrects a human clothing area included in the captured image, and a correction unit .
An impression specifying unit that specifies the impression of the person in order to specify the conversation content to be output to the person included in the captured image based on the clothing area of the person included in the captured image corrected by the correction unit.
An image processing device characterized by comprising. The predetermined organ includes a plurality of organs and includes a plurality of organs.
When the organ detection unit detects a region of the plurality of organs, the feature amount specifying unit identifies a feature amount in each of the regions of the plurality of organs detected by the organ detection unit, and the specified plurality of features. The image processing apparatus according to claim 1 , wherein the feature amount in the region of the plurality of organs is specified based on the amount. When the face detection unit detects the face areas of a plurality of people,
The organ detection unit detects a region of the predetermined organ from each face region for each face region detected by the face detection unit.
The feature amount specifying unit specifies the feature amount in the region of the predetermined organ detected by the organ detection unit for each face region detected by the face detection unit.
The correction function generation unit generates a correction function for correcting the feature amount specified by the feature amount specifying unit to a target value stored in the storage unit for each face region detected by the face detection unit. The image processing apparatus according to claim 1 or 2 , wherein the image processing apparatus is characterized by the above. The image processing apparatus according to any one of claims 1 to 3 , wherein the predetermined organ includes at least one of white eyes, teeth, nose and cheeks. In an image processing method using an image processing device that processes an image of a person's image,
The image processing device
Detects the human face area from the captured image and
Detects the area of the predetermined organ of the face from the detected facial area,
Identify the features in the area of the detected organ,
A correction function for correcting the specified feature amount to the target value related to the image in the predetermined organ of the human face stored in the storage unit is generated.
Using the generated correction function, the clothing area of the person included in the captured image is corrected .
An image processing method comprising specifying an impression of the person in order to specify a conversation content to be output to the person included in the captured image based on the clothing area of the person included in the corrected image . A program for causing a computer having a storage unit for storing a target value related to an image in a predetermined organ of a human face to process a captured image of a person.
To the computer
Detects the human face area from the captured image and
The area of the predetermined organ is detected from the detected face area, and the area is detected.
Identify the features in the area of the detected organ,
A correction function for correcting the specified feature amount to the target value stored in the storage unit is generated.
Using the generated correction function, the clothing area of the person included in the captured image is corrected .
A program characterized by executing a process of identifying the impression of the person in order to specify the conversation content to be output to the person included in the captured image based on the clothing area of the person included in the corrected image. ..

Images