JP2010257265A - Display control device and method of controlling operation of the same - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a display control device that determines the identity of a specific person in a relatively faster manner. <P>SOLUTION: A subject is imaged at predetermined intervals, and a facial image part is detected in a subject image acquired by imaging the subject (steps 61 and 62). It is determined whether or not the facial image part detected by the new imaging exists at a position corresponding to a position of the facial image part detected from the previously imaged subject image (step 64). If the facial image part exists at the corresponding position (YES in step 64), it is determined that a person whose facial part image has been detected by the new imaging and a person identified by the previous imaging are identical with each other (step 65). If the facial image part does not exist at the corresponding part (NO in step 64), a process of determining a specific person from the facial image part detected by the new imaging (step 66). <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a display control device and an operation control method thereof.

  With a digital still camera, when a person is imaged, a face image portion can be detected from the captured person image. A frame surrounding the detected face image portion is displayed, and automatic exposure processing is performed so that the brightness of the image in the frame becomes appropriate. In addition, there is a technique for detecting a face image portion of a specific person instead of simply detecting a face image portion (Patent Document 1). By detecting the face image portion of a specific person, it is possible to control the face image portion of the specific person so as to have appropriate brightness.

  However, a person recognition process that detects a face image portion of a specific person often takes more time than a process that simply detects a face image portion.

JP 2009-17135

  It is an object of the present invention to determine that a previously imaged person or animal is identical to a newly imaged person or animal relatively quickly.

  The present invention provides a display control apparatus for controlling a display device so as to display a subject image obtained by imaging a subject at a fixed period on the display screen (the image may be the same as or not the same as the imaging period). A face image portion detecting means for detecting a face image portion from a subject image of a specific frame of the image, a position of the face image portion detected by the face image portion detecting means, and the specific identification of the subject images of the plurality of frames. Determining means for determining whether or not the position of the face image portion included in the subject image of the previous frame (not necessarily the previous frame) of the one frame of the subject image corresponds, the face image by the correspondence determining unit When it is determined that the position of the part corresponds, the person or animal specified by the face image part included in the subject image of the previous frame and the face image part detection means A determination unit that determines that the person or animal specified by the issued face image portion is the same, and the face image portion in response to a determination that the position of the face image portion does not correspond by the correspondence determination unit; Recognizing means for recognizing and determining a specific person or animal having the face of the face image portion detected by the partial detecting means is provided.

  The present invention also provides an operation control method suitable for the display control device. That is, this method is an operation control method for a display control device for controlling a display device so as to display a plurality of frames of subject images obtained by continuously imaging a subject on a display screen. A face image portion is detected from a certain single frame of the subject image, and the determination means detects the position of the face image portion detected by the face image portion detection means and the specific image of the plurality of frame subject images. It is determined whether or not the position of the face image portion included in the subject image of the frame before the one frame subject image corresponds, and the determining means determines that the position of the face image portion corresponds by the correspondence determining means. Accordingly, the person or animal specified by the face image portion included in the subject image of the previous frame and the face image portion detected by the face image portion detecting means are used. And the recognition means detects the position of the face image portion by the correspondence determination means when the position of the face image portion is not determined to correspond. A specific person or animal having a face of the face image portion thus determined is recognized and determined.

  The present invention also provides an operation program suitable for the operation control method of the display control device.

  According to the present invention, a face image portion is detected from one subject image among a plurality of subject images obtained by continuous imaging. When the position of the detected face image portion corresponds to the position of the face image portion included in the subject image of the previous frame of that one frame subject image, the face image portion included in the subject image of the previous frame It is determined that the person or animal specified by is the same as the person or animal included in the image portion in the detected area. If the positions of these image parts do not correspond, a process of recognizing and determining a specific person or animal having the face of the detected face image part is performed. A particular person or animal can be determined to be identical relatively quickly.

  Determining means for determining whether or not a certain time or more has elapsed from the determination of the specific person or animal by the recognizing means in response to determining that the position of the face image portion corresponds by the correspondence determining means; and the determining means The recognition means is controlled to perform a process of determining a specific person or animal in response to the determination that a certain time or more has passed, and the determination means determines that a certain time or more has not passed. A first control unit that controls the determination unit may be further provided so that the same determination process is performed according to the above.

  The recognition means determines, for example, a specific person or animal based on a score obtained by scoring features of the specific person or animal. In this case, if the score for the face image portion detected from the subject image obtained by the imaging is larger than the score when the specific person or animal is determined before a certain time, the same determination is made. It is preferable to further include second control means for controlling the determination means so that processing is performed.

  When the recognition means is for determining a specific person based on a score obtained by scoring characteristics of the specific person or animal, when determining the specific person or animal before a certain time The determination means is configured so that the same determination processing is performed for a person specified by a face image having a large sum of the obtained score and the score for the face image portion detected from the subject image obtained by the imaging. You may make it further provide the 3rd control means to control.

It is a block diagram which shows the electric constitution of a digital still camera. It is an example of a feature / name table. The relationship between the position of the face image and the name of the face image is shown. It is an example of a to-be-photographed image. It is an example of a to-be-photographed image. It is an example of a to-be-photographed image. It is a flowchart which shows the process sequence of a digital still camera. It is a time chart of a person determination process. It is a flowchart which shows the process sequence of a digital still camera. It is a flowchart which shows the process sequence of a digital still camera. It is an example of a candidate table. It is an example of a candidate table. It is a flowchart which shows the process sequence of a digital still camera.

  FIG. 1 shows an embodiment of the present invention and is a block diagram showing an electrical configuration of a digital still camera 1.

  The entire operation of the digital still camera 1 is controlled by the CPU 2.

  The digital still camera 1 includes a memory 23 in which an operation program described later and other data are stored. The operation program may be written in the memory card 28 or the like, read out from the memory card 28 and installed in the digital still camera 1, or may be pre-installed.

  The digital still camera 1 includes a shutter release button 3 and a mode switch 4. A signal indicating depression of the shutter release button 3 is input to the CPU 2. The mode switch 4 selects an imaging mode or a reproduction mode, and can selectively turn on the switch S1 or S2. The imaging mode is set when the switch S1 is turned on, and the reproduction mode is set when the switch S2 is turned on.

  A diaphragm 5 and a zoom lens 6 are provided in front of an image sensor 7 such as a CCD. The aperture value of the aperture 5 is determined by the motor 13 controlled by the motor driver 10. The zoom position of the zoom lens 6 is determined by a motor 14 controlled by a motor driver 11.

  When the imaging mode is set by the mode switch 4, the light beam representing the subject image that has passed through the aperture 5 is imaged on the light receiving surface of the image sensor 7 by the zoom lens 6. The imaging device 7 is controlled by the timing generator 12, and the subject is imaged at a constant period (for example, 1/30 second period). A video signal representing a subject image is output from the image sensor 7 at a constant period and input to a CDS (correlated double sampling) circuit 8. The video signal subjected to correlated double sampling in the CDS circuit 8 is converted into digital image data in the analog / digital conversion circuit 9.

  The digital image data is input to the image signal processing circuit 16 by the image input controller 15 and subjected to predetermined signal processing such as gamma correction. The digital image data is written in a VRAM (Video Random Access Memory) 24, then read out and given to the image display device 27, so that it is displayed as a moving image on the display screen of the image display device 27 (with a constant cycle). Displayed).

  The digital still camera 1 according to this embodiment can detect a face image portion included in a subject image obtained by imaging. For this purpose, the digital still camera includes a face detection circuit 18. In particular, in this embodiment, not only the face image portion can be detected from the subject image, but also the person (animal) of the face image portion can be detected. Specifically, as described later, in this embodiment, a frame surrounding the face image portion is displayed on the subject image so as to specify the detected face image portion.

  Digital image data obtained by imaging and data indicating the position of the detected face image portion (data indicating the position of the region) are input to an AF (autofocus) detection circuit 21. In the AF detection circuit 21, the zoom position of the zoom lens 6 is controlled so that the detected face image portion is in focus. The digital image data obtained by imaging and the data indicating the position of the detected face image portion (data indicating the position of the area) are also input to the AE (automatic exposure) / AWB (automatic white balance) detection circuit 22. To do. In the AE / AWB detection circuit 22, the stop of the stop 5 is determined so that the brightness of the detected face image portion is an appropriate brightness. Further, white balance adjustment is also performed in the AE / AWB detection circuit 22.

  When the shutter release button 3 is pressed, the image data obtained by imaging is input to the compression processing circuit 17. The image data that has been subjected to the predetermined compression processing in the compression processing circuit 17 is input to the video encoder 20 and encoded. The encoded image data is recorded on the memory card 28 under the control of the memory controller 26.

  When the playback mode is set by the mode switch 4, the image data recorded on the memory card 28 is read. An image represented by the read image data is displayed on the display screen of the image display device 27.

  FIG. 2 is an example of a feature / name table.

  The feature / name table stores data representing the features of a person's face corresponding to the name of the person. The feature / name table also stores a person number for identifying the person. In the case of detecting an animal's face, similarly, data representing the characteristics of the animal's face and its animal name are stored in the feature / name table.

  By referring to the feature / name table, the person in the face image portion detected from the subject image can be known.

  FIG. 3 is an example of the detected specific person table.

  Specifically, as will be described later, in this embodiment, the subject is imaged at a constant period, and the face image portion is detected from the subject image obtained by the imaging. A specific person is detected from the face image detected at a predetermined timing. When a face image portion is detected from the subject image obtained by imaging the subject after the specific person is detected, the detected face image portion and the position of the face image portion when the specific person is detected If they match, it is determined that the detected face image portion is the same person without recognizing a specific person.

  The detected specific person table is used for such determination processing of the same person, and the position of the detected face image partial area of the specific person and the name of the specific person are stored in an updatable manner. When a specific person is newly determined, the name of the determined specific person and the position of the face image portion are stored in the detected specific person table.

  FIG. 4 shows an example of a subject image (a subject image on which face image detection is performed and a subject image of a specific frame among a plurality of subject images obtained by continuous imaging) 30 obtained by the current imaging. It is.

  It is assumed that the subject image 30 this time includes a person image 31. By performing face image detection processing on the subject image 30, a face image portion 32 is detected.

  FIG. 5 is an example of a subject image 40 obtained by previous imaging (subject image captured before a certain one subject image among a plurality of subject images obtained by continuous imaging) 40.

  It is assumed that the person image 41 is also included in the previous subject image 40. By performing face image detection processing on the subject image 40, a face image portion 42 is detected. For the previous subject image 40, it is assumed that the feature of the detected face image portion 42 is stored in the feature / name table, and a specific person is determined. Then, the name of the determined specific person and the position of the face image area are stored in the detected specific person table as described above.

  If the position of the face image portion 32 detected in the current subject image 30 shown in FIG. 4 is substantially the same as the position of the face image portion 42 detected in the previous subject image 40 shown in FIG. It can be considered that the person of the face image portion 32 included in the subject image 30 of this time and the person of the face image portion 42 included in the previous subject image 40 are the same. The specific person of the face image portion 32 of the current subject image 30 is determined without performing the process of determining the specific person of the face image portion 32 detected in the current subject image 30.

  FIG. 6 is another example of the previous subject image 50.

  The subject image 50 shown in FIG. 6 includes a person image 51 and a car image 53. When face image portion detection processing is performed on the subject image 50, the face image portion 52 of the person image 51 is detected. Since the position of the face image portion 52 is different from the position of the face image portion 32 detected in the current subject image 30 shown in FIG. 4, the person in the face image portion 51 and the person in the face image portion 32 are different. It is determined. For the subject image 50, a process for determining a specific person is performed again.

  FIG. 7 is a flowchart showing the processing procedure of the digital still camera.

  This embodiment is used for moving image display (so-called through-image display) performed for confirming the angle of view at the time of imaging. However, when reproducing a plurality of framed subject images, Is also applicable. In particular, in this embodiment, as described above, when the face image portion is detected from the subject image obtained in the current imaging, the position of the detected face image portion is the subject previously obtained in the imaging. If it is close to the position of the face image part included in the image, the person in the face image part included in the current subject image and the person in the face image part included in the previous subject image are the same person It is what.

  A subject is imaged at a fixed period, and a subject image is obtained (step 61). A face image portion is detected from the subject image obtained by the current imaging (step 62). Further, the position of the face image portion area included in the subject image obtained by the previous imaging is read from the detected specific person table (step 63).

  If the position of the face image portion detected in the current imaging corresponds to the position of the area read from the detected specific person table (if close) (YES in step 64), the subject obtained by the current imaging It is determined that the person of the face image portion included in the image is the same person as the person stored in the detected specific person table (step 65).

  If the position of the face image portion detected in the current imaging does not correspond to the position of the area read from the detected specific person table (NO in step 64), the subject between the previous imaging and the current imaging There is a high possibility that the person is replaced. For this purpose, a process of determining a person from the face image portion detected from the subject image obtained by the current imaging is performed (step 66).

  Data indicating the determined person and the position of the detected face image portion are stored in the detected specific person table (step 67). The subject image obtained by imaging is displayed on the display screen of the display device 27 so that a frame is displayed at the position of the detected face image portion (step 68).

  8 and 9 show another embodiment. This embodiment is included in the subject image obtained by the current imaging even when the face image portion exists in the corresponding region as described above when a certain time has passed since the previous specific person determination process. The person is determined from the face image portion. This is to take into consideration that even if a certain time or more has elapsed, even if a face image portion exists in the corresponding region, the person is often not the same person.

  FIG. 8 is a time chart of the person determination process.

  At time t0, a face image portion is detected from the subject image obtained by imaging, and is determined from a specific person from the detected face image portion. A frame of the face image portion of the determined person is displayed.

  From time t1 to time t3, the face image portion is detected from the subject image obtained by imaging. If the detected face image portion is near the position of the face image portion detected at time t0, the time It is assumed that the subject of the same person as the person determined at t0 is captured. The person determination process using the feature amount of the face image portion is not performed.

  Similarly, when a predetermined time elapses from the person determination process using the feature value of the previous face image part, the person determination process is performed again using the feature value of the face image part.

  FIG. 9 is a flowchart showing the processing procedure of the digital still camera. In this figure, the same processes as those shown in FIG.

  If the face image portion detected from the subject image obtained by the current imaging corresponds to the face image portion obtained in the previous imaging (YES in step 64), the specific person is determined using the face image portion. It is determined whether or not a certain time (a certain frame) has elapsed since the previous specific person determination process (step 71).

  If a certain period of time has elapsed (YES in step 71), even if the face image part obtained by the current imaging is close to the face image part obtained by the previous imaging, the person of the previous subject There is a high possibility that it is different from the person of the subject this time. For this purpose, a process of newly determining a specific person using the face image portion obtained by the current imaging is performed (step 66).

  If the fixed time has not passed (NO in step 71), if the position of the face image part obtained by the current imaging is close to the position of the face image part obtained by the previous imaging, Since it is considered that the person of the subject this time is the same, it is determined to be the same person (step 65).

  FIG. 10 shows still another embodiment. In this embodiment, a score when a specific person is determined using a face image portion is used. If the previous score is lower than the current score, the specific person is determined again using the newly detected face image portion. If the previous score is higher than the current score, the specific image determined by the previous imaging is determined. It is determined that the person is the same person. When the process of determining a specific person using the face image portion is performed, a score is calculated according to the feature amount of the face as described above, and it corresponds to the feature amount that gives a large score among the calculated scores. Needless to say, the person is determined to be the person in the face image portion.

  FIG. 10 is a flowchart showing the processing procedure of the digital still camera. In this figure, the same processes as those shown in FIG. 7 and FIG.

  When a certain period of time has passed since the previous specific person determination process using the face image portion (YES in step 71), the face image portion was detected from the subject image obtained by the current imaging and detected. A score for the face image portion is calculated (step 72). The previous score (not to mention stored in the memory) is compared with the current score (step 73). If the previous score is lower than the current score (YES in step 73), a process of determining a specific person anew using the detected face image is performed (step 74). If the previous score is higher than the current score (NO in step 73), the person in the previous subject image and the person in the current subject image are determined to be the same person (step 65).

  11 to 13 show still another embodiment.

  As described above, from the detected face image portion, in order to specify the person of the face, a score representing the similarity between the feature amount of the detected face image portion and the feature amount of the face of the specific person is calculated. Is done. When thinking only within a certain subject image, the score for the face image portion existing in the subject image is calculated, and the person with the feature amount that gives the large score is considered to be the person of the face image portion. In this embodiment, the person is determined using the sum of the score calculated in the subject image obtained by the previous imaging and the score obtained in the subject image obtained by the current imaging.

  11 and 12 are examples of candidate tables.

  The candidate table shows person candidate candidates of the face image portion included in the subject image obtained by imaging. FIG. 11 shows the person candidates of the face image part included in the subject image obtained by the previous imaging, and FIG. 12 shows the person candidates of the face image part included in the subject image obtained by the current imaging. Show.

  The candidate table includes candidate numbers, person numbers, and scores. The candidate number is a list of candidates that are likely to be human faces in the detected face image portion in descending order of score. The person number is a number for identifying the person of the face in the face image portion. Any code that can be identified even if it is not a number is acceptable. The score indicates the similarity between the feature amount of the face of the person identified by the person number and the feature amount of the face image portion detected in the subject image. In the candidate table shown in FIG. 11, person number “32” has the highest score, and person number “12” has the next highest score. In the candidate table shown in FIG. 12, the score of person number “12” is the highest, and the score of person number “24” is the next highest.

  In this embodiment, the person who gives the score having the highest sum of the score of the previous subject image and the score of the current subject image is determined as the face person of the face image portion detected from the current subject image. In the example shown in FIGS. 11 and 12, the sum of the scores of the person number “12” is 93 + 88 = 181, the sum of the scores of the person number “15” is 45 + 64 = 109, and the score of the person number “32” is The sum is 95, the sum of the scores of the person number “5” is 41, the sum of the scores of the person number “9” is 32, the sum of the scores of the person number “26” is 25, and the person number The sum of the scores of “8” is 15. Therefore, since the person number with the highest sum of scores is “12”, it is determined that the face image portion detected from the subject image obtained by the current imaging belongs to the person with the person number “12”.

  FIG. 13 is a flowchart showing the processing procedure of the digital still camera. In this figure, the same processes as those shown in FIG. 7, FIG. 9, and FIG.

  If a predetermined time has elapsed since the previous specific person determination process (YES in step 71), as described above, the face image partial detection process of the current subject image is performed, and the previous score and current time are determined for each person number. The sum with the score is calculated. The person identified by the person number with the highest sum of the calculated scores is determined as the specific person (step 75).

  Needless to say, each table described above is stored in the memory 23.

  Each face image portion can be processed in the same manner as described above not only when one face image portion is detected in the subject image but also when a plurality of face image portions are detected.

  In the embodiment described above, the position of the face image portion detected by the previous imaging is stored, but the size and orientation of the face may be stored in addition to the position of the face image portion. . In addition to the name that identifies the person, the age of gender, etc. may be stored, the number of times corresponding to the face image portion detected from the previously captured subject image, the timing for determining the specific person, etc. It may be.

  In the above-described embodiment, the moving image display for confirming the angle of view at the time of imaging in the digital still camera is performed. However, the present invention is not limited to the digital still camera, but is a movie video camera, a digital movie / Needless to say, it can also be applied to still cameras.

1 Digital still camera (imaging device)
2 CPU (decision means, person recognition means)
18 Face detection circuit
19 Area detection circuit

Claims (6)

In a display control device for controlling a display device to display a subject image obtained by continuously imaging a subject at a constant cycle on a display screen,
A face image portion detecting means for detecting a face image portion from a certain one frame subject image among the plurality of frame subject images;
The position of the face image portion detected by the face image portion detection means and the position of the face image portion included in the subject image of the frame before the certain one subject image among the plurality of subject images. Judgment means for judging whether to correspond,
In response to the fact that the position of the face image portion is determined to correspond by the correspondence determination means, the person or animal specified by the face image portion included in the subject image of the previous frame and the face image portion detection means Determining means for determining that the person or animal specified by the detected face image portion is the same, and the face image portion in response to the fact that the position of the face image portion has not been determined to correspond by the correspondence determining means; Recognition means for recognizing and determining a specific person or animal having the face of the face image portion detected by the partial detection means;
A display control device. Determining means for determining whether or not a certain time or more has elapsed from the determination of the specific person or animal by the recognizing means in response to determining that the position of the face image portion corresponds by the correspondence determining means; and the determining means The recognition means is controlled to perform a process of determining a specific person or animal in response to the determination that a certain time or more has passed, and the determination means determines that a certain time or more has not passed. First control means for controlling the determination means so that the same determination processing is performed according to
The display control apparatus according to claim 1, further comprising: The recognition means determines a specific person or animal based on a score obtained by scoring characteristics of the specific person or animal.
When the score for the face image portion detected from the subject image obtained by the above imaging is larger than the score when it is determined as a specific person or animal before a certain time, the same determination process is performed. Second control means for controlling the determining means as
The display control device according to claim 2, further comprising: The recognition means is for determining a specific person based on a score obtained by scoring characteristics of the specific person or animal.
It is specified by a face image having a large sum of a score obtained when a specific person or animal is determined before a certain time and a score for a face image portion detected from the subject image obtained by the above imaging. Third control means for controlling the determination means so that the same determination processing is performed for a person
The display control apparatus according to claim 2, further comprising: In an operation control method of a display control device for controlling a display device to display a plurality of subject images obtained by continuously imaging a subject on a display screen,
A face image part detecting means detects a face image part from a subject image of one specific frame among the plurality of subject images;
The position of the face image portion detected by the face image portion detection means and the face image portion included in the subject image of the frame before the certain one subject image among the plurality of subject images. To determine whether or not
When the determining means determines that the position of the face image portion corresponds to the position of the face image portion, the person or animal specified by the face image portion included in the subject image of the previous frame and the face image are determined. Determining that the person or animal identified by the face image portion detected by the partial detection means is identical;
The recognizing means recognizes a specific person or animal having the face of the face image portion detected by the face image portion detecting means when the correspondence determining means does not determine that the position of the face image portion corresponds. To decide,
An operation control method for a display control device. A program for controlling a display control device for controlling a display device so as to display a plurality of subject images obtained by continuously imaging a subject on a display screen,
The face image portion is detected from the subject image of one specific frame among the subject images of the plurality of frames,
It is determined whether or not the position of the detected face image portion corresponds to the position of the face image portion included in the subject image of the frame in front of the certain specific subject image of the plurality of subject images. ,
When it is determined that the position of the face image portion corresponds, the person or animal specified by the face image portion included in the subject image of the previous frame and the person specified by the detected face image portion or Determine that the animal is identical,
A computer readable control for controlling the display control device to recognize and determine a specific person or animal having the face of the detected face image portion when the position of the face image portion is not determined to correspond program.

Images