JP5383207B2 - Information processing apparatus, control method, computer program, and storage medium - Google Patents

Description

  The present invention relates to an image and video display method in an information processing apparatus capable of subject recognition.

  2. Description of the Related Art In recent years, imaging optical system imaging devices such as digital cameras and video cameras detect a human face area when the subject is a person, and have a function of focus adjustment and exposure control based on the detected face area. Proposed. In many of the proposals, a face area as a main face is determined from a plurality of detected face areas, and the control of the camera is changed so that the focus and exposure of the main face are optimized. When a plurality of human faces are detected, it is also desired to optimize the focus and exposure of the detected plurality of face areas.

  On the other hand, many techniques for extracting face feature points of a person from a photographed image and comparing the extracted feature points with face information data related to the person to be recognized have been proposed. With this technique, the imaging apparatus can recognize one or more subjects being photographed and can change the control of the camera in accordance with the subjects. In general, in order to perform face recognition, it is necessary for an imaging apparatus to register face information data such as a face feature point, a face size, and a face image in advance in a database or the like. In Cited Document 1, a technique is proposed in which a plurality of face images with different shooting conditions (for example, face orientation, lighting conditions, etc.) of the same person are classified and registered for each subject.

JP 2007-150603 A

  In Cited Document 1, in order to increase the accuracy of face recognition, the imaging device registers face images taken under various shooting conditions in groups for each subject. However, since only the face information of the same person is registered in one group, the face information of a plurality of persons is not managed in a group.

  In addition, when a large number of people are shown, the face recognition function cannot search for a plurality of specified people on the screen, and the photographer can instantly determine who is and who is not in the registered group. It is difficult to distinguish.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide photographing information useful to a photographer.

In order to achieve the above object, an information processing apparatus of the present invention comprises the following arrangement.
  An information processing apparatus having a storage unit that stores a face image of a person and facial feature information included in the face image in a group for a plurality of persons in advance, the acquisition unit acquiring an image, and the acquisition unit Face detection means for detecting a face area of a person from the acquired image and extracting face feature information of the face area, face feature information of a plurality of persons, and face feature information detected by the face detection means And judging means for judging whether or not the detected face is any one of a plurality of persons, and when the detected face is judged to be any one of a plurality of persons And a combining unit that combines the face images of the plurality of persons with the images acquired by the acquiring unit and displays the combined images on the display unit. Face images and face images of undetected people And having a synthesizing means to vary the manner of display.

According to the present invention, when a plurality of subjects are recognized from the shooting screen by the subject recognition function, they are registered in advance in a group, and all the images of the plurality of subjects registered in the grouping are displayed and recognized easily. Identify the subject. Therefore, the person designated by the photographer on the screen can be easily found. In addition, information on group members registered in a part of the shooting screen (for example, the number of shot images and shooting time) is displayed in an easy-to-understand manner. Can be easily compared.

  FIG. 1 is a diagram showing the configuration of a video camera that is an embodiment of the present invention. In this embodiment, a video camera which is one of information processing apparatuses will be described. However, the present invention can also be applied to other imaging apparatuses such as a digital still camera.

  The imaging generation unit 100 photoelectrically converts the subject image. As shown in FIG. 1, an optical system, a lens driving source, and a camera signal processing system are provided.

  The imaging optical system includes a first fixed lens group 101, a zoom lens 102 that is a lens group that performs zooming, a diaphragm 103, a second fixed lens group 104, a so-called competition that corrects the focal adjustment function and movement of the focal plane due to zooming. A focus lens 105 having a function is provided.

  The zoom lens drive source 110 drives the zoom lens 102, and the focus lens drive source 111 drives the focus lens 105. The zoom lens drive source 110 and the focus lens drive source 111 each have a motor and a driver unit. The zoom lens drive source 110 and the focus lens drive source 111 drive each lens based on the target position and drive speed calculated by the camera microcomputer 112.

  The rear stage of the focus lens group includes an image sensor 106 such as a CMOS sensor or a CCD sensor. The CDS / AGC 107 samples and holds the image signal output from the image sensor 106 to obtain an appropriate signal level.

  The camera signal processing circuit 108 converts the output signal of the CDS / AGC 107 into a corresponding signal to be input to a display unit 118 such as a liquid crystal display or a finder having a function of displaying, and an image storage unit 109 described later.

  The image storage unit 109 records a moving image or a still image, and a magnetic tape, a semiconductor memory, a DVD, or the like is used as a recording medium. The image storage unit 109 also specifies an address of a memory area such as a semiconductor memory or a DVD, and records an output signal of the camera signal processing circuit 108. Moreover, it stores as imaging information for every photographed person.

  The camera microcomputer 112 displays an arbitrary captured image on the display unit 118 by designating and accessing an address. The camera microcomputer 112 controls the operation of each part of the camera according to a program stored in a ROM (not shown). For example, the camera microcomputer 112 drives the focus lens 105 by a known contrast AF method based on the image signal output from the camera signal processing circuit 108 and performs exposure control by focus adjustment or a known AE calculation.

  The face detection unit 113 of the camera microcomputer 112 extracts facial feature information of the photographed person from the face area portion in the image signal output from the camera signal processing circuit 108.

  In addition, the face detection unit 113 extracts face information from the captured image recorded in the image storage unit 109 and detects the face area, the face size, and the like of the subject. The face feature information means the detected face size, the positional relationship of feature points such as eyes, nose, and mouth, and a face image (index image) indicating a face area extracted from the selected image.

  The face determination unit 114 performs individual identification of a target person to be photographed from these face feature information by a known technique, and acquires the acquired face feature information of the person and other stored face feature information. To determine whether the person is the same person as the other face information.

  The face determination unit 114 performs face recognition processing for determining whether or not the face of the person in the shooting screen detected by the face detection unit 113 is the face of a person registered in the face information registration unit 119. Depending on the result, focus adjustment and exposure compensation are performed.

  In this embodiment, the face detection unit 113 and the face determination unit 114 are described so as to be realized by software in the camera microcomputer 112, but face detection, face recognition, and face extraction are performed by hardware separately from the camera microcomputer 112. These functions may be realized.

  In addition, the camera microcomputer 112 identifies a member of a group registered in the face information registration unit 119 from images and videos already captured and recorded in the image storage unit 109. The camera microcomputer 112 includes an information recording unit 115 that calculates, for each person, the number of images captured in the past of each person in the target group, video time, and the like (hereinafter referred to as shooting information).

  The shooting information calculated by the information recording unit 115 is sent to the image synthesis unit 117 to be combined with a part of the captured image as shooting information.

  The camera microcomputer 112 includes a frame display generation unit 116. The frame display generation unit 116 includes a circuit that generates a display frame signal such as a display surrounding the recognized human face (hereinafter simply referred to as a face frame) in accordance with the processing result of the face detection unit 113. Further, the frame display generation unit 116 combines the video signal output from the camera signal processing circuit 108 by the image combining unit 117 and the shooting information signal output from the information recording unit 115 described above, and outputs the combined signal to the display unit 118. .

  For example, the frame display generation unit 116 changes the color of the frame so that the face frame of a person who is not registered in the face information registration unit 119 is displayed in green, and the face frame of a registered person is displayed in red. Can do. The display by the color of the face frame allows the user to know the state of face detection and face recognition. Further, the shape and state of the frame may be changed such that the face frame during face detection is displayed in a single manner and the face frame during face recognition is displayed in a double manner. Further, the frame display generation unit 116 may generate a display frame by changing the color of the face frame for each person registered in the face information registration unit 119.

  The face information registration unit 119 is a face information registration unit that associates the person detected by the face detection unit 113 and registers the feature information of the face. Face information of a person previously selected by the user is grouped and registered in the face information registration unit as group information.

  FIG. 2 shows a face frame display. With reference to this figure, an example of the subject registration method in the face information registration unit 119 will be described. First, an image recorded in the face information registration unit 119 or an image / video showing a subject to be registered is selected from the video. Then, as shown in FIG. 2, the face detection unit 113 displays a face frame 201 on the face of the person on the image. Then, the user is made to select a face to be registered from the face frame.

  FIG. 3 shows a face frame selection screen. As an example of selecting and registering, a user operates an operation button or a cross key generally provided in a camera or a video camera, and sequentially moves a selection frame 202 shown in FIG. 3 to register a face of a subject to be registered. Further, the user may directly register a subject to be selected using a touch panel provided on a liquid crystal panel mounted on the video camera.

  As described above, the user registers the selection of the subject to be registered in association with one group from a plurality of groups. By repeating this registration, a plurality of selected subjects can be registered in the face information registration unit 119 as a group.

  FIG. 4 is a flowchart showing processing of the camera microcomputer 112. This process is executed according to a computer program (not shown) stored in the camera microcomputer 112.

  In step S201, the camera microcomputer 112 determines whether or not this mode is valid. If valid, the process proceeds to step S202, and if invalid, the process is terminated.

  In step S202, the camera microcomputer 112 synthesizes the list of face group members selected from the groups registered in the face information registration unit 119 in advance with the image synthesis unit 117 and the screen being imaged by the image synthesis unit 117, and displays the display unit. 118.

  FIG. 5 is a diagram illustrating a list display example of face group members. In FIG. 5, all members of the selected group list are displayed as a member list display 203 at the bottom of the screen. The groups displayed here are those previously selected by the user and stored in the face information registration unit 119.

  FIG. 6 is a diagram illustrating an example of a selection screen for selecting group lists 1 to 3 in which a plurality of face group members are collected. As shown in FIG. 6, the camera microcomputer 112 displays a plurality of group lists registered in a display unit such as a liquid crystal panel. The user selects one group from the list by operating the marker 204 with an operation button or a cross key generally provided in a camera or video camera. In addition, the user may directly register a subject to be selected by using a touch panel provided on a liquid crystal panel mounted on the video camera.

  In step S203, the face determination unit 114 of the camera microcomputer 112 searches for a person's face on the shooting screen.

  If the camera microcomputer 112 determines in step S204 that a human face exists, the process proceeds to step S205. If no human face exists, the process ends.

  In step S <b> 205, the face determination unit 114 of the camera microcomputer 112 searches for a member in the group specified by the user stored in the face information registration unit 119 on the shooting screen.

  In step S206, the camera microcomputer 112 proceeds to step S207 if there is a member of the designated group on the shooting screen, and ends the process if it does not exist.

  In step S207, the frame display generation unit 116 makes the member face display form in the member list display 203 displayed on the shooting screen different in step S202, depending on whether the member exists on the shooting screen or not.

  FIG. 7 is a diagram illustrating a display example indicating that some persons in the face group list exist on the screen. The designated member list display 203 is displayed in step S202 at the bottom of FIG. Since member C and member D do not exist on the shooting screen, only members that do not exist in the group are displayed with changes such as changing the color of face display 206 in member list display 203 at the bottom of the screen. How to add changes is not limited to this. The faces of non-existing members are grayed out, displayed in a reduced size, displayed semi-transparently, or an icon indicating that they do not exist are displayed nearby. Various methods are conceivable. By changing the display method (display form) in this way, the user can recognize at a glance which face of the members in the specified group can be detected and who cannot be detected. Can do.

  In step S208, the frame display generation unit 116 displays the face frame display 207 on the face of the group member existing on the shooting screen.

  FIG. 8 is a diagram illustrating a face frame display example when a person in the face group list exists on the screen. The frame display generation unit 116 may display the face frame display 207 so that the same person in the member list display displayed on a part of the shooting screen matches the color or pattern.

  Thus, with the configuration of FIG. 8, the user can easily know which person in the group is located at which position on the shooting screen.

  After step S208, the frame display generation unit 116 causes the image synthesis unit 117 to synthesize the display image generated by the processing executed in this flow with the video signal output by the camera signal processing, and displays it on the display unit 118. To do.

  FIG. 9 is a flowchart showing processing of displaying photographing information by the camera microcomputer.

  In step S301, the camera microcomputer 112 determines whether to display the shooting information in a partial area on the shooting screen. If the camera microcomputer 112 determines in step S301 to display, the process proceeds to step S302, and if it is determined not to display, the process ends.

  In step S302, the information recording unit 115 searches for the group member registered in the face information registration unit 119 from the image data stored in the image storage unit 109, and calculates the number of registered images. .

In addition, the information recording unit 115 of the camera microcomputer 112 searches for the person registered in the face information registration unit 119 from the image data recorded in the image storage unit 109, and captures the number of registered images or the number of videos. The shooting time is calculated. In the case of video, the information recording unit 115 extracts group members registered in the face information registration unit 119 for each predetermined frame, and estimates a rough shooting time from the number of frames of the extracted members. Good.

  In step S303, the camera microcomputer 112 confirms whether or not the shooting information has been acquired. If it has been acquired, the process proceeds to step S304. If it has not been acquired, the process of step S302 is repeated.

  In step S304, the camera microcomputer 112 synthesizes the shooting frequency information with the member list by the image synthesizer 117, displays the synthesizer information on the display unit 118, and then ends the process.

  FIG. 10 is a diagram illustrating a display method of shooting frequency information that is shooting history information. As shown in FIG. 10, the camera microcomputer 112 displays the number of images recorded in the image storage unit 109, the time of the moving image, etc. near the face of the group member in the member list displayed in a part of the shooting screen. The face frame display 207 is performed. Then, the image information display generation process ends.

  After step S304, the display image generated by the processing executed in this flow is combined with the video signal output by the camera signal processing by the image combining unit 117 and displayed on the display unit 118.

  In this way, in the configurations of FIGS. 9 and 10, information of group members registered in a part of the shooting screen (for example, the number of shot images and shooting time) is displayed in an easy-to-understand manner. The photographer can easily compare the shooting information of the members in the group.

<Other embodiments>
The present invention is also achieved by supplying a program that realizes the functions of the above-described embodiments directly or remotely to a system or apparatus, and the system or apparatus reads and executes the code of the supplied computer program. The Accordingly, the computer program code itself installed in the computer in order to realize the functional processing of the present invention by the computer is also included in the technical scope of the present invention.

  In this case, the program may be in any form as long as it has a program function, such as an object code, a program executed by an interpreter, or script data supplied to the OS.

  Examples of the storage medium for supplying the program include a floppy (registered trademark) disk, a hard disk, an optical disk (CD, DVD), a magneto-optical disk, a magnetic tape, a nonvolatile memory card, and a ROM.

  Further, the functions of the above-described embodiments are realized by the computer executing the read program. In addition, based on the instructions of the program, an OS or the like running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments can also be realized by the processing.

Furthermore, a program that can be read from the storage medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. Thereafter, the CPU of the function expansion board or function expansion unit performs part or all of the actual processing based on the instructions of the program, and the functions of the above-described embodiments are realized by the processing.

It is a block diagram showing the Example of this invention. It is a figure showing the display of a face frame. It is a figure showing the selection screen of a face frame. It is a flowchart showing the process of a camera microcomputer. It is a figure which shows the list display example of a face group. It is a figure which shows the example of the selection screen of a face group. It is a figure (1) which shows the example of a display showing that the person in a face group list exists in a screen. It is FIG. (2) which shows the example of a face frame display when the person in a face group list exists in a screen. It is a flowchart showing the process of imaging | photography frequency display with a camera microcomputer. It is a figure showing the display method of imaging frequency information.

101 First lens group 102 Variable magnification lens 103 Diaphragm 104 Second lens group 105 Focus lens 106 Image sensor 107 CDSAGC
108 camera signal processing circuit 109 storage device 110 zoom drive source 111 focus drive source 112 camera microcomputer 113 face detection unit 114 face determination unit 115 information recording unit 116 frame display generation unit 117 image composition unit 118 display unit 119 face information registration unit 201 face Detection frame display 202 Face detection frame display used for group registration 203 Group list display 204 Marker display used for group selection 205 Group list display 206 Display example in group list display of group members not on the screen 207 Face frame display

Claims (9)

An information processing apparatus having a storage unit that stores a face image of a person and facial feature information included in the face image by grouping a plurality of persons in advance,
An acquisition means for acquiring an image;
A face detection means for detecting a face area of a person from the image acquired by the acquisition means and extracting facial feature information of the face area ;
Comparing the feature information of the extracted face by the feature information and the face detecting means face before Symbol plurality of persons, the detected face to determine whether any of the face of the plurality of persons and the judgment means that,
If the detected face is determined to be one of the faces of the front Kifuku number of the person, the display face images of the plurality of persons in the synthesis to the display means of the acquired images by the acquisition unit a synthetic means you, the face images of the plurality of persons, varying the Viewing manner in the human face image in which the face is not detected as the face image of the person detected by said face detecting means Combining means;
An information processing apparatus comprising: Of the faces detected by the face detection means, face information registration means for registering face images of the faces that are not the faces of the plurality of persons and their feature information in the group and storing them in the storage means The information processing apparatus according to claim 1, further comprising: The synthesizing unit further includes a face frame surrounding the face area of the detected face by the determining unit for the face determined by the determining unit as one of the faces of the plurality of persons. The information processing apparatus according to claim 1, wherein the information processing apparatus is combined with the acquired image. The image acquired by the acquisition unit is stored in the storage unit,
  The information processing apparatus further includes means for acquiring information indicating the shooting frequency of the person in the image stored in the storage means for each of the plurality of persons.
  The synthesizing unit further synthesizes information indicating the photographing frequency of each person together with the images acquired by the acquiring unit together with the face images of the plurality of persons.
The information processing apparatus according to any one of claims 1 to 3. 5. The information indicating the shooting frequency is the number of images stored in the storage unit including the face of the person or the shooting time of the video for each of the plurality of persons. The information processing apparatus described in 1. An image capturing unit that captures an image of the subject using the image capturing optical system and generates an image;
  The acquisition unit acquires an image generated by the imaging unit.
The information processing apparatus according to claim 1, wherein the information processing apparatus is an information processing apparatus. A control method of an information processing apparatus having a storage unit that stores a face image of a person and facial feature information included in the face image in a group for a plurality of persons in advance,
  An acquisition step in which the acquisition unit of the information processing apparatus acquires an image;
  A face detection step in which the face detection means of the information processing apparatus detects a human face area from the image acquired in the acquisition step, and extracts facial feature information of the face area;
  The determination unit of the information processing apparatus compares the facial feature information of the plurality of persons with the facial feature information detected in the face detection step, and the detected face is one of the plurality of persons. A determination step of determining whether the face is present;
  When the combining unit of the information processing apparatus determines that the detected face is any one of the plurality of persons, the face images of the plurality of persons are converted into images acquired in the acquisition step. A combining step of combining and displaying the plurality of person face images with a face image of a person whose face has been detected in the face detection step and a face image of a person who has not been detected in the face detection step; A synthesis process that makes the way different,
A method for controlling an information processing apparatus, comprising: Computer program for causing a computer to function as each unit except for the display means and the storage means of the information processing apparatus according to any one of claims 1 to 6.   A computer-readable storage medium storing the computer program according to claim 8.

Images