JP6033044B2 - Image display apparatus, control method thereof, control program, and imaging apparatus - Google Patents

Description

  The present invention relates to an image display device, a control method thereof, a control program, and an imaging device, and more particularly, to a technique for displaying specific information that defines a specific region of an image.

  In general, an imaging device such as a digital camera is provided with an image display device for displaying an image obtained as a result of photographing. In this type of imaging apparatus, a specific area of the subject is detected and a detection frame for specifying the specific area is displayed together with the image.

  For example, there is an imaging apparatus that detects a face area as a specific area of a subject and displays a detection frame indicating the face area on a display device (see Patent Document 1). In this case, if the attributes of the detection frame (such as the color and shape of the detection frame) are changed according to the facial feature value (such as the face angle) of the subject and displayed on the display device, the user can display the facial feature of the subject. Can be grasped intuitively.

  Furthermore, in addition to a face detection unit that detects a face area of a subject, an imaging device such as a monitoring camera that includes a human body detection unit that detects a human body part (human body area) of the subject is known.

  By the way, in an imaging device including a face detection unit and a human body detection unit, the face detection unit and the human body detection unit may operate independently and processing times thereof may be different. Then, a face detection frame and a human body detection frame are displayed on the display device according to the face detection result and the human body detection result, respectively.

  FIG. 11 is a diagram illustrating an example of display of a conventional face detection frame and human body detection frame.

  As illustrated, a face detection frame 201 is displayed on the screen of the display device 200, and a human body detection frame 202 is displayed so as to surround the face detection frame 201.

  Here, conventional display processing of the face detection frame and the human body detection frame will be described.

  FIG. 12 is a timing chart showing the display timing of the conventional face detection frame and human body detection frame. In FIG. 12, it is assumed that the face detection process is completed earlier than the human body detection process.

  At time 1, the subject D1 is captured, and the face detection process and the human body detection process related to the subject D1 are started. Subsequently, the face detection of the subject D1 is completed at time 2, and the face detection frame of the subject D1 is displayed on the screen. In addition, the subject D2 is captured at time 2, and the face detection process and the human body detection process related to the subject D2 are started.

  At time 3, the face detection of the subject D2 is completed, and the face detection frame of the subject D2 is displayed on the screen. In addition, the subject D3 is captured at time 3, and the face detection process and the human body detection process related to the subject D3 are started.

  At time 4, the face detection of the subject D3 is completed, and the face detection frame of the subject D3 is displayed on the screen. At this time, since the human body detection process related to the subject D1 is completed, the human body detection frame of the subject D1 is displayed on the screen. At time 4, the subject D4 is captured, and the face detection process and the human body detection process related to the subject D4 are started.

  In this way, the process is performed in the same manner until time (t + 2). That is, the same processing as that in time 4 is repeated until the release is performed. As a result, at time (t + 1), the face detection frame up to the subject Dt is displayed on the screen, and the human body detection frame up to the subject D (t-2) is displayed on the screen.

  FIG. 13 is a diagram illustrating an example of a face detection frame and a human body detection frame displayed on the screen by processing according to the timing chart illustrated in FIG. FIG. 13A shows a screen display at time t, and FIG. 13B shows a screen display at time (t + 2).

  As shown in FIG. 13A, when the face detection frame 201 of the subject 203 is displayed at time t, the human body detection frame 202 of the subject 203 is shown at time (t + 2) as shown in FIG. Will be displayed. That is, only the face detection frame 201 is displayed when the face detection is completed at the time t, and the human body detection frame 203 is displayed together with the face detection frame 202 when the human body detection is completed together with the face detection at the time (t + 2). It will be.

JP 2007-274207 A

  As described above, when the face detection unit and the human body detection unit operate independently and each processing time is different, only the detection frame of the area detected earlier (here, the face detection frame) is temporarily stored. May be displayed.

  If the time interval from when the face detection frame is displayed to when the human body detection frame is displayed is long, the user may feel uncomfortable.

  Further, as described above, when the time required for the face detection process and the human body detection process is different, if the subject is moving, the shift of the face detection frame and the shift of the human body detection frame are different.

  FIG. 14 is a diagram for explaining a shift between a conventional face detection frame and a human body detection frame. FIG. 14A is a diagram showing a state where only the face detection frame is displayed, and FIG. 14B is a diagram showing a state where the face detection frame and the human body detection frame are displayed.

  In FIG. 14, it is assumed that the subject 203 is stationary and the subject 204 is moving. In this case, when the time required for the face detection process and the human body detection process is different, first, the face detection frame 201 of the subject 203 is displayed at the time t, and the face detection frame 205 of the subject 204 is displayed. And

  Then, it is assumed that the human body detection frame 202 of the subject 203 is displayed and the face detection frame 206 of the subject 204 is displayed at time (t + n) (here, n = 2). In this case, since the human body detection process requires more time than the face detection process, if the subject moves like the subject 204, the human body detection frame 206 is set according to the time from the face detection process to the human body detection process. Actually, the image is displayed with a deviation from the position where the subject exists.

  As a result, it becomes difficult for the user to accurately visually recognize the moving subject by the human body detection frame.

  Accordingly, a first object of the present invention is to provide an image display device capable of simultaneously displaying a frame indicating a face region of a subject and a frame indicating a human body region on the screen, a control method thereof, a control program, and an imaging device. There is.

  A second object of the present invention is to provide an image display device, a control method thereof, a control program, and an imaging device capable of reducing a shift between a detection area and a detection frame in a moving subject. There is.

  In order to achieve the above object, an image display device according to the present invention, when displaying an image including a subject image on a display unit, a first display body that specifies a first region for the subject image, and a second display An image display device that displays a second display for specifying a region in association with the subject image, wherein the first detection result is obtained by detecting the position of the first region in the subject image in the image. According to the first detection result, the second detection means for detecting the position of the second region in the subject image in the image and obtaining the second detection result, and the first detection result. Estimating means for estimating the position of the second region and obtaining an estimation result; and the first display body, the second detection result, and the estimation result, respectively, according to the first detection result, the second detection result, and the estimation result. 2 display bodies and the second region Display control means for controlling display of the estimated display body for estimating the position on the display unit, and the estimating means displays the first display body according to the first detection result when the first display body is displayed. The estimation result is obtained according to the first detection result only when the second detection result is not obtained by the second detection means.

  According to the present invention, the frame indicating the first area that is the face area of the subject image and the frame indicating the second area that is the human body area can be simultaneously displayed on the screen, and the user can view the entire subject in a short time. It can be performed.

  Further, according to the present invention, it is possible to reduce the difference between the detection area and the detection frame in the moving subject, and the user can accurately visually recognize the moving subject by the detection frame.

It is a block diagram which shows the structure about an example of an imaging device provided with the image display apparatus by the 1st Embodiment of this invention. It is a figure which shows an example of the face information obtained by the face detection part shown in FIG. It is a figure which shows an example of the human body information obtained by the human body detection part shown in FIG. It is a figure which shows an example of the face display information required when the system control part shown in FIG. 1 produces | generates a face detection frame. It is a figure which shows an example of the human body display information required when the system control part shown in FIG. 1 produces | generates a human body detection frame. 3 is a flowchart for explaining frame display processing of a face detection frame and a human body detection frame performed by the imaging apparatus 100 shown in FIG. 1. 2A and 2B are diagrams for explaining the result of frame display processing performed by the imaging apparatus illustrated in FIG. 1, in which FIG. 1A illustrates a state in which a face detection frame and a human body prediction frame are displayed, and FIG. The figure which shows the state where the frame and the human body detection frame were displayed, (c) is the figure which shows the state where only the face detection frame is displayed, (d) is the figure which shows the state where only the human body detection frame is displayed. It is a timing chart which shows the display timing of the face detection frame, human body detection frame, and human body prediction frame which are performed with the imaging device shown in FIG. It is a flowchart for demonstrating the frame display process performed with the imaging device by the 1st Embodiment of this invention. It is a figure for demonstrating the calculation of the movement vector performed with the imaging device by the 2nd Embodiment of this invention, (a) is a figure which shows the display of the face detection frame in the time t, (b) is time (t + n). FIG. 6 is a diagram illustrating display of a face detection frame and a human body detection frame in FIG. It is a figure which shows an example of the display of the conventional face detection frame and a human body detection frame. It is a timing chart which shows the display timing of the conventional face detection frame and human body detection frame. It is a figure which shows an example of the face detection frame displayed on the screen by the process according to the timing chart shown in FIG. 12, and a human body detection frame, (a) is a figure which shows the screen display at the time t, (b) is time (t + 2). FIG. It is a figure for demonstrating the shift | offset | difference of the conventional face detection frame and a human body detection frame, (a) is a figure which shows the state in which only the face detection frame was displayed, (b) is a face detection frame and a human body detection frame displayed It is a figure which shows the state made.

  Hereinafter, an example of an image display device according to an embodiment of the present invention will be described with reference to the drawings.

[First Embodiment]
FIG. 1 is a block diagram illustrating a configuration of an example of an imaging apparatus including an image display apparatus according to the first embodiment of the present invention.

  The illustrated image pickup apparatus is a digital camera (hereinafter simply referred to as a camera), and the camera 100 has a photographing lens unit (hereinafter simply referred to as a photographing lens) 103 including a focus lens. A shutter 101 having a function is arranged.

  An optical image (subject image) that has passed through the photographing lens 103 and the shutter 101 is formed on the imaging unit 22 that is an imaging element. The imaging unit 22 is composed of a CCD or CMOS image sensor, and outputs an electrical signal (analog signal, that is, an image signal) corresponding to the optical image. This analog signal is supplied to the A / D converter 23 where it is converted into a digital signal (image data).

  A barrier 102 is disposed on the front side of the photographic lens 103. The imaging system including the photographing lens 103, the shutter 101, and the imaging unit 22 is covered by the barrier 102, so that the imaging system is prevented from being soiled and damaged.

  The image processing unit 24 performs resize processing such as predetermined pixel interpolation and reduction, and color conversion processing on the image data from the A / D converter 23 or the image data from the memory control unit 15. Further, the image processing unit 24 performs a predetermined calculation process using image data obtained as a result of photographing. The system control unit 50 performs exposure control and distance measurement control based on the calculation result. That is, the system control unit 50 performs TTL (through-the-lens) AF (autofocus) processing, AE (automatic exposure) processing, and EF (flash pre-flash) processing.

  Further, the image processing unit 24 performs predetermined calculation processing using image data obtained as a result of shooting, and performs TTL AWB (auto white balance) processing based on the calculation result.

  The image data output from the A / D converter 23 is written into the memory 32 via the image processing unit 24 and the memory control unit 15 or via the memory control unit 15. In the memory 32, image data that is output from the A / D converter 23 is written, and image data to be displayed on the display unit 28 is stored. Note that the memory 32 has a sufficient storage capacity for storing a predetermined number of still image data, moving image data for a predetermined time, and audio data. In addition, the memory 32 also serves as an image display memory (video memory).

  The D / A converter 13 converts the image display image data stored in the memory 32 into an analog signal and supplies it to the display unit 28. As a result, an image corresponding to the image data is displayed on the display unit 28. The display unit 28 is, for example, an LCD.

  The nonvolatile memory 56 is an electrically erasable / recordable memory, and, for example, an EEPROM is used. The nonvolatile memory 56 stores constants and programs for operating the system control unit 50. This program is a program for executing various flowcharts described later.

  The system control unit 50 controls the camera 100. The system control unit 50 executes a program recorded in the above-described nonvolatile memory 56 and performs each process described later. For example, a RAM is used as the system memory 52. In the system memory 52, constants and variables for the operation of the system control unit 50, a program read from the nonvolatile memory 56, and the like are expanded. Further, the system control unit 50 executes display control by controlling the memory 32, the D / A converter 13, the display unit 28, and the like.

  The mode switch 60, the shutter button 61, and the operation unit 70 are input operation units for inputting various operation instructions to the system control unit 50. The shutter button 61 is for instructing photographing and has a first shutter switch and a second shutter switch.

  The mode switch 60 is used when the operation mode of the system control unit 50 is switched to any one of a still image recording mode, a moving image recording mode, a reproduction mode, and the like.

  When the shutter button 61 is in a so-called half-pressed state by operating the shutter button 61, the first shutter switch is turned on (imaging preparation instruction), and the first shutter switch signal SW1 is given to the system control unit 50. When the first shutter switch signal SW1 is received, the system control unit 50 starts AF (autofocus) processing, AE (automatic exposure) processing, AWB (auto white balance) processing, EF (flash pre-flash) processing, and the like. To do.

  When the shutter button 61 is in a fully-pressed state, the second shutter switch is turned on (shooting instruction), and the second shutter switch signal SW2 is given to the system control unit 50. Upon receiving the second shutter switch signal SW2, the system control unit 50 starts a series of photographing processing operations from reading out an image signal from the imaging unit 22 to writing image data in the recording medium 200.

  When various function icons displayed on the display unit 28 are selected and operated, the operation members such as buttons and keys provided in the operation unit 70 are appropriately assigned functions for each scene and function as various function buttons. .

  Examples of the function buttons include an end button, a return button, an image advance button, a jump button, a narrowing button, and an attribute change button. For example, when a menu button is pressed, a menu screen for performing various settings is displayed on the display unit 28. The user can make various settings intuitively using the menu screen displayed on the display unit 28 and the four-way button or the SET button.

  The power supply control unit 80 includes a battery detection circuit, a DC-DC converter, a switch circuit that switches a block to be energized, and the like, and detects whether or not a battery is installed, the type of battery, and the remaining battery level. Further, the power control unit 80 controls the DC-DC converter based on the detection result and the instruction from the system control unit 50, and supplies a necessary voltage to each unit including the recording medium 200 for a necessary period.

  The power supply unit 30 includes a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li battery, and an AC adapter. An interface (I / F) 18 is an interface with a recording medium 200 such as a memory card or a hard disk. When the recording medium 200 is a memory card, the memory card is constituted by a semiconductor memory, for example. The power switch 72 is a switch for turning on / off the power of the camera 100.

  As shown in the figure, a face detection unit (face detection unit) 104 and a human body detection unit (body detection unit) 105 are connected to the system control unit 50, and the face detection unit 104 is connected to the image data stored in the memory 32. The face area (first area) of the subject image is detected. The human body detection unit 105 detects a human body region (second region: body region) of the subject image in the image data stored in the memory 32. Each of the face area and the human body area is a specific area in the image.

  The above-described camera 100 can perform shooting using so-called center single point AF and face AF. Here, the central one-point AF means that AF is performed on one central position on the photographing screen. The face AF refers to performing AF on the face area detected by the face detection function (that is, the face detection unit 104).

  Here, the face detection function will be described. Under the control of the system control unit 50, the face detection unit 104 performs horizontal bandpass filter processing on the image data input from the memory 32. Further, under the control of the system control unit 50, the face detection unit 104 performs vertical bandpass filter processing on the image data that has been subjected to horizontal bandpass filter processing. By these horizontal and vertical band pass filter processes, edge components are detected from the image data.

  Subsequently, under the control of the system control unit 50, the face detection unit 104 performs pattern matching on the detected edge component, and extracts a candidate group of eyes, nose, mouth, and ears. Then, under the control of the system control unit 50, the face detection unit 104 selects candidates that satisfy a preset condition (for example, conditions such as the distance and inclination of two eyes) in the extracted eye candidate group. Is determined. Then, the face detection unit 104 narrows the eye pairs as eye candidate groups.

  Furthermore, under the control of the system control unit 50, the face detection unit 104 associates the narrowed-down eye candidate group with other parts (nose, mouth, ear) that form a face corresponding to the eye. Further, the face detection unit 104 performs a preset non-face condition filtering process to detect a face (face area). Then, the face detection unit 104 outputs face information indicating the detection result of the face area to the system control unit 50. The system control unit 50 stores face information in the system memory 52.

  Next, the human body detection function will be described. The human body detection unit 105 performs horizontal bandpass filter processing on the image data input from the memory 32. Further, under the control of the system control unit 50, the face detection unit 104 performs vertical bandpass filter processing on the image data that has been subjected to horizontal bandpass filter processing. By these horizontal and vertical band pass filter processes, edge components are detected from the image data.

  Next, under the control of the system control unit 50, the human body detection unit 105 performs pattern matching on the detected edge component, and determines whether or not it corresponds to the contour shape of the human body region. The human body detection unit 105 assumes that an edge component corresponding to the contour shape of the human body region is the human body region. The human body detection unit 105 outputs human body information indicating the detection result of the human body region to the system control unit 50. The system control unit 50 stores human body information in the system memory 52. Since the human body shape pattern has more types than the face shape pattern, the time required for the human body region detection by the human body detection unit 105 is longer than the time required for the face detection unit 104 to detect the face region. .

  FIG. 2 is a diagram illustrating an example of face information (also referred to as face detection information) obtained by the face detection unit 104 illustrated in FIG.

  The face information includes an X axis (pixel) and a Y axis (pixel), and a face horizontal detection position (Xf) 307 is defined on the X axis. The face horizontal detection position (Xf) 307 is a position that is the starting point in the horizontal direction (X-axis direction) in the face area detected by the face detection unit 104.

  Further, a face vertical detection position (Yf) 308 is defined on the Y axis. The face vertical detection position (Yf) 308 is a position that is a start point in the vertical direction (Y-axis direction) in the face area detected by the face detection unit 104.

  In the face information, a face horizontal detection size (Hf) 302 is defined. The face horizontal detection size (Hf) 302 is size information indicating the size of the face area in the horizontal direction. In the face information, a face vertical detection size (Vf) 303 is defined. The face vertical detection size (Vf) 303 is size information indicating the vertical size of the face area.

  The face detection position information 301 indicates a set of the face horizontal detection position 307 and the face vertical display position 308. The face detection size information 309 indicates a set of the face horizontal detection size 302 and the face vertical detection size 303.

  In the face information, a face yaw direction detection angle (Θfy) 304 is defined. The face yaw direction detection angle (Θfy) 304 is angle information indicating the rotation angle of the face with respect to the horizontal direction.

  In the face information, a face roll direction detection angle (Θfr) 305 is defined. This face roll direction detection angle (Θfr) 305 is angle information indicating the rotation angle of the face with respect to the vertical direction.

  In addition, a face pitch direction detection angle (Θfp) 306 is defined in the face information. The face pitch direction detection angle (Θfp) 306 is angle information indicating the inclination of the face with respect to the vertical direction.

  The face detection angle information 310 indicates a set of a face yaw direction detection angle 304, a face roll direction detection angle 305, and a face pitch detection angle 306.

  FIG. 3 is a diagram illustrating an example of human body information (also referred to as human body detection information) obtained by the human body detection unit 105 illustrated in FIG.

  The human body information includes an X axis and a Y axis, and a horizontal human body detection position (Xb) 317 is defined on the X axis. The human body horizontal detection position (Xb) 317 is a position that is the starting point in the horizontal direction (X-axis direction) in the human body region detected by the human body detection unit 105.

  Furthermore, a human body vertical detection position (Yb) 318 is defined on the Y axis. The human body vertical detection position (Yb) 318 is a position that is a starting point in the vertical direction (Y-axis direction) in the human body region detected by the human body detection unit 105.

  The human body detection position information 311 indicates a set of the human body horizontal detection position 317 and the human body vertical detection position 318.

  Further, the human body horizontal detection size (Hb) 312 is defined in the human body information. This human body horizontal detection size (Hb) 312 is size information indicating the horizontal size of the human body region. A human body vertical detection size (Vb) 313 is defined in the human body information. This human body vertical detection size (Vb) 313 is size information indicating the size of the human body region in the vertical direction.

  The human body detection size information 319 indicates a set of the human body horizontal detection size 312 and the human body vertical detection size 313.

  In the human body information, a human yaw direction detection angle (Θby) 314 is defined. The human body yaw direction detection angle (Θby) 314 is angle information indicating the rotation angle of the human body with respect to the horizontal direction.

  In the human body information, a human body roll direction detection angle (Θbr) 315 is defined. This human body roll direction detection angle (Θbr) 315 is angle information indicating the rotation angle of the human body with respect to the vertical direction.

  In addition, a human body pitch direction detection angle (Θbp) 316 is defined in the human body information. The human body pitch direction detection angle (Θbp) 316 is angle information indicating the inclination of the human body with respect to the vertical direction.

  The human body detection angle information 320 indicates a set of the human body yaw direction detection angle 314, the human body roll direction detection angle 315, and the human body pitch detection angle 316.

  FIG. 4 is a diagram showing an example of face display information necessary when the system control unit 50 shown in FIG. 1 generates a face detection frame.

  The face display information includes an X axis (pixel) and a Y axis (pixel), and a horizontal face display position (XF) 327 is defined on the X axis. The face horizontal display position (XF) 327 is a position that is a starting point in the horizontal direction (X-axis direction) in the face detection frame.

  Further, a face vertical display position (YF) 328 is defined on the Y axis. The face vertical display position (YF) 328 is a position that is a starting point in the vertical direction (Y-axis direction) in the face detection frame.

  The face display position information 321 indicates a set of the face horizontal display position 327 and the face vertical display position 328.

  The face display information defines a face horizontal display size (HF) 322. The face horizontal display size (HF) 322 is size information indicating the size of the face detection frame in the horizontal direction. In the face display information, a face vertical display size (VF) 323 is defined. The face vertical display size (VF) 323 is size information indicating the size of the face detection frame.

  The face display size information 329 indicates a set of the face horizontal display size 322 and the face vertical display size 323.

  The face display information defines a face yaw direction display angle (ΘFy) 324. The face yaw direction display angle (ΘFy) 324 is angle information indicating the rotation angle of the face detection frame with respect to the horizontal direction.

  In the face display information, a face roll direction display angle (ΘFr) 325 is defined. The face roll direction display angle (ΘFr) 325 is angle information indicating the rotation angle of the face detection frame with respect to the vertical direction. In addition, a face pitch direction display angle (ΘFp) 326 is defined in the face display information. The face pitch direction display angle (ΘFp) 326 is angle information indicating the inclination of the face detection frame with respect to the vertical direction.

  The face display angle information 330 indicates a set of the face yaw direction display angle 324, the face roll direction display angle 325, and the face pitch display angle 326.

  FIG. 5 is a diagram illustrating an example of human body display information necessary when the system control unit 50 illustrated in FIG. 1 generates a human body detection frame.

  The human body display information includes an X axis (pixel) and a Y axis (pixel), and a human body horizontal display position (XB) 337 is defined on the X axis. The human body horizontal display position (XB) 337 is a position that is a starting point in the horizontal direction (X-axis direction) in the human body detection frame. Furthermore, a human body vertical display position (YB) 338 is defined on the Y axis. The human body vertical display position (YB) 338 is a position that is a starting point in the vertical direction (Y-axis direction) in the human body detection frame.

  The human body display position information 331 indicates a set of the human body horizontal display position 337 and the human body vertical display position 338.

  The human body display information defines a human body horizontal display size (HB) 332. The human body horizontal display size (HB) 332 is size information indicating the horizontal size of the human body detection frame. A human body vertical display detection size (VB) 333 is defined in the human body display information. The human body vertical display size (VB) 333 is size information indicating the size of the human body detection frame in the vertical direction.

  The human body display size information 339 indicates a set of the human body horizontal display size 332 and the human body vertical display size 333.

  In the human body display information, a human body yaw direction display angle (ΘBy) 334 is defined. The human body yaw direction display angle (ΘBy) 334 is angle information indicating the rotation angle of the human body detection frame with respect to the horizontal direction.

  In the human body display information, a human body roll direction display angle (ΘBr) 335 is defined. This human body roll direction display angle (ΘBr) 335 is angle information indicating the rotation angle of the human body detection frame with respect to the vertical direction. In addition, a human body pitch direction display angle (ΘBp) 336 is defined in the human body display information. The human body pitch direction display angle (ΘBp) 336 is angle information indicating the inclination of the human body detection frame with respect to the vertical direction.

  The human body display angle information 340 indicates a set of the human body yaw direction display angle 334, the human body roll direction display angle 335, and the human body pitch display angle 336.

  FIG. 6 is a flowchart for explaining frame display processing of the face detection frame and the human body detection frame performed by the imaging apparatus 100 shown in FIG. In the frame display process shown in FIG. 6, the system control unit 50 controls the display unit 28 via the memory control unit 15.

  Under the control of the system control unit 50, the face detection unit (first detection unit) 104 and the human body detection unit (second detection unit) 105 capture the subject in the display image data stored in the memory 32 (step S401). When the subject is captured, the face detection unit 104 and the human body detection unit 105 start face detection and human body detection, respectively (step S402). Here, the face is the first region and the human body is the second region.

  Subsequently, the system control unit 50 determines whether or not face detection in the face detection unit 104 is completed (step S403). If face detection is not completed (NO in step S403), system control unit 50 stands by.

  On the other hand, when the face detection is completed (YES in step S403), the system control unit 50 determines whether or not a face exists according to the face detection result of the face detection unit 104 (step S404).

  If a face is present (YES in step S404), system control unit 50 determines whether or not human body detection in human body detection unit 105 has been completed (step S405). If the human body detection is not completed (NO in step S405), the system control unit 50 predicts the human body region according to the face detection result (first detection result) and obtains a human body prediction region (human body estimation result) (step). S406). Then, the system control unit 50 displays a face detection frame (first display body) for specifying the face area according to the face detection result on the screen with a solid line, and estimates the human body area according to the human body prediction area (estimation result). The human body estimation frame (estimated display body) to be displayed is displayed on the screen as a broken line (dotted line) (step S407).

  When the human body detection is completed (YES in step S405), system control unit 50 determines whether or not a human body exists according to the human body detection result of human body detection unit 105 (step S408). If a human body exists (YES in step S408), system control unit 50 determines a human body detection frame (second body detection frame and second human body region) according to the face detection result and the human body detection result (second detection result), respectively. Is displayed on the screen as a solid line (step S409).

  On the other hand, if there is no human body (NO in step S408), system control unit 50 displays only the face detection frame with a solid line on the screen according to the face detection result (step S410).

  If there is no face in step S404 (NO in step S404), the system control unit 50 determines whether or not the human body detection in the human body detection unit 105 is completed (step S411). If human body detection is not completed (NO in step S411), system control unit 50 stands by.

  When the human body detection is completed (YES in step S411), the system control unit 50 determines whether or not a human body exists (step S412). If a human body exists (YES in step S412), system control unit 50 displays the human body detection frame on the screen as a solid line according to the human body detection result of human body detection unit 105 (step S413).

  In this way, the system control unit 50 displays the face detection frame, the human body detection frame, and the estimated detection frame on the display unit 28 with different display forms.

  After the process of step S407, S409, S410, or S413, the system control unit 50 determines whether a release has been performed (step S414). That is, the system control unit 50 determines whether or not the shutter button 61 is fully pressed.

  When the release is performed (YES in step S414), the subject is shot under the control of the system control unit 50 (step S15). Then, the system control unit 50 ends the frame display process.

  If release is not performed (YES in step S414), the process returns to step S401. If no human body is present in step S412 (NO in step S412), system control unit 50 proceeds to the process of step S414.

  FIG. 7 is a diagram for explaining the result of the frame display process performed by the imaging apparatus 100 shown in FIG. FIG. 7A is a diagram showing a state where the face detection frame and the human body estimation frame are displayed, and FIG. 7B is a diagram showing a state where the face detection frame and the human body detection frame are displayed. FIG. 7C is a diagram showing a state where only the face detection frame is displayed, and FIG. 7D is a diagram showing a state where only the human body detection frame is displayed.

  In the process of step S407 described above, the face detection frame 501 and the human body prediction frame 502 shown in FIG. At this time, since the human body prediction frame 502 indicates a human body prediction region predicted according to the face detection result, it is displayed with a dotted line. Thereby, the user can know that the human body prediction frame 502 is displayed by prediction. The relative position and size of the human body prediction frame relative to the face detection frame 501 are set in advance, and are obtained by calculation using the position of the face detection frame 501 as a reference.

  In the process of step S409, the face detection frame 501 and the human body detection frame 503 shown in FIG. Thus, the user can know that both the face area and the human body area have been detected.

  In the process of step S410, only the face detection frame 501 shown in FIG. 7C is displayed on the display unit 28. As a result, the user can know that the human body region has not been detected.

  In the process of step S413, only the human body detection frame 503 shown in FIG. 7D is displayed on the display unit 28. This allows the user to know that no face area has been detected.

  FIG. 8 is a timing chart showing display timings of the face detection frame, the human body detection frame, and the human body prediction frame performed by the imaging apparatus 100 shown in FIG.

  Now, when the subject D1 is captured at time 1, face detection processing and human body detection processing of the subject D1 are started. When the face detection of the subject D1 is completed at time 2, the face detection frame related to the subject D1 is displayed on the screen.

  At this time, since the human body detection process is not completed for the subject D1, the human body region related to the subject D1 is predicted (estimated) according to the face detection result of the subject D1. Then, the human body prediction frame (human body prediction frame) of the subject D1 is displayed on the screen according to the human body prediction region.

  Note that the subject D2 is captured at time 2, and the face detection process and the human body detection process related to the subject D2 are started.

  When face detection of the subject D2 is completed at time 3, a face detection frame related to the subject D2 is displayed on the screen. At this time, since the human body detection process has not been completed for the subjects D1 and D2, the human body region related to the subject D2 is predicted according to the face detection result of the subject D2. Then, the human body prediction frame of the subject D2 is displayed on the screen according to the human body prediction area.

  At time 3, the subject D3 is captured, and the face detection process and human body detection process of the subject D3 are started.

  When face detection of the subject D3 is completed at time 4, a face detection frame related to the subject D3 is displayed on the screen. At this time, since the human body detection process related to the subject D1 is completed, the human body detection frame (solid line) is displayed on the screen for the subject D1 instead of the human body prediction frame (dotted line).

  Note that the subject D4 is captured at time 4, and the face detection process and the human body detection process related to the subject D4 are started.

  In this way, the process is performed in the same manner until time (t + 2). That is, the same processing as that in time 4 is repeated until the release is performed. As a result, at time (t + 1), the face detection frame (solid line) up to the subject Dt is displayed on the screen, and the human body prediction frame (dotted line) is displayed on the screen for the subjects D (t−1) and Dt. Further, a human body detection frame (solid line) up to the subject D (t-2) is displayed on the screen.

  As described above, in the first embodiment of the present invention, when the human body detection process is not completed when the face detection process is completed, the face detection frame and the human body prediction frame are displayed on the screen. The frame indicating the face area and the frame indicating the human body area can be simultaneously displayed on the screen. As a result, the user can easily visually recognize the entire subject without waiting for the human body detection process to end.

[Second Embodiment]
Subsequently, an imaging device including an image display device according to a second embodiment of the present invention will be described. The configuration of the imaging apparatus according to the second embodiment is the same as that of the imaging apparatus shown in FIG.

  FIG. 9 is a flowchart for explaining frame display processing performed by the imaging apparatus 100 according to the first embodiment of the present invention.

  In the following description, a case where an image is displayed on the display unit 28 for each of a plurality of consecutive frames will be described. At this time, the system control unit 50 can process the process from the face detection to the display of the face detection frame within one frame. The process from the human body detection to the display of the human body detection frame can be performed with n frames ( n is an integer of 2 or more).

  Under the control of the system control unit 50, the face detection unit 104 and the human body detection unit 105 capture a subject in the display image data (one frame) stored in the memory 32 (step S901). Then, the face detection unit 104 starts face detection for the captured subject and stores the face information in the system memory 52 as described above (step S902). In parallel with this, the human body detection unit 105 performs human body detection on the captured subject (step S903). As described above, the time required for the human body detection unit 105 to detect the human body is longer than the time required for the face detection unit 104 to detect the face.

  Then, the system control unit 50 displays a face detection frame on the display unit 28 according to the previously obtained face information (or face display information), that is, the face display position information 321 (step S904).

  Subsequently, the system control unit 50 determines whether the human body detection by the human body detection unit 105 is completed (step S905).

  If the detection of the human body is not completed (NO in step S905), that is, if it is determined that the human body detection is not yet completed in the current frame, the system control unit 50 captures the next frame stored in the memory 32 ( Step S906).

  Subsequently, the system control unit 50 performs face detection and face tracking using the frame captured in step S906 for the face for which face detection has been performed in step S902 (step S907).

  If one face is successfully detected in both the current frame and the previous frame, the system control unit 50 regards the detected face area as the same person, associates them, and stores the position. When a plurality of faces can be detected, the face area closest to the position between the frames is regarded as the face area of the same person, and the positions are stored in association with each other.

  On the other hand, if the face detection has failed in the current frame despite the fact that the face area has been specified in the previous frame, the system control unit 50 performs, for example, motion tracking to perform face tracking. . In other words, in the case of a moving image, motion vector detection is performed on the subject region for the current frame and the previous frame, and the position of the subject region is sequentially corrected using the motion vector detection result (motion vector amount) as the amount of movement of the subject region. Tracking processing is performed. The system control unit 50 obtains a motion vector amount for the current frame and the previous frame by using a correlation operation such as block matching in the subject area.

  The system control unit 50 displays a face detection frame on the display unit 28 according to the result of face detection or face tracking (step S908). Then, the system control unit 50 returns to the process of step S905.

  On the other hand, when the detection of the human body is completed (YES in step S905), system control unit 50 calculates a movement vector indicating the movement direction and movement amount of the face during the period required for the detection of the human body (step S909). .

  Assuming that the time when the subject is captured in step S901 is t, the time when the detection of the human body is completed is time (t + n). Here, referring to FIG. 14 described above, since the display of the face detection frame is processed within one captured frame, at time t, as shown in FIG. 14A, the face detection frame 201 and the face detection frame are displayed. 205 is displayed according to the face display position information 321.

  Furthermore, since new detection results or tracking results are obtained for the face at any time, the position of the new face can be displayed in the face frames 201 and 205 at the time (t + n) shown in FIG. .

  On the other hand, since the display of the human body detection frame requires n frames, at time (t + n), the human body detection frame is displayed according to the human body display position information 331 related to the subject captured at time t.

  Therefore, if the position does not move during n frames like the subject 203, the position of the human body and the human body detection frame 202 are displayed even if the human body detection frame 202 is displayed on the screen according to the human body display position information 331 at time t. Does not cause any discomfort to the user.

  On the other hand, when the position of the subject 204 moves during n frames like the subject 204, the face detection frame 205 can be displayed at the new face position of the subject 204 at time (t + n). However, if the human body detection frame 206 is displayed according to the human body display position information 331 at time t, the human body detection frame 206 is displayed with a deviation from the position of the human body of the subject 204.

  In order to reduce the above-described shift, the system control unit 50 calculates a face movement vector in step S909. That is, the system control unit 50 calculates the movement vector of the face area according to the face display position information 321 of the frame at time t and the face position of the frame at time (t + n).

  Here, the face position of the frame at time (t + n) is face display position information obtained as a result of the face tracking performed in step S907 for the current frame.

  FIG. 10 is a diagram for explaining calculation of a movement vector performed by the imaging apparatus 100 according to the second embodiment of the present invention. FIG. 10A is a diagram showing the display of the face detection frame at time t, and FIG. 10B is a diagram showing the display of the face detection frame and the human body detection frame at time (t + n).

  Assume that the subject 1001 moves during n frames. Assume that the position coordinates of the face detection frame 1002 indicating the face area detected at time t are (Xf (t), Yf (t)). Further, the position coordinates of the face detection frame 1003 obtained as a result of face tracking at time (t + n) are (Xf (t + n), Yf (t + n)).

  Using the position coordinates (Xf (t), Yf (t)) and the position coordinates (Xf (t + n), Yf (t + n)), the face movement vector V = (Vx, Vy) is expressed by the following equation (1): Indicated by

(Vx, Vy) = ((Xf (t + n) −Xf (t)), (Yf (t + n) −Yf (t))) (1)
Subsequently, the system control unit 50 corrects the human body display position information 331 detected by the human body detection unit 105 using the movement vector V, and obtains a correction detection result (corrected human body display position information (step S910).

  Since the direction and the distance that the same person moves at a predetermined time are estimated to be the same for the face and the human body, the system control unit 50 determines the human body display position information (Xb (t + n), Yb (t + n) at time (t + n). )) Is calculated according to equations (2) and (3).

Xb (t + n) = Xb (t) + Vx (2)
Yb (t + n) = Yb (t) + Vy (3)
In addition to the human body display position information, the face size change amount (size change amount) is also calculated according to the face detected at time t and face display size information 329 obtained as a result of face tracking at time (t + n). be able to. Then, the human body display size information 339 at time t is corrected in accordance with the amount of change in face size so as to reduce the deviation between the size of the human body detection frame displayed at time (t + n) and the actual human body size. Good.

  Subsequently, the system control unit 50 displays the human body detection frame 1004 on the display unit 28 in accordance with the human body display position information (Xb (t + n), Yb (t + n)) corrected in step S910 (step S911). Then, the system control unit 50 once ends the human body frame display process. By repeating this human body frame display process, the face detection frame and the human body detection frame are continuously displayed.

  In this way, by tracking the face position at time (t + n) and correcting the human body display position information, the face display frame 1006 and the human body display frame 1007 indicated by dotted lines are moved by the direction and movement amount indicated by the arrow 1005. Thus, the face detection frame 1003 and the human body detection frame 1004 can be displayed in accordance with the actual face region and human body region.

  As described above, in the second embodiment of the present invention, the face detection unit 104 detects the face area (face) in the subject, and calculates the face movement vector until the human body detection is completed. Then, after the human body is detected by the human body detection unit 105, the human body display position information is corrected according to the movement vector, and the human body detection frame is displayed on the display unit.

  As a result, when the subject is moving, the human body detection frame can be displayed while reducing the deviation between the human body detection position and the current human body position.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

  As is clear from the above description, in the example shown in FIG. 1, at least the system control unit 50, the system memory 52, the face detection unit 104, the human body detection unit 105, the memory 32, the memory control unit 15, and the D / A converter. 13 and the display unit 28 constitute an image display device. The photographing lens 103, the shutter 101, the imaging unit 22, the A / D converter 23, and the image processing unit 24 constitute an imaging unit.

  As mentioned above, although this invention was demonstrated based on embodiment, this invention is not limited to these embodiment, Various forms of the range which does not deviate from the summary of this invention are also contained in this invention. .

  For example, the function of the above embodiment may be used as a control method, and the control method may be executed by the image display apparatus. The program having the functions of the above-described embodiments may be used as a control program, and the control program may be executed by a computer included in the layer display device. The control program is recorded on a computer-readable recording medium, for example.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various recording media, and the computer (or CPU, MPU, etc.) of the system or apparatus reads the program. To be executed.

DESCRIPTION OF SYMBOLS 15 Memory control part 22 Image pick-up part 24 Image processing part 28 Display part 50 System control part 60 Mode change switch 61 Shutter button 70 Operation part 104 Face detection part 105 Human body detection part

Claims (7)

When an image including a subject image is displayed on the display unit, a first display body that specifies a first area and a second display body that specifies a second area are associated with the subject image. An image display device for displaying
First detection means for detecting a position of the first region in the subject image and obtaining a first detection result in the image;
Second detection means for detecting a position of the second region in the subject image in the image and obtaining a second detection result;
Estimating means for estimating a position of the second region according to the first detection result and obtaining an estimation result;
An estimated display body that estimates the positions of the first display body, the second display body, and the second region according to the first detection result, the second detection result, and the estimation result, respectively. Display control means for controlling display on the display unit,
The estimation unit is configured to display the first display object according to the first detection result only when the second detection result is not obtained by the second detection unit. An image display device characterized in that the estimation result is obtained according to the detection result of 1.   The display control means displays the estimated display body on the display unit only when the estimation result is obtained by the estimation means, and when the second detection result is obtained by the second detection means, The image display apparatus according to claim 1, wherein the second display body is displayed on the display unit instead of the estimated display body.   3. The display control unit according to claim 1, wherein the display control unit displays the first display body, the second display body, and the estimated display body on the display unit in different display forms. Image display device. The first detection means detects a face area as the first area, and the second detection means detects a body area that is the entirety of the subject image as the second area. Item 4. The image display device according to any one of Items 1 to 3 . When an image including a subject image is displayed on the display unit, a first display body that specifies a first area and a second display body that specifies a second area are associated with the subject image. A method for controlling an image display device to display
A first detection step of detecting a first detection result by detecting a position of the first region in the subject image in the image;
A second detection step of detecting a position of the second region in the subject image in the image and obtaining a second detection result;
An estimation step for estimating a position of the second region in accordance with the first detection result and obtaining an estimation result;
An estimated display body that estimates the positions of the first display body, the second display body, and the second region according to the first detection result, the second detection result, and the estimation result, respectively. A display control step of controlling display on the display unit,
In the estimation step, when the first display body is displayed according to the first detection result, only when the second detection result is not obtained in the second detection step, the first display body is displayed. A control method characterized in that the estimation result is obtained according to a detection result of 1. When an image including a subject image is displayed on the display unit, a first display body that specifies a first area and a second display body that specifies a second area are associated with the subject image. A control program used in an image display device for displaying
A computer provided in the image display device,
A first detection step of detecting a first detection result by detecting a position of the first region in the subject image in the image;
A second detection step of detecting a position of the second region in the subject image in the image and obtaining a second detection result;
An estimation step for estimating a position of the second region in accordance with the first detection result and obtaining an estimation result;
An estimated display body that estimates the positions of the first display body, the second display body, and the second region according to the first detection result, the second detection result, and the estimation result, respectively. A display control step of controlling display on the display unit,
In the estimation step, when the first display body is displayed according to the first detection result, only when the second detection result is not obtained in the second detection step, the first display body is displayed. A control program for obtaining the estimation result according to a detection result of 1. The image display device according to any one of claims 1 to 4 ,
Imaging means for capturing a plurality of subjects and obtaining image data;
The image pickup apparatus, wherein the display control means displays an image corresponding to the image data on the display unit.

Images