JP2011095984A - Image display apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image display apparatus for reducing a load to be imposed on the recognition processing of a command related to the operation instruction of a user based on a captured image. <P>SOLUTION: The image display apparatus includes: a display unit 4 for displaying an image; an imaging unit 14 for capturing the image of an operator who appreciates an image displayed at the display unit; a face detection unit 18 for detecting the face of the operator based on the captured image captured by the imaging unit; a hand detection region setter 26 for setting a hand detection region for detecting the hand of the operator based on the position of the face in the captured image of the operator detected by the face detection unit; a hand detection unit 28 for detecting the hand of the operator in the hand detection region; a command recognition unit 32 for recognizing the operation, shape or position of the hand of the operator detected by the hand detection unit as an operation instruction; and a display control unit 13 for controlling the display mode of the image displayed at the display unit based on the operation instruction recognized by the command recognition unit. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image display device.

  There is known an image display device that includes a camera that captures a user and recognizes a command related to a user's operation instruction based on a captured image captured by the camera (see, for example, Patent Document 1). In this image display apparatus, command recognition is started by using a user's hand to the camera as a trigger.

JP 2009-75685 A

  However, in this image display device, it is necessary to always observe the movement of the user's hand as a trigger for command recognition in the entire captured image, and the load on the command recognition process related to the user's operation instruction is large.

  The objective of this invention is providing the image display apparatus which can reduce the load concerning the recognition process of the command which concerns on the user's operation instruction based on a captured image.

  The image display device of the present invention includes a display unit that displays an image, an imaging unit that captures an operator who appreciates an image displayed on the display unit, and the operator based on a captured image captured by the imaging unit. A face detection unit for detecting the face of the operator, and hand detection for setting a hand detection area for detecting the hand of the operator based on the position of the face in the captured image of the operator detected by the face detection unit An area setting unit; a hand detection unit that detects the operator's hand in the hand detection region; and a command recognition that recognizes the operation, shape, or position of the operator's hand detected by the hand detection unit as an operation instruction. And a display control unit that controls a display mode of an image displayed on the display unit based on the operation instruction recognized by the command recognition unit.

  According to the image display device of the present invention, it is possible to reduce a load on a command recognition process related to a user operation instruction.

It is a figure which shows the front surface of DPF which concerns on the 1st Embodiment of this invention. It is a figure which shows the back surface of DPF which concerns on the 1st Embodiment of this invention. It is a block diagram which shows the system configuration | structure of DPF which concerns on the 1st Embodiment of this invention. It is a figure for demonstrating the hand detection area | region of DPF which concerns on the 1st Embodiment of this invention. It is a flowchart which shows the command recognition process of DPF which concerns on the 1st Embodiment of this invention. It is a flowchart which shows the command recognition process of DPF which concerns on the 2nd Embodiment of this invention. It is a figure for demonstrating the hand detection area | region of DPF which concerns on other embodiment of this invention. It is a figure for demonstrating the hand detection area | region of DPF which concerns on other embodiment of this invention.

  Hereinafter, an image display apparatus according to a first embodiment of the present invention will be described with reference to the drawings, taking a digital photo frame (hereinafter referred to as DPF) as an example. FIG. 1 is a diagram showing the front surface of the DPF according to the first embodiment, and FIG. 2 is a diagram showing the back surface. On the front surface of the DPF 2, there are provided a display unit 4 configured by an LCD or the like that electrically displays information such as images and time, and an imaging lens 6 that captures a user who views an image displayed on the display unit 4. Yes. A support member 8 used for installing the DPF 2 is provided on the back surface of the DPF 2.

  FIG. 3 is a block diagram showing a system configuration of the DPF according to the first embodiment. The DPF 2 includes a CPU 12. The CPU 12 includes a display control unit 13 that controls a display mode of the display unit 4 based on a command that will be described later, an imaging unit 14 that includes an imaging element that images subject light via the imaging lens 6, and the like. A memory 16 that stores a predetermined number of frames or a predetermined time for a captured image captured by the imaging unit 14, and facial recognition data is generated by extracting feature points, a specific shape, a specific color, and the like from the captured image data. A face detection unit 18 that detects the face of the user, a face area calculation unit 20 that calculates the area of the face detected from the face recognition data, a detection time measurement unit 22 that measures the detection time of the detected face, and an operating speed of the detected face A face motion detection unit for detecting a hand, a hand detection region setting unit for setting a hand detection region for detecting a user's hand based on the detected face position, and a set hand detection region A hand detection unit 28 that detects a user's hand by extracting a feature point, a specific shape, a specific color, a position, and the like, a hand area calculation unit 30 that calculates an area in a captured image of the detected hand, and hand detection A hand motion recognition unit 32 for recognizing the motion, shape or position of the user's hand detected in the region; a hand motion storage unit 34 for storing a predetermined hand motion, shape or position and a command related to an operation instruction in association with each other; A memory card 36 for storing images and information to be displayed on the display unit 4 is connected.

FIG. 4 is a diagram for explaining a DPF hand detection area according to the first embodiment. When the face detection unit 18 detects the user's face 38 in the captured image 40, a hand detection area of ± a ₁ pixel in the X-axis direction and ± b ₁ pixel in the Y-axis direction with reference to the approximate center of the detected face. 42 is set. Therefore, the hand detection unit 28 detects the hand 44 in the hand detection area 42, but does not detect the hand 46 outside the hand detection area 42. Further, as indicated by an arrow 48, the hand detection area 42 moves in accordance with the face motion.

  Next, the DPF command recognition process according to the first embodiment will be described with reference to the flowchart shown in FIG. When a user who views an image displayed on the display unit 4 in the forward direction of the display unit 4 is imaged by the imaging unit 14 (step S1: Y), captured image data for a predetermined number of frames or a predetermined time is stored in the memory 16. Store (step S2). When imaging by the imaging part 14 is not performed (step S1: N), it waits until imaging is performed.

  Next, the face detection unit 18 detects a human face from the captured image data, and determines whether or not a face has been detected (step S3). When a face is detected, it is determined whether or not the person with the detected face has an intention to operate the DPF 2 by determining whether or not the operation intention determination conditions shown in steps S4 to S8 are met. . In other words, a person whose face is detected is determined as a user by using a condition that satisfies an operation intention determination condition as a trigger. When a plurality of human faces are detected, a person located at a distance closest to the DPF 2 may be detected, or a person registered in advance may be detected. On the other hand, if no face is detected, the process returns to step S1.

  When a face is detected in step S3, it is first determined whether or not the position of the face detected by the face detection unit 18 is in a specific region as an operation intention determination condition (step S4). Here, for example, when the face position is near the center of the image, it is determined that the face position is in the specific region. On the other hand, if the detected position of the face is the edge of the image, it is determined that the face position is outside the specific area. When the face position is not in the specific area (step S4: N), the process returns to step S1.

  If the face position is in the specific area (step S4: Y), the process proceeds to step S5, and it is determined whether or not the next operation intention determination condition is met. That is, it is determined whether the face area in the captured image calculated by the face area calculation unit 20 occupies a predetermined area or more (step S5). If it occupies a predetermined area or more, it is determined that the person whose face is detected has an intention to operate the DPF 2, and the process proceeds to step S9.

  On the other hand, when the detected face area is equal to or smaller than the predetermined area in the image (step S5: N), it is determined whether or not the face detection time measured by the detection time measuring unit 22 is equal to or longer than the predetermined time. (Step S6). If the face detection time is a predetermined time or more, that is, if the user is watching the display unit 4 continuously for a predetermined time or more, it is determined that the person whose face is detected has an intention to operate the DPF 2. The process proceeds to step S9.

  On the other hand, when the face detection time is less than or equal to the predetermined time (step S6: N), it is determined whether or not the face motion speed detected by the face motion detection unit 24 is less than or equal to the predetermined speed (step S7). . If the speed is less than or equal to the predetermined speed, it is determined that the person whose face is detected has an intention to operate the DPF 2, and the process proceeds to step S9.

  On the other hand, when the facial motion speed is equal to or higher than the predetermined speed (step S7: N), the face area in the captured image calculated by the face area calculation unit 20 and the hand in the captured image calculated by the hand area calculation unit 30 A ratio between the face area and the hand area is calculated using the area, and it is determined whether or not the area ratio is within a predetermined range (step S8). If the area ratio is within the predetermined range, it is determined that the person whose face is detected has an intention to operate the DPF 2, and the process proceeds to step S9. On the other hand, when the area ratio of the face and the hand is not within the predetermined range (step S8: N), the process is terminated.

If it is determined in step S4 to S8 that the person whose face is detected has an intention to operate the DPF 2, the person whose face is detected is determined to be the user, and the user's hand is detected. A detection area is set (step S9). At this time, as shown in FIG. 4, a hand detection area 42 of ± a ₁ pixel in the X-axis direction and ± b ₁ pixel in the Y-axis direction is set with reference to the approximate center of the user's face.

  When the hand detection unit 28 detects the user's hand in the hand detection area 42 (step 10: Y), the CPU 12 starts control of the hand movement recognition unit 32 and analyzes the hand movement, shape, and position (step S11). . Although not particularly limited, when analyzing the shape and position of the hand, for example, one frame of the photographed image is analyzed, and when analyzing the motion, a plurality of frames are analyzed. When a plurality of hands are detected in the hand detection area 42, the motion, shape, or position of the hand closest to the DPF 2 may be analyzed, or the hand area calculated by the hand area calculation unit 30 may be analyzed. The motion, shape or position of the hand with the largest may be analyzed.

  When the analyzed hand motion, shape, or position corresponds to the hand motion stored in the hand motion storage unit 34 (step S12: Y), a command associated with the hand motion, shape, or position is executed. (Step S13). For example, in the case of moving the hand from left to right, image feed is executed. If the operation is to close the hand from the open state, a process of turning off the power is executed. When the palm is brought close to the DPF 2, the image displayed on the display unit 4 is enlarged, and when the palm is moved away, the image is reduced. In the sleep state in which the DPF 2 performs imaging by the imaging unit 14 without performing display by the display unit 4, a process of restoring the power source from the sleep state to the activated state is performed when the hand is opened from the closed state. May be. In addition to the movement of the hand, for example, when both arms are crossed and a cross is formed, a process of turning off the power may be executed. Note that after the process of step S13, the captured image data for a predetermined number of frames or a predetermined time stored in the memory 16 may be deleted.

  If no hand is detected in step S10, if the hand motion, shape, or position analyzed in step S12 does not correspond to the hand motion stored in the hand motion storage unit 34, the processing in step S1 is performed. Return.

  According to the DPF according to the present embodiment, since a human face is detected and an operation intention determination is performed on the condition that the detected face is in a specific area, a load on a command recognition process related to a user operation instruction Can be reduced.

  In addition, since the operation intention determination is performed and the hand movement recognition is performed for the person whose operation intention is recognized, it is possible to prevent the operation of the DPF that is not intended by the user. In addition, when a hand is not detected in the hand detection area, hand motion recognition is not performed, so it is possible to reduce the load on command recognition processing related to the user's operation instructions, It is possible to prevent DPF operation unintended by the user due to position recognition.

  In the above-described embodiment, the operation intention determination is performed for one person when there are a plurality of persons detected by the face detection unit 18, but the operation intention is determined for all the detected persons. A configuration may be adopted in which a hand detection area is set for a person who is determined to make an operation intention and a hand is detected.

  Further, in the process of step S7 shown in the flowchart of FIG. 5 according to the above-described embodiment, it is configured to determine whether or not the face motion speed detected by the face motion detection unit 24 is equal to or lower than a predetermined speed. Further, it is possible to determine whether or not the acceleration of the detected face motion is equal to or less than a predetermined value, or whether or not the amount of movement of the detected face in the captured image is equal to or less than a predetermined value. Good.

  Next, DPF command recognition processing according to the second embodiment of the present invention will be described with reference to FIG. In the first embodiment, the command recognition process is performed for a predetermined number of frames or captured images for a predetermined time. However, in the second embodiment, the hand is tracked after the hand is detected. Thus, the configuration for performing the command recognition process is changed. Therefore, a detailed description of the same configuration as that of the first embodiment is omitted, and only different portions will be described in detail. Further, the same components as those in the first embodiment will be described with the same reference numerals.

  FIG. 6 is a flowchart showing command recognition processing according to the second embodiment. When the imaging of the user viewing the image displayed on the forward display unit 4 by the imaging unit 14 is started (step S21: Y), the storage of the captured image data in the memory 16 is started (step S22). If imaging by the imaging unit 14 is not started (step S21: N), the process waits until imaging is started.

  Next, the face detection unit 18 detects a human face from the captured image data, and determines whether or not a face has been detected (step S23). When a face is detected, it is determined whether or not the person whose face is detected has an intention to operate the DPF 2 by determining whether or not the operation intention determination conditions shown in steps S24 to S28 are met. . On the other hand, when a face is not detected, it waits until a face is detected.

  The processing shown in steps S24 to S29 is the same as the processing shown in steps S4 to S9 in the flowchart of FIG. If the face position is not in the specific area in step S24 or if the area ratio of the face to the hand is not within the predetermined range in step S28, the process returns to step S23 and the face is detected for new captured image data. It is determined whether or not it has been done.

  When the hand detection unit 28 detects the user's hand in the hand detection area 42 (step 30: Y), the CPU 12 starts control of the hand movement recognition unit 32 and analyzes the movement, shape, and position of the hand (step S31). . When the analyzed hand motion, shape or position corresponds to the hand motion stored in the hand motion storage unit 34 (step S32: Y), a command associated with the hand motion, shape or position is executed. (Step S33).

  If no hand is detected in step S30, and if the hand motion, shape or position analyzed in step S32 does not correspond to the hand motion stored in the hand motion storage unit 34, the process shown in step S34 is performed. Proceed to

  Next, it is determined whether or not all the operation intention determination conditions are satisfied (step S34). If the operation intention determination conditions are not satisfied, the process is terminated. That is, i) If the user's face does not satisfy the specific area, the imaging is stopped (step S35), and the process is terminated. Also, ii) the user's face area in the image is greater than or equal to a predetermined area, iii) the face detection time is greater than or equal to a predetermined time, vi) the user's face motion speed is less than a certain speed, v) When none of the four conditions that the area ratio of the face and the hand is within the predetermined range is satisfied, the imaging is stopped (step S35), and the process is terminated. On the other hand, when the above-described face detection condition i) is satisfied and any one of ii) to v) is satisfied, the process returns to step S30 to determine whether or not a hand has been detected for new captured image data. continue.

  According to the DPF according to the present embodiment, a human face is detected, an operation intention determination is performed on the condition that the detected face is in a specific area, and after face detection is performed once, the face detection is performed. Therefore, it is possible to reduce the load on command recognition related to user operation instructions. Further, the process may be terminated when no hand is detected in step S34.

In each of the above-described embodiments, the configuration in which the user's hand is detected and the motion of the hand is analyzed has been described as an example. However, as shown in FIG. In this case, a fulcrum detection area 50 of ± a ₂ pixels in the X-axis direction and −b ₂ pixels in the Y-axis direction is set based on the detected human face 38. The hand detection unit 28 detects the hand only when the fulcrum detection region 50 has, for example, an elbow 52 as a fulcrum and the hand detection region 42 has a hand 44, and the hand motion recognition unit 32 detects the motion and shape of the hand. Alternatively, position analysis is performed. Therefore, the hand can be detected with high accuracy. Also in this case, as indicated by arrows 48 and 52, the hand detection area 42 and the fulcrum detection area 50 move in accordance with the movement of the face 38.

Moreover, as shown in FIG. 8, it is good also as a structure which detects only one motion of a user's right hand or left hand. In this case, a point of + a ₃ pixels or −a ₃ pixels in the X-axis direction and −b ₃ pixels in the Y-axis direction is set as the center position 58 of the hand detection area 56 with reference to the detected human face 38. Here, in FIG. 8, the case where the right hand of the user is detected is taken as an example, and a point of −a ₃ pixels in the X-axis direction and −b ₃ pixels in the Y-axis direction is set as the center position 58 with reference to the human face 38. . Then, ± a ₄ pixels in the X-axis direction with respect to the center position 58, and sets the hand detection region 56 of the ± b ₄ pixels in the Y-axis direction. Therefore, only the movement of one hand can be detected, and the operation of the DPF 2 not desired by the user due to erroneous detection of the movement of the hand can be prevented. Further, as indicated by the arrow 60, the hand detection area 56 moves in accordance with the movement of the face 38.

  In each of the above-described embodiments, i) satisfies that the user's face is in a specific region as the operation intention determination, and ii) the user's face area in the image is equal to or larger than a predetermined area, iii) One of four conditions: face detection time is a predetermined time or longer, vi) the operation speed of the user's face is a predetermined speed or less, and v) the area ratio of the face to the hand is within a predetermined range. However, as a configuration for determining that there is an operation intention when a predetermined number of predetermined conditions among the operation intention determination conditions i) to v) are satisfied. Alternatively, a configuration may be adopted in which it is determined that there is an operation intention when a determination criterion in which a predetermined number of specific operation intention determination conditions are combined is satisfied. Moreover, for example, when i) is satisfied, it may be determined that there is an operation intention when any one of the operation intention determination conditions of i) to v) is satisfied.

  Similarly, the command recognition process is terminated when the above-mentioned embodiment does not satisfy the above-mentioned i) or the conditions for ii) to v) as the termination condition of the command-recognition process in the above-described embodiment. , I) to v) may be configured to terminate the command recognition process when all the conditions are not satisfied. Moreover, it is good also as a structure which complete | finishes command recognition processing, when it stops satisfy | filling the predetermined number of conditions defined beforehand among these, and when it becomes no longer satisfying the determination criteria which combined the predetermined number of specific operation intention determination conditions Alternatively, the command recognition process may be terminated.

DESCRIPTION OF SYMBOLS 2 ... DPF, 4 ... Display part, 12 ... CPU, 13 ... Display control part, 14 ... Imaging part, 16 ... Memory, 18 ... Face detection part, 20 ... Face area calculation part, 26 ... Hand detection area setting part , 28 ... hand detection unit, 30 ... hand area calculation unit, 32 ... hand motion recognition unit, 34 ... hand motion storage unit

Claims (8)

A display for displaying an image;
An imaging unit for imaging an operator who appreciates an image displayed on the display unit;
A face detection unit that detects the face of the operator based on a captured image captured by the imaging unit;
A hand detection region setting unit that sets a hand detection region for detecting the operator's hand based on the position of the face in the captured image of the operator detected by the face detection unit;
A hand detection unit for detecting the hand of the operator in the hand detection region;
A command recognition unit for recognizing the operation, shape or position of the operator's hand detected by the hand detection unit as an operation instruction;
An image display device comprising: a display control unit that controls a display mode of an image displayed on the display unit based on the operation instruction recognized by the command recognition unit. A face position determination unit that determines whether or not the position of the operator's face detected by the face detection unit exists in a specific region in the captured image;
When the face position determination unit determines that the face position exists in the specific area, the hand detection area setting unit detects the operator's hand based on the position of the operator's face. The image display apparatus according to claim 1, wherein the hand detection area for setting is set. A face area calculation unit that calculates an area of the operator's face detected by the face detection unit;
A face area determination unit that determines whether or not the area of the face is a predetermined value or more,
When the face area determination unit determines that the area of the face is greater than or equal to the predetermined value, the hand detection region setting unit sets the hand detection region for detecting the operator's hand The image display device according to claim 1, wherein the image display device is an image display device. A detection time measurement unit that measures a face detection time during which the face of the operator is detected by the face detection unit;
A time determination unit that determines whether or not the face detection time is a predetermined time or more,
When the time determination unit determines that the face detection time is equal to or longer than a predetermined time, the hand detection region setting unit sets the hand detection region for detecting the operator's hand. The image display device according to claim 1, wherein the image display device is a display device. A movement detection unit for detecting movement of the face of the operator detected by the face detection unit;
When the motion speed detection unit detects that the movement of the face is equal to or less than a predetermined value, the hand detection region setting unit sets the hand detection region for detecting the operator's hand. The image display apparatus according to claim 1, wherein the image display apparatus is an image display apparatus. A face area calculation unit that calculates an area of the face in the captured image of the operator detected by the face detection unit;
A hand area calculation unit that calculates an area of a hand in the captured image of the operator detected by the hand detection unit;
An area comparison unit that compares the area of the face with the area of the hand;
The image display device according to claim 1, wherein the hand detection region setting unit sets the hand detection region for detecting the operator's hand based on a comparison result in the area comparison unit. A storage unit for storing the operation, shape or position of the operator's hand in association with a predetermined operation instruction;
When the command recognition unit recognizes that the movement, shape or position of the operator's hand is the movement, shape or position stored in the storage unit, the display control unit The image display apparatus according to claim 1, wherein an operation instruction associated with a shape or a position is executed.   The face detection unit stops detecting the operator's face when the hand detection region for detecting the operator's hand is set by the hand detection region setting unit. The image display apparatus as described in any one of Claims 1-7.

Images