JP2011095985A - Image display apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prevent the erroneous recognition of a command related to the operation instruction of a user based on a captured image. <P>SOLUTION: An image display apparatus includes: a display unit 4 for displaying an image; a light emitting unit 5 for emitting infrared light to an operator who appreciates an image displayed at the display unit; an imaging unit 14 for picking up the image of infrared light reflected by the operator; a face detection unit 18 for detecting the face of the operator based on the captured image captured by the imaging unit; a hand detection region setter 26 for setting a hand detection region for detecting the hand of the operator based on the position of the face in the captured image of the operator detected by the face detection unit; a hand detection unit 28 for detecting the hand of the operator in the hand detection region; a command recognition unit 32 for recognizing the operation, shape or position of the hand of the operator as an operation instruction based on the detection results of the hand detection unit; and a display control unit 13 for controlling the display mode of the image to be displayed at the display unit based on the operation instruction recognized by the command recognition unit. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image display device.

  There is known an image display device that includes a camera that captures a user and recognizes a command related to a user's operation instruction based on a captured image captured by the camera (see, for example, Patent Document 1). In this image display apparatus, command recognition is started by using a user's hand to the camera as a trigger.

JP 2009-75685 A

  However, in this image display device, it is necessary to always observe the movement of the user's hand as a trigger for command recognition in the entire captured image, and the load on the command recognition process related to the user's operation instruction is large. Further, particularly in a dark place, there is a risk of erroneously recognizing the action of the user's hand as a trigger.

  The objective of this invention is providing the image display apparatus which can prevent the misrecognition of the command which concerns on the user's operation instruction based on a captured image.

  An image display device according to the present invention includes a display unit that displays an image, a light emitting unit that emits infrared light to an operator who views the image displayed on the display unit, and a red light reflected by the operator. An imaging unit that captures external light, a face detection unit that detects the operator's face based on a captured image captured by the imaging unit, and a face in the captured image of the operator that is detected by the face detection unit A hand detection region setting unit for setting a hand detection region for detecting the operator's hand based on the position of the operator, a hand detection unit for detecting the operator's hand in the hand detection region, and the hand detection unit A command recognition unit for recognizing the operation, shape, or position of the operator's hand as an operation instruction based on the detection result of the image, and an image displayed on the display unit based on the operation instruction recognized by the command recognition unit Display mode Characterized in that it comprises a Gosuru display control unit.

  According to the image display device of the present invention, it is possible to prevent erroneous recognition of a command even in a dark place while reducing a load on a command recognition process related to a user operation instruction.

It is a figure which shows the front surface of DPF which concerns on the 1st Embodiment of this invention. It is a figure which shows the back surface of DPF which concerns on the 1st Embodiment of this invention. It is a block diagram which shows the system configuration | structure of DPF which concerns on the 1st Embodiment of this invention. It is a figure for demonstrating the hand detection area | region of DPF which concerns on the 1st Embodiment of this invention. It is a flowchart which shows the command recognition process of DPF which concerns on the 1st Embodiment of this invention. It is a flowchart which shows the command recognition process of DPF which concerns on the 2nd Embodiment of this invention. It is a figure for demonstrating the hand detection area | region of DPF which concerns on other embodiment of this invention. It is a figure for demonstrating the hand detection area | region of DPF which concerns on other embodiment of this invention.

  Hereinafter, an image display apparatus according to a first embodiment of the present invention will be described with reference to the drawings, taking a digital photo frame (hereinafter referred to as DPF) as an example. FIG. 1 is a diagram showing the front surface of the DPF according to the first embodiment, and FIG. 2 is a diagram showing the back surface. On the front surface of the DPF 2 is a display unit 4 configured by an LCD or the like that electrically displays information such as images and time, and an infrared ray that emits infrared light to a user who views the image displayed on the display unit 4 An LED 5 and an imaging lens 6 that images infrared light reflected by the user are provided. A support member 8 used for installing the DPF 2 is provided on the back surface of the DPF 2.

  FIG. 3 is a block diagram showing a system configuration of the DPF according to the first embodiment. The DPF 2 includes a CPU 12. The CPU 12 includes an infrared LED 5, a display control unit 13 that controls the display mode of the display unit 4 based on a command to be described later, and imaging that reflects infrared light reflected by the subject and through the imaging lens 6. An image capturing unit 14 including elements, a memory 16 that stores a captured image captured by the image capturing unit 14 for a predetermined number of frames or a predetermined time, and facial recognition by extracting feature points, a specific shape, and the like from captured image data A face detection unit 18 that generates data and detects the face of the user, a face area calculation unit 20 that calculates the area of the face detected from the face recognition data, a detection time measurement unit 22 that measures the detection time of the detected face, and Face motion detection unit 24 for detecting the motion speed of the face, hand detection region setting unit for setting a hand detection region for detecting the user's hand based on the detected face position 6. A hand detection unit 28 that detects a user's hand by extracting a feature point, a specific shape, and the like in the set hand detection region, a hand area calculation unit 30 that calculates an area in the captured image of the detected hand, and hand detection Hand motion recognition unit 32 for recognizing the user's hand motion, shape or position detected in the region, hand motion storage unit 34 for storing commands related to operation instructions in association with predetermined hand motion, shape or position, display A memory card 36 for storing images and information to be displayed on the unit 4 is connected. Here, the imaging unit 14 includes a monochrome imaging element to capture infrared rays reflected by the subject.

FIG. 4 is a diagram for explaining a DPF hand detection area according to the first embodiment. When the face detection unit 18 detects the user's face 38 in the captured image 40, a hand detection area of ± a ₁ pixel in the X-axis direction and ± b ₁ pixel in the Y-axis direction with reference to the approximate center of the detected face. 42 is set. Therefore, the hand detection unit 28 detects the hand 44 in the hand detection area 42, but does not detect the hand 46 outside the hand detection area 42. Further, as indicated by an arrow 48, the hand detection area 42 moves in accordance with the face motion.

  Next, the DPF command recognition process according to the first embodiment will be described with reference to the flowchart shown in FIG. The infrared LED 5 emits infrared light to a user who views the image displayed on the display unit 4 in the front direction of the DPF 2 (step S1), and the imaging unit 14 captures infrared light included in the subject light. If it is performed (step S2: Y), the captured image data for a predetermined number of frames or a predetermined time is stored in the memory 16 (step S3). When imaging by the imaging part 14 is not performed (step S2: N), it waits until imaging is performed.

  Next, the face detection unit 18 detects a human face from the captured image data, and determines whether or not a face has been detected (step S4). When a face is detected, it is determined whether or not the person whose face is detected has an intention to operate the DPF 2 by determining whether or not the operation intention determination conditions shown in steps S5 to S9 are met. To do. In other words, a person whose face is detected is determined as a user by using a condition that satisfies an operation intention determination condition as a trigger. On the other hand, if no face is detected, the process returns to step S1.

  If a plurality of human faces are detected, the face of the person located at the closest distance to the DPF 2 may be detected, or a person registered in advance may be detected. When detecting the face of a person located at the closest distance to the DPF, for example, the time from when infrared light is emitted by the infrared LED 5 until it is reflected by the subject and received by the imaging unit 14 of the DPF 2 is measured. Thus, the distance between the DPF 2 and the person's face may be obtained.

  When a face is detected in step S4, it is first determined whether or not the position of the face detected by the face detection unit 18 is in a specific area as an operation intention determination condition (step S5). Here, for example, when the face position is near the center of the image, it is determined that the face position is in the specific region. On the other hand, if the detected position of the face is the edge of the image, it is determined that the face position is outside the specific area. If the position of the face is not in the specific area (step S5: N), the process returns to step S1.

  When the face position is in the specific area (step S5: Y), the process proceeds to step S6, and it is determined whether or not the next operation intention determination condition is met. That is, it is determined whether or not the face area in the captured image calculated by the face area calculation unit 20 occupies a predetermined area or more (step S6). If it occupies a predetermined area or more, it is determined that the person whose face is detected has an intention to operate the DPF 2, and the process proceeds to step S10.

  On the other hand, when the detected face area is equal to or smaller than the predetermined area in the image (step S6: N), it is determined whether or not the face detection time measured by the detection time measuring unit 22 is equal to or longer than the predetermined time. (Step S7). If the face detection time is a predetermined time or more, that is, if the user is watching the display unit 4 continuously for a predetermined time or more, it is determined that the person whose face is detected has an intention to operate the DPF 2. The process proceeds to step S10.

  On the other hand, when the face detection time is equal to or shorter than the predetermined time (step S7: N), it is determined whether or not the face motion speed detected by the face motion detection unit 24 is equal to or lower than the predetermined speed (step S8). . If the speed is less than or equal to the predetermined speed, it is determined that the person whose face is detected has an intention to operate the DPF 2, and the process proceeds to step S10.

  On the other hand, when the facial motion speed is equal to or higher than the predetermined speed (step S8: N), the face area in the captured image calculated by the face area calculation unit 20 and the hand in the captured image calculated by the hand area calculation unit 30 A ratio between the face area and the hand area is calculated using the area, and it is determined whether or not the area ratio is within a predetermined range (step S9). If the area ratio is in the predetermined range, it is determined that the person whose face is detected has an intention to operate the DPF 2, and the process proceeds to step S10. On the other hand, when the area ratio of the face and the hand is not within the predetermined range (step S9: N), the process is terminated.

If it is determined in steps S5 to S9 that the person whose face is detected has an intention to operate the DPF 2, the person whose face is detected is determined to be the user, and the user's hand is detected. A detection area is set (step S10). At this time, as shown in FIG. 4, a hand detection area 42 of ± a ₁ pixel in the X-axis direction and ± b ₁ pixel in the Y-axis direction is set with reference to the approximate center of the user's face.

  When the hand detection unit 28 detects the user's hand in the hand detection area 42 (step 11: Y), the CPU 12 starts control of the hand motion recognition unit 32 and analyzes the motion, shape, or position of the hand (step S12). . Although not particularly limited, when analyzing the shape and position of the hand, for example, one frame of the photographed image is analyzed, and when analyzing the motion, a plurality of frames are analyzed. When a plurality of hands are detected in the hand detection area 42, the motion, shape, or position of the hand closest to the DPF 2 may be analyzed, or the hand area calculated by the hand area calculation unit 30 may be analyzed. The motion, shape or position of the hand with the largest may be analyzed.

  When the analyzed hand motion, shape or position corresponds to the hand motion, shape or position stored in the hand motion storage unit 34 (step S13: Y), the recognized hand motion, shape or position is recognized. A command related to the operation instruction associated with is executed (step S14). For example, in the case of moving the hand from left to right, image feed is executed. If the operation is to close the hand from the open state, a process of turning off the power is executed. When the palm is brought close to the DPF 2, the image displayed on the display unit 4 is enlarged, and when the palm is moved away, the image is reduced. Even when the DPF 2 performs an operation of opening from the closed state in the sleep state in which the image is captured by the imaging unit 14 without performing the display on the display unit 4, the process of recovering from the sleep state to the activated state is performed. Good. In addition to the movement, shape, or position of the hand, for example, when both arms are crossed and a cross is formed, a process of turning off the power may be executed. Note that after the process of step S13, the captured image data for a predetermined number of frames or a predetermined time stored in the memory 16 may be deleted.

  If the hand is not detected in step S11, the hand motion, shape or position analyzed in step S13 does not correspond to the hand motion, shape or position stored in the hand motion storage unit 34. The process returns to step S1.

  According to the DPF according to the present embodiment, since the operation intention is determined and the hand motion recognition is performed for the person who is recognized for the operation intention, it is possible to prevent the DPF operation unintended by the user.

  In addition, since the person's face is detected and the operation intention is determined on the condition that the detected face is in the specific area, it is possible to reduce the load on the command recognition process related to the user's operation instruction.

  In addition, when a hand is not detected in the hand detection area, hand motion recognition is not performed. Therefore, it is possible to reduce the load on the command recognition processing related to the user's operation instruction, and to recognize the motion of hands other than the user. It is possible to prevent DPF operation unintended by the user.

  In addition, since infrared light is imaged, command recognition related to user operation instructions can be performed even in a dark place. In addition, since it can be configured with a monochrome camera for imaging infrared light, manufacturing costs can be reduced.

  In the above-described embodiment, the operation intention determination is performed for one person when there are a plurality of persons detected by the face detection unit 18, but the operation intention is determined for all the detected persons. A configuration may be adopted in which a hand detection area is set for a person who is determined to make an operation intention and a hand is detected.

  Next, DPF command recognition processing according to the second embodiment of the present invention will be described with reference to the drawings. In the first embodiment, the command recognition process is performed for a predetermined number of frames or captured images for a predetermined time. However, in the second embodiment, the hand is tracked after the hand is detected. Thus, the configuration for performing the command recognition process is changed. Therefore, a detailed description of the same configuration as that of the first embodiment is omitted, and only different portions will be described in detail. Further, the same components as those in the first embodiment will be described with the same reference numerals.

  FIG. 6 is a flowchart showing command recognition processing according to the second embodiment. The infrared LED 5 starts emitting infrared light to the user viewing the image displayed on the display unit 4 in the forward direction of the DPF 2 (step S21), and starts imaging the infrared light included in the subject light by the imaging unit 14. Then (step S22: Y), storage of the captured image data in the memory 16 is started (step S23). When imaging by the imaging unit 14 is not started (step S22: N), the process waits until imaging is started.

  Next, the face detection unit 18 detects a human face from the captured image data, and determines whether or not a face has been detected (step S24). When a face is detected, it is determined whether or not the person whose face is detected has an intention to operate the DPF 2 by determining whether or not the operation intention determination conditions shown in steps S25 to S29 are met. . On the other hand, when a face is not detected, it waits until a face is detected.

  The processing shown in steps S25 to S30 is the same as the processing shown in steps S5 to S10 in the flowchart of FIG. If the face position is not in the specific area in step S25 or if the area ratio of the face to the hand is not within the predetermined range in step S29, the process returns to step S24 and the face is detected for new captured image data. It is determined whether or not it has been done.

  When the hand detection unit 28 detects the user's hand in the hand detection area 42 (step 31: Y), the CPU 12 starts control of the hand motion recognition unit 32 and analyzes the hand motion, shape, and position (step S32). . When the analyzed hand motion corresponds to the hand motion, shape, or position stored in the hand motion storage unit 34 (step S33: Y), an operation instruction associated with the hand motion, shape, or position The command related to is executed (step S34).

  If no hand is detected in step S31, the hand motion, shape or position analyzed in step S33 does not correspond to the hand motion, shape or position stored in the hand motion storage unit 34. The process proceeds to step S35.

  Next, it is determined whether or not all the operation intention determination conditions are satisfied (step S35). If the operation intention determination conditions are not satisfied, the process is terminated. That is, i) If the user's face does not satisfy the specific area, the imaging is stopped (step S36), and the process is terminated. Also, ii) the user's face area in the image is greater than or equal to a predetermined area, iii) the face detection time is greater than or equal to a predetermined time, vi) the user's face motion speed is less than a certain speed, v) When none of the four conditions that the area ratio of the face and the hand is within the predetermined range is satisfied, the imaging is stopped (step S36), and the process is terminated. On the other hand, if the above-described face detection condition i) is satisfied and any one of ii) to v) is satisfied, the process returns to step S31 to determine whether or not a hand has been detected for the new captured image data. continue.

  According to the DPF according to the present embodiment, since the operation intention is determined and the hand motion recognition is performed for the person who is recognized for the operation intention, the operation of the DPF that is not intended by the user can be prevented.

  In addition, since the person's face is detected, the operation intention is determined on the condition that the detected face is in the specific area, and the face detection is not performed once the face is detected. The load on command recognition can be reduced. Further, the process may be terminated when no hand is detected in step S35.

In each of the above-described embodiments, the configuration in which the hand is detected and the motion, shape, or position of the hand is analyzed has been described as an example. However, as shown in FIG. Good. In this case, a fulcrum detection area 50 of ± a ₂ pixels in the X-axis direction and −b ₂ pixels in the Y-axis direction is set based on the detected human face 38. The hand detection unit 28 detects a hand only when the fulcrum detection region 50 has, for example, an elbow 52 which is a fulcrum and the hand detection region 42 has a hand 44, and the hand motion recognition unit 32 detects the motion and shape of the hand. Or perform position analysis. Therefore, the hand can be detected with high accuracy. In addition, it is possible to perform command recognition related to user operation instructions with high accuracy even in a dark place. Also in this case, as indicated by arrows 48 and 52, the hand detection area 42 and the fulcrum detection area 50 move in accordance with the movement of the face 38.

Moreover, as shown in FIG. 8, it is good also as a structure which detects only one motion of a user's right hand or left hand. In this case, a point of + a ₃ pixels or −a ₃ pixels in the X-axis direction and −b ₃ pixels in the Y-axis direction is set as the center position 58 of the hand detection area 56 with reference to the detected human face 38. Here, in FIG. 8, the case where the right hand of the user is detected is taken as an example, and a point of −a ₃ pixels in the X-axis direction and −b ₃ pixels in the Y-axis direction is set as the center position 58 with reference to the human face 38. . Then, ± a ₄ pixels in the X-axis direction with respect to the center position 58, and sets the hand detection region 56 of the ± b ₄ pixels in the Y-axis direction. Therefore, only the movement, shape and position of one hand can be detected, and the operation of the DPF 2 which is not desired by the user due to erroneous detection of the movement, shape or position of the hand can be prevented. Further, as indicated by the arrow 60, the hand detection area 56 moves in accordance with the movement of the face 38.

  In each of the above-described embodiments, i) satisfies that the user's face is in a specific region as the operation intention determination, and ii) the user's face area in the image is equal to or larger than a predetermined area, iii) One of four conditions: face detection time is a predetermined time or longer, vi) the operation speed of the user's face is a predetermined speed or less, and v) the area ratio of the face to the hand is within a predetermined range. However, as a configuration for determining that there is an operation intention when a predetermined number of predetermined conditions among the operation intention determination conditions i) to v) are satisfied. Alternatively, a configuration may be adopted in which it is determined that there is an operation intention when a determination criterion in which a predetermined number of specific operation intention determination conditions are combined is satisfied. Moreover, for example, when i) is satisfied, it may be determined that there is an operation intention when any one of the operation intention determination conditions of i) to v) is satisfied.

  Similarly, the command recognition process is terminated when the above-mentioned embodiment does not satisfy the above-mentioned i) or the conditions for ii) to v) as the termination condition of the command-recognition process in the above-described embodiment. , I) to v) may be configured to terminate the command recognition process when all the conditions are not satisfied. Moreover, it is good also as a structure which complete | finishes command recognition processing, when it stops satisfy | filling the predetermined number of conditions defined beforehand among these, and when it becomes no longer satisfying the determination criteria which combined the predetermined number of specific operation intention determination conditions Alternatively, the command recognition process may be terminated.

DESCRIPTION OF SYMBOLS 2 ... DPF, 4 ... Display part, 5 ... Infrared LED, 12 ... CPU, 13 ... Display control part, 14 ... Imaging part, 16 ... Memory, 18 ... Face detection part, 20 ... Face area calculation part, 26 ... Hand detection area setting unit, 28 ... hand detection unit, 30 ... hand area calculation unit, 32 ... hand motion recognition unit, 34 ... hand motion storage unit

Claims (5)

A display for displaying an image;
A light emitting unit for emitting infrared light to an operator who views an image displayed on the display unit;
An imaging unit for imaging infrared light reflected by the operator;
A face detection unit that detects the face of the operator based on a captured image captured by the imaging unit;
A hand detection region setting unit that sets a hand detection region for detecting the operator's hand based on the position of the face in the captured image of the operator detected by the face detection unit;
A hand detection unit for detecting the hand of the operator in the hand detection region;
A command recognition unit that recognizes the operation, shape, or position of the operator's hand as an operation instruction based on a detection result by the hand detection unit;
An image display device comprising: a display control unit that controls a display mode of an image displayed on the display unit based on the operation instruction recognized by the command recognition unit.   2. The hand detection area setting unit determines whether or not to set a hand detection area for detecting the operator's hand based on a detection result by the face detection unit. Image display device. A storage unit for storing the operation, shape or position of the operator's hand in association with a predetermined operation instruction;
When the command recognition unit recognizes that the movement, shape or position of the operator's hand is the movement, shape or position of the hand stored in the storage unit, the display control unit The image display apparatus according to claim 1, wherein an operation instruction associated with a movement, shape, or position of a hand is executed.   The face detection unit stops detecting the operator's face when the hand detection region for detecting the operator's hand is set by the hand detection region setting unit. The image display apparatus as described in any one of Claims 1-3. A face area calculation unit that calculates an area of the face in the captured image of the operator detected by the face detection unit;
A hand area calculation unit that calculates an area of a hand in the captured image of the operator detected by the hand detection unit;
An area comparison unit that compares the area of the face with the area of the hand;
The said hand detection area | region setting part sets the said hand detection area | region for detecting the said operator's hand based on the comparison result in the said area comparison part, The any one of Claims 1-4 characterized by the above-mentioned. The image display device described in 1.

Images