WO2021229741A1 - ジェスチャ検出装置およびジェスチャ検出方法 - Google Patents

ジェスチャ検出装置およびジェスチャ検出方法 Download PDF

Info

Publication number
WO2021229741A1
WO2021229741A1 PCT/JP2020/019231 JP2020019231W WO2021229741A1 WO 2021229741 A1 WO2021229741 A1 WO 2021229741A1 JP 2020019231 W JP2020019231 W JP 2020019231W WO 2021229741 A1 WO2021229741 A1 WO 2021229741A1
Authority
WO
WIPO (PCT)
Prior art keywords
hand
detection
information
image
gesture
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2020/019231
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
瑞貴 川瀬
乃輔 大橋
勇樹 江戸
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Mitsubishi Electric Corp
Original Assignee
Mitsubishi Electric Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mitsubishi Electric Corp filed Critical Mitsubishi Electric Corp
Priority to PCT/JP2020/019231 priority Critical patent/WO2021229741A1/ja
Priority to US17/915,176 priority patent/US20230123623A1/en
Priority to JP2022522427A priority patent/JP7325625B2/ja
Publication of WO2021229741A1 publication Critical patent/WO2021229741A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/98—Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/60—Extraction of image or video features relating to illumination properties, e.g. using a reflectance or lighting model
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/107—Static hand or arm
    • G06V40/11—Hand-related biometrics; Hand pose recognition
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language

Definitions

  • This disclosure relates to a gesture detection device and a gesture detection method.
  • Patent Document 1 proposes an image processing device that improves the detection accuracy of a region of interest including a person's hand included in image data.
  • the gesture detection device detects the occupant's hand based on the image. Therefore, depending on the state of the image, the detected object detected as the occupant's hand may not be the true hand.
  • the present disclosure is for solving the above-mentioned problems, and an object of the present disclosure is to provide a gesture detection device that accurately determines whether or not a hand detected based on an image is a true hand.
  • the gesture detection device includes a detection information acquisition unit, a determination unit, and a rejection unit.
  • the detection information acquisition unit acquires the hand detection result in the gesture of the hand of the occupant of the vehicle and the hand information in the image in which the hand is detected.
  • the hand of the occupant of the vehicle is detected based on the image taken by the image pickup device provided in the vehicle.
  • the determination unit determines whether or not the hand is a true hand based on at least one predetermined condition for the hand information in the video.
  • the rejection unit rejects the detection result of the hand detected based on the image when the hand is not a true hand.
  • a gesture detection device that accurately determines whether or not a hand detected based on an image is a true hand.
  • FIG. It is a functional block diagram which shows the structure of the gesture detection apparatus in Embodiment 1.
  • FIG. It is a figure which shows an example of the structure of the processing circuit included in a gesture detection device. It is a figure which shows another example of the structure of the processing circuit included in the gesture detection apparatus. It is a flowchart which shows the gesture detection method in Embodiment 1.
  • FIG. It is a functional block diagram which shows the structure of the gesture detection apparatus in Embodiment 2.
  • FIG. It is a figure which shows the information of a hand in the image in which a hand is detected, and an example of a predetermined condition. It is a figure which shows an example of the open hand detected by the detection part. It is a figure which shows an example of the closed hand detected by the detection part.
  • FIG. 1 It is a figure which shows an example of the closed hand detected by the detection part. It is a figure which shows an example of the closed hand detected by the detection part. It is a flowchart which shows the gesture detection method in Embodiment 2. It is a functional block diagram which shows the structure of the gesture detection device and the device which operates in connection with it in Embodiment 3. FIG.
  • FIG. 1 is a functional block diagram showing the configuration of the gesture detection device 100 according to the first embodiment. Further, FIG. 1 shows an image pickup device 110 as a device that operates in connection with the gesture detection device 100. The image pickup device 110 is provided in the vehicle. The image pickup device 110 captures an image of an occupant inside the vehicle. The gesture detection device 100 detects the gesture of the hand of the occupant of the vehicle based on the image.
  • the gesture detection device 100 includes a detection unit 10, a detection information acquisition unit 20, a determination unit 30, and a rejection unit 40.
  • the detection unit 10 detects the hand of the occupant of the vehicle based on the image taken by the image pickup device 110.
  • the detection information acquisition unit 20 acquires the hand detection result and the hand information in the video in which the hand is detected.
  • the hand detection result includes, for example, information such as the position coordinates (detection position) of the occupant's hand in the image, the contour of the hand, and the area surrounding the hand.
  • the hand detection result is information for operating the device (vehicle-mounted device) mounted on the vehicle by the gesture of the hand.
  • the hand information in the image is, for example, the brightness of the hand in the image, the position of the hand, the size of the hand, the amount of movement of the hand, the angle of the hand, the texture of the hand (the pattern depending on the brightness), the shape of the hand, etc. Contains information.
  • the hand information is information used to determine whether the detected hand is a true hand.
  • the determination unit 30 determines whether or not the hand detected based on the image is a true hand based on at least one predetermined condition regarding the information of the hand in the image.
  • the rejection unit 40 rejects the detection result of that hand.
  • the gesture detection device 100 does not use the detection result of the rejected hand by the rejection unit 40 in the subsequent processing, or does not output the detection result of the rejected hand to the outside.
  • FIG. 2 is a diagram showing an example of the configuration of the processing circuit 90 included in the gesture detection device 100.
  • Each function of the detection unit 10, the detection information acquisition unit 20, the determination unit 30, and the rejection unit 40 is realized by the processing circuit 90. That is, the processing circuit 90 has a detection unit 10, a detection information acquisition unit 20, a determination unit 30, and a rejection unit 40.
  • the processing circuit 90 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), or an FPGA (Field). -ProgrammableGateArray), or a circuit that combines these.
  • the functions of the detection unit 10, the detection information acquisition unit 20, the determination unit 30, and the rejection unit 40 may be individually realized by a plurality of processing circuits, or may be collectively realized by one processing circuit.
  • FIG. 3 is a diagram showing another example of the configuration of the processing circuit included in the gesture detection device 100.
  • the processing circuit includes a processor 91 and a memory 92.
  • the processor 91 executes the program stored in the memory 92, the functions of the detection unit 10, the detection information acquisition unit 20, the determination unit 30, and the rejection unit 40 are realized.
  • each function is realized by executing the software or firmware described as a program by the processor 91.
  • the gesture detection device 100 has a memory 92 for storing the program and a processor 91 for executing the program.
  • the program describes a function in which the gesture detection device 100 acquires the hand detection result in the gesture of the hand of the occupant of the vehicle and the hand information in the video in which the hand is detected.
  • the detection result of the hand is detected based on the image taken by the image pickup device 110 provided in the vehicle.
  • the program also includes a function for determining whether or not the hand is a true hand based on at least one predetermined condition for the hand information in the video. Further, the program describes a function of rejecting the detection result of the hand detected based on the image when the hand is not a true hand.
  • the program causes a computer to execute the procedure or method of the detection unit 10, the detection information acquisition unit 20, the determination unit 30, and the rejection unit 40.
  • the processor 91 is, for example, a CPU (Central Processing Unit), an arithmetic unit, a microprocessor, a microcomputer, a DSP (Digital Signal Processor), or the like.
  • the memory 92 is, for example, non-volatile or volatile such as RAM (RandomAccessMemory), ROM (ReadOnlyMemory), flash memory, EPROM (ErasableProgrammableReadOnlyMemory), EEPROM (ElectricallyErasableProgrammableReadOnlyMemory). It is a semiconductor memory.
  • the memory 92 may be any storage medium used in the future, such as a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, or a DVD.
  • Each function of the detection unit 10, the detection information acquisition unit 20, the determination unit 30, and the rejection unit 40 may be partially realized by dedicated hardware and the other part may be realized by software or firmware. In this way, the processing circuit realizes each of the above functions by hardware, software, firmware, or a combination thereof.
  • FIG. 4 is a flowchart showing the gesture detection method in the first embodiment.
  • step S1 the detection unit 10 detects the hand of the occupant of the vehicle based on the image captured by the image pickup device 110.
  • step S2 the detection information acquisition unit 20 acquires the hand detection result and the hand information in the video in which the hand is detected.
  • step S3 the determination unit 30 determines whether or not the hand detected based on the image is a true hand based on at least one predetermined condition regarding the information of the hand in the image.
  • step S4 when the hand detected based on the image is not a true hand, the rejection unit 40 rejects the detection result of that hand. This completes the gesture detection method.
  • the gesture detection device 100 in the first embodiment includes a detection information acquisition unit 20, a determination unit 30, and a rejection unit 40.
  • the detection information acquisition unit 20 acquires the hand detection result in the gesture of the hand of the occupant of the vehicle and the hand information in the image in which the hand is detected.
  • the hand of the occupant of the vehicle is detected based on the image taken by the image pickup device 110 provided in the vehicle.
  • the determination unit 30 determines whether or not the hand is a true hand based on at least one predetermined condition for the hand information in the video.
  • the rejection unit 40 rejects the detection result of the hand detected based on the image when the hand is not a true hand.
  • Such a gesture detection device 100 accurately determines whether or not the hand detected based on the image is a true hand.
  • the gesture detection method in the first embodiment acquires the hand detection result in the gesture of the hand of the occupant of the vehicle and the hand information in the image in which the hand is detected.
  • the hand of the occupant of the vehicle is detected based on the image taken by the image pickup device 110 provided in the vehicle.
  • the gesture detection method determines whether or not the hand is a true hand based on at least one predetermined condition for the hand information in the video, and if the hand is not a true hand, the video. Reject the detection result of the hand detected based on.
  • the gesture detection device and the gesture detection method according to the second embodiment will be described.
  • the second embodiment is a subordinate concept of the first embodiment, and the gesture detection device according to the second embodiment includes each configuration of the gesture detection device 100 according to the first embodiment. The same configuration and operation as in the first embodiment will be omitted.
  • FIG. 5 is a functional block diagram showing the configuration of the gesture detection device 101 according to the second embodiment.
  • the image pickup device 110 is provided in the front center of the vehicle interior.
  • the image pickup apparatus 110 photographs the interior of the vehicle at a wide angle, and photographs both the driver's seat and the passenger seat at the same time.
  • the image pickup device 110 is, for example, a camera that shoots infrared rays, a camera that shoots visible light, and the like.
  • the gesture detection device 101 according to the second embodiment detects the gesture of the hand of the occupant of the vehicle based on the image captured by the image pickup device 110.
  • the gesture is a gesture for operating a device (vehicle-mounted device) mounted on a vehicle.
  • the in-vehicle device is, for example, an air conditioner, an audio system, or the like.
  • the gesture detected by the gesture detection device 101 controls the temperature of the air conditioner, adjusts the volume of the audio, and the like.
  • the gesture detection device 101 includes a video information acquisition unit 50, a detection unit 10, a detection information acquisition unit 20, a determination unit 30, and a rejection unit 40.
  • the video information acquisition unit 50 acquires the video captured by the image pickup device 110.
  • the detection unit 10 detects the hand of the occupant of the vehicle based on the image.
  • the detection unit 10 detects the occupant's hand, for example, by matching the information captured in the image with a predetermined hand shape. More specifically, the detection unit 10 detects the position coordinates (detection position) of the occupant's hand in the image.
  • the detection unit 10 may detect the contour of the occupant's hand or the area surrounding the hand.
  • the area surrounding the hand is, for example, a rectangular frame area including the outline of the hand.
  • the hand detection result in the second embodiment includes at least one of the detection position of the hand, the contour of the hand, and the area surrounding the hand.
  • the detection information acquisition unit 20 acquires the detection result of the hand and the information of the hand in the image in which the hand is detected. The details of the hand information will be described later.
  • the detection information acquisition unit 20 acquires, for example, hand information from the detection unit 10.
  • the detection information acquisition unit 20 may acquire the information by requesting the information of the hand by itself based on the image in which the hand is detected.
  • the determination unit 30 determines whether or not the hand detected by the detection unit 10 is a true hand based on predetermined conditions for the hand information in the video.
  • FIG. 6 is a diagram showing an example of hand information in a video in which a hand is detected and predetermined conditions for the hand information.
  • the determination unit 30 in the second embodiment determines whether or not the hand detected by the detection unit 10 is a true hand based on any one or more predetermined conditions shown below.
  • the hand detected by the detection unit 10 shows an open hand (par hand) and a closed hand (thumbs-up hand) as an example, but is limited to the hand in those states. It's not a thing.
  • the hand detected by the detection unit 10 may be a hand indicating a number, a hand indicating a direction, or the like.
  • the detection information acquisition unit 20 acquires, for example, information on the detection luminance difference in the image in which the hand is detected, as information on the hand.
  • the detected luminance difference is the luminance difference between the luminance of the hand and the luminance around the hand.
  • 7 and 8 are diagrams showing an example of an open hand (par hand) and a closed hand (thumbs-up hand) detected by the detection unit 10, respectively.
  • the detected luminance difference corresponds to the luminance difference between the luminance of the hand 2 in the vicinity of the boundary between the hand 2 and the background 3 in the video and the luminance of the background 3 in the vicinity of the boundary.
  • the determination unit 30 determines that the hand detected based on the image is the true hand. That is, the predetermined condition for the above-mentioned hand information is a condition regarding the brightness difference between the brightness of the hand and the brightness around the hand.
  • the detection information acquisition unit 20 acquires information on the position of the hand in the image, for example, as information on the hand.
  • the determination unit 30 determines that the detected hand is a true hand.
  • the predetermined range is, for example, a range in which the detection unit 10 detects a hand based on an image. That is, the predetermined condition for the above-mentioned hand information is a condition regarding the position of the hand.
  • the detection information acquisition unit 20 may acquire information on the size of the hand in addition to the position of the hand. As a result, the determination unit 30 determines whether or not the entire hand is included in the detection range.
  • the determination unit 30 determines that the hand is a true hand. ..
  • the determination unit 30 determines that the hand is a true hand. judge.
  • the detection information acquisition unit 20 acquires, for example, information on the brightness of the hand and the size of the hand in the image in which the hand is detected, as information on the hand.
  • the determination unit 30 makes a determination regarding the size of the hand shown below. For example, when the brightness of the hand is overexposed, the size of the hand is not determined.
  • the determination unit 30 determines that the hand detected based on the image is the true hand. That is, the predetermined conditions for the above-mentioned hand information are the conditions regarding the brightness of the hand and the size of the hand.
  • 9 and 10 are diagrams showing an example of an open hand and a closed hand detected by the detection unit 10, respectively.
  • the size of the hand is defined, for example, by the number of pixels in the vertical direction (A direction in each figure) or the number of pixels in the horizontal direction (B direction in each figure) of the hand in the image.
  • the determination unit 30 determines that the hand is a true hand.
  • the detection information acquisition unit 20 acquires, for example, information on the amount of movement of the center of the hand in the image in which the hand is detected, as information on the hand.
  • the determination unit 30 determines that the hand detected based on the image is the true hand. That is, the predetermined condition for the above-mentioned hand information is a condition regarding the amount of movement of the hand.
  • the amount of movement is, for example, the number of pixels per unit time.
  • 11 and 12 are diagrams showing an example of an open hand and a closed hand detected by the detection unit 10, respectively.
  • the determination unit 30 determines that the number of pixels corresponding to the movement amount of the center of the palm is within a predetermined number of pixels. Determine that the hand is a true hand.
  • the determination unit 30 has a pixel number corresponding to the movement amount of the center of the thumbs-up hand within a predetermined pixel number range. If so, it is determined that the hand is a true hand.
  • the detection information acquisition unit 20 acquires information on the angle of the hand in the image in which the hand is detected, for example, as information on the hand.
  • the determination unit 30 determines that the hand detected based on the image is the true hand. That is, the predetermined condition for the above-mentioned hand information is a condition regarding the angle of the hand.
  • the angle of the hand is calculated, for example, by the detection unit 10 matching the shape of the hand in the image with the predetermined shape of the hand.
  • the angle of the hand corresponds to the angle formed by the palm and the front surface (the surface including the image pickup surface) of the image pickup apparatus 110.
  • the palm may be read as the back of the hand.
  • 13 and 14 are diagrams showing an example of an open hand and a closed hand detected by the detection unit 10, respectively. Hand angles include Roll angle, Pitch angle and Yaw angle.
  • the detection information acquisition unit 20 acquires, for example, information on the texture of the hand (pattern due to brightness) in the image in which the hand is detected, as information on the hand.
  • Information on the texture of the hand is obtained, for example, based on the brightness distribution of the hand in the image. If the texture matches or resembles a predetermined texture, the determination unit 30 determines that the hand detected based on the image is the true hand. That is, the predetermined condition for the above-mentioned hand information is a condition regarding the texture of the hand.
  • the detection information acquisition unit 20 acquires information on the shape of the hand in the image in which the hand is detected, for example, as information on the hand.
  • the determination unit 30 determines that the hand is a true hand based on the shape of the hand. That is, the predetermined condition for the above-mentioned hand information is a condition regarding the shape of the hand.
  • 15 and 16 are diagrams showing an example of an open hand detected by the detection unit 10, respectively. In the hand shown in FIG. 15, the fingers are in contact with each other. On the other hand, in the hand shown in FIG. 16, there is a gap between the fingers.
  • the determination unit 30 determines that the hand is a true hand, for example, when there is a gap of a predetermined size between the fingers in the image.
  • the condition regarding the shape of the hand is the condition regarding the degree of opening of the fingers.
  • 17 and 18 are diagrams showing an example of an open hand detected by the detection unit 10, respectively. In the hand shown in FIG. 17, the fingers are extended. On the other hand, in the hand shown in FIG. 18, the finger is bent. For example, when the angle of the finger is within a predetermined angle range with respect to the surface defined by the palm, the determination unit 30 determines that the hand is a true hand.
  • the condition regarding the shape of the hand is the condition regarding the degree of bending of the finger.
  • 19 and 20 are diagrams showing an example of a closed hand detected by the detection unit 10, respectively. The thumbs-up hand shown in FIG. 19 is firmly held except for the thumb.
  • the fingers other than the thumb are half-opened.
  • the determination unit 30 determines that the hand is a true hand.
  • the condition regarding the shape of the hand is the condition regarding how to hold the closed hand.
  • the determination unit 30 determines, for example, that the hand is a true hand when the number of fingers of the hand detected by the detection unit 10 is equal to or greater than a predetermined number. May be good.
  • the condition regarding the shape of the hand is the condition regarding the defect of the finger.
  • the rejection unit 40 rejects the detection result of the hand detected by the detection unit 10 when the hand is not a true hand.
  • the gesture detection device 101 does not use the detection result of the rejected hand in the subsequent processing or outputs it to the outside. "Rejecting” may mean invalidating the hand detection result or not adopting it as the hand detection result.
  • FIG. 21 is a flowchart showing the gesture detection method in the second embodiment.
  • step S10 the video information acquisition unit 50 acquires the video captured by the image pickup device 110.
  • step S20 the detection unit 10 detects the hand of the occupant of the vehicle based on the image.
  • step S30 the detection information acquisition unit 20 acquires the hand detection result and the hand information in the video in which the hand is detected.
  • step S40 the determination unit 30 determines whether or not the hand detected by the detection unit 10 is a true hand based on a predetermined condition regarding the hand information. If the hand is determined to be a true hand, step S50 is executed. If it is determined that the hand is not a true hand, step S60 is executed.
  • step S50 the gesture detection device 101 adopts the detection result of the hand determined by the determination unit 30 to be the true hand.
  • the gesture detection device 101 uses the detection result of the true hand in the subsequent processing or outputs it to the outside.
  • step S60 the rejection unit 40 rejects the detection result of the hand determined by the determination unit 30 to be not a true hand.
  • the gesture detection device 101 does not use the detection result of the rejected hand in the subsequent processing or outputs it to the outside. This completes the gesture detection method.
  • the gesture detection device 101 rejects an inaccurate detection result from the detection results of the hand once detected based on a predetermined condition. In other words, the gesture detection device 101 selects only highly accurate detection results. Therefore, the detection accuracy of the hand is improved. As a result, the operation accuracy of the in-vehicle device is improved.
  • the occupant When the occupant operates the equipment mounted on the vehicle, the occupant may look into the information displayed on the dashboard, center console, etc. of the vehicle to check it. In that state, if the occupant makes a gesture to operate the device, the image shows the head larger and brighter than the occupant's hand.
  • the detection unit 10 erroneously detects the occupant's head in the image as a grasped hand.
  • the gesture detection device 101 in the second embodiment rejects the detection result of the head which is larger than the predetermined size and has high brightness. That is, the gesture detection device 101 rejects the detection result of the erroneously detected occupant's head based on the conditions regarding the size and brightness of the hand in the image.
  • the baby's head may be detected as a occupant's hand. Even in that case, the gesture detection device 101 rejects the detection result of the erroneously detected baby's head based on the predetermined conditions. As a result, the accuracy of hand detection is improved.
  • the image pickup device 110 is also photographing a part other than the occupant's hand. Therefore, the detection unit 10 erroneously detects the wrinkle pattern of the occupant's mask or the brightness distribution based on the crease pattern of the occupant's clothes as the occupant's hand. The positions of the occupant's masks and wrinkled parts of clothing do not move significantly, even when the occupant makes gestures to operate the equipment. On the other hand, the occupant's hand is located on the steering wheel during normal driving. When the occupant makes a gesture for operating the device, he / she moves his / her hand to the center console side where the image pickup device 110 is provided and then makes the gesture.
  • the gesture detection device 101 rejects the result of erroneous detection based on the condition regarding the amount of movement of the hand in the video. As a result, the accuracy of hand detection is improved. Similarly, when the scenery outside the vehicle, for example, the shape of a cloud is erroneously detected as a hand, the same effect is obtained when the accessory worn by the occupant is detected as a hand.
  • the patterns of wrinkles, gloves, accessories, tattoos, etc. on clothes worn by occupants differ from those of bare hands in the reflection characteristics for near-infrared light. Therefore, when the image captured by the image pickup device 110 is a near-infrared image, the gesture detection device 101 rejects the result of erroneous detection based on the condition regarding the texture of the hand.
  • the conditions relating to the texture may include the conditions of the infrared image of the hand, the conditions of the infrared image of the palm line, and the like. As a result, the accuracy of hand detection is improved.
  • the detection unit 10 detects the hand of the occupant of the vehicle and the skeleton of the occupant of the vehicle based on the image captured by the image pickup device 110. At the time of hand detection, the detection unit 10 detects the occupant's hand, for example, by matching the information captured in the image with a predetermined hand shape. When detecting the skeleton, the detection unit 10 detects the skeleton of the occupant by, for example, matching the information captured in the image with a predetermined body shape.
  • the detection information acquisition unit 20 acquires the detection result of the hand, the information of the hand in the image in which the hand is detected, and the information of the skeleton.
  • the hand information in the first modification of the second embodiment is information on the position of the hand in the image.
  • the determination unit 30 determines that if the position of the occupant's hand is at a predetermined position in the skeleton, the hand detected based on the image is the true hand. Judge that there is. That is, in the first modification of the second embodiment, the predetermined condition for the hand information is the condition regarding the position of the hand in the skeleton of the occupant in the image.
  • the rejection unit 40 rejects the detection result of the hand detected by the detection unit 10 when the hand is not a true hand.
  • the gesture detection device 101 having such a configuration rejects an inaccurate detection result from the detection results of the hand once detected based on a predetermined condition. Therefore, the detection accuracy of the hand is improved. As a result, the operation accuracy of the in-vehicle device is improved.
  • the determination unit 30 in the second modification of the second embodiment combines a plurality of predetermined conditions to determine whether or not the hand is a true hand. For example, the determination unit 30 makes a determination based on the condition regarding the brightness difference between the brightness of the hand and the brightness around the hand. As a result of the determination, the determination unit 30 again makes a determination using the condition regarding the size of the hand for the hand determined to be a true hand.
  • the plurality of predetermined conditions are various combinations of the conditions exemplified in the second embodiment.
  • Such a gesture detection device 101 more accurately detects the hand in the gesture of the occupant's hand of the vehicle. Therefore, false detection is reduced.
  • the gesture detection device shown in each of the above embodiments can be applied to a system constructed by appropriately combining a navigation device, a communication terminal, a server, and the functions of applications installed in the navigation device.
  • the navigation device includes, for example, a PND (Portable Navigation Device) and the like.
  • the communication terminal includes, for example, a mobile terminal such as a mobile phone, a smartphone and a tablet.
  • FIG. 22 is a functional block diagram showing the configuration of the gesture detection device 100 and the device that operates in connection with the gesture detection device 100 in the third embodiment.
  • the gesture detection device 100 and the communication device 130 are provided in the server 300.
  • the gesture detection device 100 acquires an image taken by the image pickup device 110 provided in the vehicle 1 via the communication device 140 and the communication device 130.
  • the gesture detection device 100 acquires the detection result of the hand detected based on the image and the information of the hand in the image in which the hand is detected.
  • the gesture detection device 100 determines whether or not the hand is a true hand based on predetermined conditions for the hand information.
  • the gesture detection device 100 rejects the detection result of the hand detected based on the image when the hand is not a true hand.
  • the device (vehicle-mounted device 120) mounted on the vehicle 1 is controlled based on a gesture by a hand that has not been rejected.
  • gesture detection device 100 may be provided in the server 300, and some of the other parts may be provided in the vehicle 1 in a distributed manner.
  • each embodiment can be freely combined, and each embodiment can be appropriately modified or omitted.
  • 1 vehicle 10 detection unit, 20 detection information acquisition unit, 30 judgment unit, 40 rejection unit, 50 video information acquisition unit, 100 gesture detection device, 101 gesture detection device, 110 imaging device, 120 in-vehicle device, 130 communication device, 140 Communication device, 300 servers.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Psychiatry (AREA)
  • General Health & Medical Sciences (AREA)
  • Social Psychology (AREA)
  • Health & Medical Sciences (AREA)
  • Quality & Reliability (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)
  • User Interface Of Digital Computer (AREA)
  • Navigation (AREA)
PCT/JP2020/019231 2020-05-14 2020-05-14 ジェスチャ検出装置およびジェスチャ検出方法 Ceased WO2021229741A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
PCT/JP2020/019231 WO2021229741A1 (ja) 2020-05-14 2020-05-14 ジェスチャ検出装置およびジェスチャ検出方法
US17/915,176 US20230123623A1 (en) 2020-05-14 2020-05-14 Gesture detecting apparatus and gesture detecting method
JP2022522427A JP7325625B2 (ja) 2020-05-14 2020-05-14 ジェスチャ検出装置およびジェスチャ検出方法

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2020/019231 WO2021229741A1 (ja) 2020-05-14 2020-05-14 ジェスチャ検出装置およびジェスチャ検出方法

Publications (1)

Publication Number Publication Date
WO2021229741A1 true WO2021229741A1 (ja) 2021-11-18

Family

ID=78525504

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/019231 Ceased WO2021229741A1 (ja) 2020-05-14 2020-05-14 ジェスチャ検出装置およびジェスチャ検出方法

Country Status (3)

Country Link
US (1) US20230123623A1 (https=)
JP (1) JP7325625B2 (https=)
WO (1) WO2021229741A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2024032322A (ja) * 2022-08-29 2024-03-12 フォルシアクラリオン・エレクトロニクス株式会社 駐車支援装置及び駐車支援方法

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20240107515A (ko) * 2022-12-30 2024-07-09 현대자동차주식회사 사용자 인터페이스 장치 및 방법, 이를 포함하는 차량

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2017191496A (ja) * 2016-04-14 2017-10-19 株式会社東海理化電機製作所 ジェスチャ判定装置
JP2019101826A (ja) * 2017-12-04 2019-06-24 アイシン精機株式会社 ジェスチャ判定装置およびプログラム

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6792986B2 (ja) * 2016-09-27 2020-12-02 株式会社日立製作所 生体認証装置

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2017191496A (ja) * 2016-04-14 2017-10-19 株式会社東海理化電機製作所 ジェスチャ判定装置
JP2019101826A (ja) * 2017-12-04 2019-06-24 アイシン精機株式会社 ジェスチャ判定装置およびプログラム

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2024032322A (ja) * 2022-08-29 2024-03-12 フォルシアクラリオン・エレクトロニクス株式会社 駐車支援装置及び駐車支援方法

Also Published As

Publication number Publication date
JPWO2021229741A1 (https=) 2021-11-18
US20230123623A1 (en) 2023-04-20
JP7325625B2 (ja) 2023-08-14

Similar Documents

Publication Publication Date Title
JP5261554B2 (ja) 指先ポインティング、ジェスチャに基づく車両用ヒューマンマシンインタフェース
US10894546B2 (en) Method and apparatus for in-vehicle impairment detection with driver verification
JP5466610B2 (ja) 視線推定装置
CN104583904B (zh) 用于输入对机动车部件的控制命令的方法
CN107949863A (zh) 使用生物体信息的认证装置以及认证方法
JP2009252105A (ja) プロンプター式操作装置
JP6342874B2 (ja) 画像認識装置
JPWO2019135281A1 (ja) 視線方向較正装置、視線方向較正方法および視線方向較正プログラム
JP7325625B2 (ja) ジェスチャ検出装置およびジェスチャ検出方法
KR101892390B1 (ko) 사용자 인터페이스, 이동 수단 및 사용자의 손을 인식하기 위한 방법
JP2018055264A (ja) ジェスチャ検出装置
JP6581482B2 (ja) 画像認識装置
JP5912177B2 (ja) 操作入力装置、操作入力方法及び操作入力プログラム
JP6494110B2 (ja) 車両設定装置
JP2019074965A (ja) 運転不能状態検出装置及び運転不能状態検出システム
JP2020013348A (ja) ジェスチャ検出装置、ジェスチャ検出方法、およびジェスチャ検出制御プログラム
JP2023047159A (ja) ジェスチャ検出装置、ドア開閉システム、及びジェスチャ検出方法
JP7289406B2 (ja) ジェスチャ検出装置およびジェスチャ検出方法
JP2017227966A (ja) ジェスチャ判定装置
WO2021005702A1 (ja) 顔検出処理装置および顔検出処理方法
WO2021240668A1 (ja) ジェスチャ検出装置およびジェスチャ検出方法
WO2021186710A1 (ja) ジェスチャ検出装置及びジェスチャ検出方法
JP7072737B2 (ja) ジェスチャ検出装置およびジェスチャ検出方法
JP2014153905A (ja) 入力装置
KR20250042892A (ko) 자율주행 차량의 주차 성능 평가 장치 및 방법

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20934979

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022522427

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20934979

Country of ref document: EP

Kind code of ref document: A1