WO2023223403A1 - 固定状況判定システム、グローブ、情報処理装置、固定状況判定方法、および、固定状況判定プログラム - Google Patents

固定状況判定システム、グローブ、情報処理装置、固定状況判定方法、および、固定状況判定プログラム Download PDF

Info

Publication number
WO2023223403A1
WO2023223403A1 PCT/JP2022/020458 JP2022020458W WO2023223403A1 WO 2023223403 A1 WO2023223403 A1 WO 2023223403A1 JP 2022020458 W JP2022020458 W JP 2022020458W WO 2023223403 A1 WO2023223403 A1 WO 2023223403A1
Authority
WO
WIPO (PCT)
Prior art keywords
fingertip
glove
worker
target object
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2022/020458
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
孝太郎 福井
敬士 西川
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Mitsubishi Electric Corp
Mitsubishi Electric Building Solutions Corp
Original Assignee
Mitsubishi Electric Corp
Mitsubishi Electric Building Solutions Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mitsubishi Electric Corp, Mitsubishi Electric Building Solutions Corp filed Critical Mitsubishi Electric Corp
Priority to JP2023579706A priority Critical patent/JP7450837B1/ja
Priority to DE112022006819.1T priority patent/DE112022006819B4/de
Priority to CN202280095967.5A priority patent/CN119173833A/zh
Priority to PCT/JP2022/020458 priority patent/WO2023223403A1/ja
Publication of WO2023223403A1 publication Critical patent/WO2023223403A1/ja
Priority to US18/931,581 priority patent/US20250061603A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/014—Hand-worn input/output arrangements, e.g. data gloves
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • A—HUMAN NECESSITIES
    • A41—WEARING APPAREL
    • A41D—OUTERWEAR; PROTECTIVE GARMENTS; ACCESSORIES
    • A41D19/00—Gloves
    • A41D19/0024—Gloves with accessories
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/57—Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30204—Marker

Definitions

  • the present disclosure relates to a fixed situation determination system, a glove, an information processing device, a fixed situation determination method, and a fixed situation determination program.
  • Patent Document 1 discloses a finger for a robot hand in which a CCD camera is installed inside a fingertip portion covered with an elastic body to measure the deformation of the elastic body in order to detect contact with an object.
  • the present disclosure aims to more accurately determine the fixation status of an object by accurately acquiring the contact position of a fingertip on the object.
  • a fixation situation determination system is a fixation situation determination system including a glove worn by a worker to grip a target object, and the fixation situation determination system determines a fixation status of the object.
  • a photographing device disposed at a position corresponding to between a pair of fingers gripping the object in the glove, and photographing the object gripped by the pair of fingers as a target image;
  • a position and orientation estimation unit that estimates the position and orientation of the target object using the target image;
  • a range calculation unit that analyzes the movement of the fingertip of the worker using the target image and calculates a fingertip movement range that is a movement range of the position of the fingertip of each finger of the worker;
  • a contact position calculation unit that calculates a contact position of the fingertip on the target object based on the position and orientation of the target object and the fingertip movement range;
  • the determination unit includes a determining unit that determines a fixing state of the target object based on excitation force information that is information about an excitation force on the target object
  • the fixation status determination system aims to determine the fixation status of an object with higher accuracy by accurately acquiring the contact position of a fingertip on the target object.
  • FIG. 1 is a diagram showing an example of the overall configuration of a fixed situation determination system according to Embodiment 1.
  • FIG. 1 is a diagram illustrating a configuration example of an information processing apparatus according to Embodiment 1.
  • FIG. 1 is a diagram illustrating a configuration example of a glove according to Embodiment 1.
  • FIG. 3 is a diagram showing a state in which the glove according to Embodiment 1 grips an object.
  • 1 is a flowchart illustrating an example of the operation of the fixed situation determination system according to the first embodiment.
  • 3 is a diagram illustrating an example of a target image according to Embodiment 1.
  • FIG. FIG. 7 is a diagram showing an image of the angle of the proximal phalanx of each finger in the range calculation process according to the first embodiment.
  • FIG. 7 is a diagram showing an image of a fingertip movement range in the range calculation process according to the first embodiment.
  • FIG. 7 is a diagram showing an image of a contact position in the contact position calculation process according to the first embodiment.
  • FIG. 7 is a diagram showing a configuration example of a glove according to Modification 1 of Embodiment 1;
  • FIG. 7 is a diagram illustrating a configuration example of a glove according to a second modification of the first embodiment.
  • FIG. 7 is a diagram illustrating a configuration example of a glove according to Modification 3 of Embodiment 1;
  • FIG. 7 is a diagram illustrating a configuration example of a glove according to a fourth modification of the first embodiment.
  • 7 is a diagram illustrating a configuration example of an information processing device according to a fifth modification of the first embodiment.
  • FIG. 1 is a diagram showing an image of a fingertip movement range in the range calculation process according to the first embodiment.
  • FIG. 7 is a diagram showing an image of a contact position
  • FIG. 1 is a diagram showing an example of the overall configuration of a fixed situation determination system 500 according to the present embodiment.
  • the fixation status determination system 500 is a system that determines the fixation status of the target object 300 by the worker 10 wearing gloves, gripping the target object 300, and applying vibration to the target object 300.
  • the fixed situation determination system 500 includes a glove 200 that is worn by the worker 10 and grips the object 300, and an information processing device 100 that communicates with the glove 200. Communication between the glove 200 and the information processing device 100 may be prioritized or wireless. The detailed configuration of the information processing device 100 and the glove 200 will be described below.
  • FIG. 2 is a diagram illustrating a configuration example of information processing device 100 according to the present embodiment.
  • Information processing device 100 is a computer.
  • the information processing device 100 includes a processor 910 and other hardware such as a memory 921, an auxiliary storage device 922, an input interface 930, an output interface 940, and a communication device 950.
  • Processor 910 is connected to other hardware through wired or wireless communication and controls these other hardware.
  • the information processing device 100 includes a position and orientation estimation section 110, a range calculation section 120, a contact position calculation section 130, a determination section 140, and a storage section 160 as functional elements.
  • the storage unit 160 stores an object database 161.
  • the functions of the position and orientation estimation unit 110, range calculation unit 120, contact position calculation unit 130, and determination unit 140 are realized by software.
  • the storage unit 160 is included in the memory 921. Note that the storage unit 160 may be included in the auxiliary storage device 922 or may be provided in a distributed manner between the memory 921 and the auxiliary storage device 922.
  • the processor 910 is a device that executes a fixed situation determination program.
  • the fixed situation determination program is a program that realizes the functions of the position and orientation estimation section 110, the range calculation section 120, the contact position calculation section 130, and the determination section 140. Note that the fixed situation determination program also includes a program that implements the functions of the glove, which will be described later.
  • the processor 910 is an IC that performs arithmetic processing. Specific examples of the processor 910 are a CPU, a DSP, and a GPU.
  • the IC is an integrated circuit.
  • CPU is an abbreviation for Central Processing Unit.
  • DSP is an abbreviation for Digital Signal Processor.
  • GPU is an abbreviation for Graphics Processing Unit.
  • Memory 921 is a storage device that temporarily stores data.
  • a specific example of the memory 921 is SRAM or DRAM.
  • SRAM is an abbreviation for Static Random Access Memory.
  • DRAM is an abbreviation for Dynamic Random Access Memory.
  • Auxiliary storage device 922 is a storage device that stores data.
  • a specific example of the auxiliary storage device 922 is an HDD.
  • the auxiliary storage device 922 may be a portable storage medium such as an SD (registered trademark) memory card, CF, NAND flash, flexible disk, optical disk, compact disc, Blu-ray (registered trademark) disc, or DVD.
  • SD registered trademark
  • SD Secure Digital
  • CF is an abbreviation for CompactFlash®.
  • DVD is an abbreviation for Digital Versatile Disk.
  • the input interface 930 is a port connected to an input device such as a mouse, keyboard, or touch panel.
  • Input interface 930 is specifically a USB terminal. Note that the input interface 930 may be a port connected to a LAN.
  • USB is an abbreviation for Universal Serial Bus.
  • LAN is an abbreviation for Local Area Network.
  • the output interface 940 is a port to which a cable of an output device such as a display is connected.
  • the output interface 940 is a USB terminal or an HDMI (registered trademark) terminal.
  • the display is specifically an LCD.
  • Output interface 940 is also referred to as a display interface.
  • HDMI registered trademark
  • LCD is an abbreviation for Liquid Crystal Display.
  • the communication device 950 has a receiver and a transmitter.
  • the communication device 950 is connected to a communication network such as a LAN, the Internet, WiFi (registered trademark), or a telephone line.
  • Communication device 950 is specifically a communication chip or NIC.
  • NIC is an abbreviation for Network Interface Card.
  • the fixed situation determination program is executed in the information processing device 100.
  • the fixed situation determination program is loaded into processor 910 and executed by processor 910.
  • the memory 921 stores not only the fixed situation determination program but also the OS.
  • OS is an abbreviation for Operating System.
  • Processor 910 executes the fixed situation determination program while executing the OS.
  • the fixed situation determination program and the OS may be stored in the auxiliary storage device 922.
  • the fixed situation determination program and OS stored in auxiliary storage device 922 are loaded into memory 921 and executed by processor 910. Note that part or all of the fixed situation determination program may be incorporated into the OS.
  • the information processing device 100 may include a plurality of processors that replace the processor 910. These plurality of processors share the execution of the fixed situation determination program.
  • Each processor like processor 910, is a device that executes a fixed situation determination program.
  • Data, information, signal values, and variable values used, processed, or output by the fixed situation determination program are stored in memory 921, auxiliary storage device 922, or registers or cache memory within processor 910.
  • the “units” in the position/orientation estimation unit 110, range calculation unit 120, contact position calculation unit 130, and determination unit 140 are replaced with “circuit,” “process,” “procedure,” “process,” or “circuitry.” It's okay.
  • the fixed situation determination program causes the computer to execute position and orientation estimation processing, range calculation processing, contact position calculation processing, determination processing, and image setting processing.
  • the "processing" of position/orientation estimation processing, range calculation processing, contact position calculation processing, and determination processing is referred to as a "program,””programproduct,””computer-readable storage medium that stores a program," or "computer that stores a program.” It may also be read as "readable recording medium”.
  • the fixed situation determination method is a method performed by the information processing device 100 executing a fixed situation determination program.
  • the fixed situation determination program may be provided stored in a computer-readable recording medium. Further, the fixed situation determination program may be provided as a program product.
  • FIG. 3 is a diagram showing a configuration example of glove 200 according to the present embodiment.
  • FIG. 4 is a diagram showing a state in which the glove 200 according to the present embodiment grips the object 300.
  • the glove 200 is worn on the hand of the worker 10 who grips the object 300 when the worker 10 grips the object 300 and vibrates it.
  • FIG. 3 shows a glove 200 worn on the right hand of the worker 10.
  • the left diagram in FIG. 3 is a diagram of the glove 200 seen from the palm side.
  • the right diagram in FIG. 3 is a diagram of the glove 200 seen from the back of the hand.
  • FIG. 4 shows the worker 10 wearing the glove 200 and gripping the object 300.
  • the glove 200 includes an imaging device 201, a contact force sensor 202, a sensor data collection module 203, and a marker 204.
  • the photographing device 201 is disposed in the glove 200 at a position corresponding to between a pair of fingers that grip an object 300, and photographs the object 300 gripped by the pair of fingers as a target image 25.
  • the photographing device 201 is, for example, a device in which a module having an optical image sensor is arranged on a substrate.
  • the photographing device 201 is, for example, a CCD camera. Normally, people grasp objects using their thumb and fingers other than the thumb. In FIG. 3, the worker 10 is grasping an object using fingers including the thumb and index finger. Therefore, the photographing device 201 is placed in the glove 200 at a position corresponding to between the thumb and index finger on the palm side. Note that any other position may be used as long as the gripped object 300 is photographed. It suffices if it is placed at the base of either finger. Alternatively, it may be placed near the center of the palm.
  • the marker 204 is placed at a position corresponding to the inside of the proximal phalanx of each finger of the worker 10 in the glove 200. Specifically, the marker 204 is printed on the glove 200 at a position corresponding to the inside of the proximal phalanx of each finger of the worker 10 .
  • the photographing device 201 photographs an object image 25 including the position of the marker when the glove 200 grips the object. That is, the target image 25 also includes the marker 204 printed on each finger of the pair of fingers.
  • the contact force sensor 202 is arranged in the glove 200 at a position corresponding to the inside of the fingertip of the worker 10.
  • the contact force sensor 202 acquires information on the excitation force applied to the object 300 as excitation force information 26 .
  • the excitation force information 26 is, for example, the magnitude of the excitation force.
  • the sensor data collection module 203 collects various sensor data and transmits it to the information processing device 100. Specifically, the sensor data collection module 203 collects the target image 25 from the photographing device 201 and the excitation force information 26 from the contact force sensor 202 . Then, the sensor data collection module 203 transmits the target image 25 and the excitation force information 26 to the information processing device 100 via wired communication or wireless communication.
  • the sensor data collection module 203 is a computer that collects sensor data such as the target image 25 and the excitation force information 26 and transmits it to the information processing device 100.
  • the information processing device 100 includes hardware such as an input interface for inputting sensor data, a processor, and a communication device that communicates with the information processing device 100. The description of each hardware is the same as that for the information processing device 100.
  • the operation procedure of the fixed situation determination system 500 corresponds to a fixed situation determination method. Further, a program that realizes the operation of the fixed situation determination system 500 corresponds to a fixed situation determination program.
  • FIG. 5 is a flowchart illustrating an example of the operation of the fixed situation determination system 500 according to the present embodiment.
  • step S101 the photographing device 201 photographs the object 300 gripped by a pair of fingers as the target image 25. Specifically, the photographing device 201 photographs the object 300 held by the pair of fingers so as to include the marker 204 printed on each finger of the pair of fingers.
  • the sensor data collection module 203 transmits the target image 25 to the information processing device 100.
  • FIG. 6 is a diagram showing an example of the target image 25 captured by the photographing device 201 according to the present embodiment. As shown in FIG. 6, depending on the shape of the object 300, the contact portion between the object 300 and the fingertip may not be photographed. Although not shown, the target image 25 also includes markers 204 printed on each finger of the pair of fingers.
  • step S102 the position and orientation estimating unit 110 estimates the position and orientation of the target object 300 using the target image 25.
  • the position and orientation estimation unit 110 estimates the position and orientation of the target object 300 by comparing the target image 25 and the image candidates 61 stored in the target object database 161. Specifically, it is as follows.
  • the object database 161 stores images of each of a plurality of combinations of positions and orientations of the object 300 as image candidates 61.
  • the image candidates 61 are a plurality of three-dimensional three-dimensional data obtained by photographing the object 300 at various angles and at various combinations of positions.
  • the position and orientation estimation unit 110 estimates the position and orientation of the target object 300 in the target image 25 by comparing the target image 25 and the image candidate 61 and selecting the image candidate 61 with the highest degree of similarity.
  • step S103 the range calculation unit 120 analyzes the movement of the fingertip of the worker 10 using the target image 25, and calculates a fingertip movement range R that is a movement range of the position of the fingertip of each finger of the worker.
  • Step S103 includes step S31 and step S32. Specifically, it is as follows.
  • FIG. 7 is a diagram showing an image of the angle of the proximal phalanx of each finger in the range calculation process according to the present embodiment.
  • the range calculation unit 120 uses the position of the marker 204 on the proximal phalanx in the target image 25 to calculate the angle of the proximal phalanx of each finger of the worker 10.
  • the example in FIG. 7 shows how the angle ⁇ 1 of the proximal phalanx of the thumb and the angle ⁇ 2 of the proximal phalanx of the index finger are calculated.
  • FIG. 8 is a diagram showing an image of the fingertip movement range R in the range calculation process according to the present embodiment.
  • the range calculation unit 120 calculates the fingertip movement range R based on the angle of the proximal phalanx of each finger of the worker 10.
  • the range calculation unit 120 calculates the fingertip movement range R of each finger based on the angle of the proximal phalanx of each finger using information about the joint structure of a normal finger.
  • step S104 the contact position calculation unit 130 calculates the contact position P of the fingertip on the target object 300 based on the position and orientation of the target object 300 and the fingertip movement range R.
  • FIG. 9 is a diagram showing an image of the contact position P in the contact position calculation process according to the present embodiment.
  • the contact position calculation unit 130 calculates the surface position of the target object 300 from the position and orientation of the target object 300. Then, the contact position calculation unit 130 calculates the contact position P between the object 300 and the fingertip from the intersection between the surface position of the object 300 and the fingertip movement range R.
  • step S105 the determination unit 140 determines the fixation status of the target object 300 based on the excitation force information 26, which is information about the excitation force on the target object 300 at the contact position P of the fingertip, and the displacement of the target object 300. judge.
  • Step S105 includes step S51 and step S52. Specifically, it is as follows.
  • step S51 the worker 10 wears the glove 200, grips the object 300, and excites the object 300.
  • the sensor data collection module 203 transmits the magnitude of the excitation force acquired by the contact force sensor to the information processing device 100 as excitation force information 26.
  • the excitation force information 26 is acquired by the contact force sensor 202 and transmitted from the glove 200 to the information processing device 100 via the sensor data collection module 203.
  • step S51 the determination unit 140 determines the fixation status of the target object 300 based on the contact position P, the excitation force information 26, and the displacement of the target object 300.
  • the displacement of the target object 300 is acquired from the target image 25 by the determination unit 140.
  • the determination unit 140 calculates the magnitude of the excitation force and the direction of the excitation force at the contact position P, obtains the displacement that is the response of the object 300, and integrates these calculation results to determine the direction of the excitation force of the object 300. Determine the fixed status or fixed state. As described above, from the relationship between the displacement of the object 300 and the excitation force, it is possible to evaluate the fixation status using the relationship as an index.
  • FIG. 10 is a diagram showing a configuration example of a glove 200 according to Modification 1 of the present embodiment.
  • the glove 200 may include a motion sensor on the outside of each fingertip of the worker 10 to detect movement of the fingertip.
  • the range calculation unit 120 calculates a fingertip movement range R based on the angle of the proximal phalanx of each finger of the worker 10 and the movement of the fingertip detected by the motion sensor 205.
  • the motion sensor 205 is, for example, an acceleration sensor that can measure acceleration.
  • the range calculation unit 120 estimates the fingertip position from the amount of displacement from the initial position. The number of motion sensors to be used may be changed based on variations in the assumed gripping method.
  • the accuracy of estimating the position of the fingertip is improved, and the excitation position can be estimated accurately and stably.
  • FIG. 11 is a diagram showing a configuration example of a glove 200 according to a second modification of the present embodiment.
  • the glove 200 may include a fixture 206 that is provided on each of the pair of fingers in the glove 200 and that fixes the imaging device 201.
  • the fixture 206 includes, for example, a ring 261 provided on each of a pair of fingers in the glove 200, and a rigid body 262 that connects the ring 261 and the photographing device 201.
  • the ring 261 fixed to the finger and the photographing device 201 are connected by the rigid body 262
  • the accuracy of estimating the position of the finger and the photographing device 201 is improved, and the accuracy of estimating the fingertip position is also improved. do.
  • FIG. 12 is a diagram showing a configuration example of a glove 200 according to modification 3 of the present embodiment.
  • the marker 204 may have a protrusion shape.
  • markers corresponding to the proximal phalanx three-dimensional objects having different shapes for each finger may be formed or fixed on the surface of the glove.
  • a protrusion with a distinctive shape as a position detection marker placed on the palm side of the proximal phalanx, the angular range of the proximal phalanx in which the marker can be detected by the camera is expanded, and the fingertip position can be determined more stably. Estimates can be made.
  • FIG. 13 is a diagram showing a configuration example of a glove 200 according to a fourth modification of the present embodiment.
  • a plurality of photographing devices 201 may be provided. That is, a plurality of cameras may be installed on the glove.
  • two or more camera modules as optical imaging devices on the glove, the angular range in which markers can be detected by the camera is expanded, and fingertip position estimation can be performed more stably.
  • the functions of position and orientation estimation section 110, range calculation section 120, contact position calculation section 130, and determination section 140 are realized by software.
  • the functions of the position and orientation estimation section 110, the range calculation section 120, the contact position calculation section 130, and the determination section 140 may be realized by hardware.
  • the fixed situation determination system 500 includes an electronic circuit 909 instead of the processor 910.
  • FIG. 14 is a diagram illustrating a configuration example of an information processing apparatus 100 according to a fifth modification of the present embodiment.
  • the electronic circuit 909 is a dedicated electronic circuit that realizes the functions of the position and orientation estimation section 110, the range calculation section 120, the contact position calculation section 130, and the determination section 140.
  • Electronic circuit 909 is specifically a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, a logic IC, a GA, an ASIC, or an FPGA.
  • GA is an abbreviation for Gate Array.
  • ASIC is an abbreviation for Application Specific Integrated Circuit.
  • FPGA is an abbreviation for Field-Programmable Gate Array.
  • the functions of the position and orientation estimation section 110, the range calculation section 120, the contact position calculation section 130, and the determination section 140 may be realized by one electronic circuit, or may be realized by being distributed among multiple electronic circuits.
  • part of the functions of the position and orientation estimation section 110, range calculation section 120, contact position calculation section 130, and determination section 140 may be realized by electronic circuits, and the remaining functions may be realized by software. Furthermore, some or all of the functions of the position and orientation estimating section 110, the range calculating section 120, the contact position calculating section 130, and the determining section 140 may be realized by firmware.
  • Each processor and electronic circuit is also referred to as processing circuitry. That is, the functions of the position and orientation estimation section 110, the range calculation section 120, the contact position calculation section 130, and the determination section 140 are realized by processing circuitry. Note that the computer mounted on the glove 200 is also similar to the fifth modification.
  • a local worker wears gloves equipped with a contact force sensor, grips and vibrates an object, and uses the sensor information at that time to send a message to a remote inspector.
  • Present haptic information For example, in a hand-eye system that uses a camera installed in the robot hand to detect the position of an object when it is grasped by a robot hand, the fingertips that are grasping are in the blind spot of the object. I can't figure it out. For this reason, it is not possible to grasp the application position and direction of the excitation force applied from the finger to the object, which may reduce the determination accuracy.
  • the fixation status of the object can be determined with high precision.
  • a different marker is printed for each finger on the palm side corresponding to the bone at the base of the finger (proximal phalanx), and a small camera is installed in the space between the thumb and index finger.
  • proximal phalanx a small camera is installed in the space between the thumb and index finger.
  • the following processing is performed. (1) The position of the object is calculated by a camera using the shape data of the object to be inspected. (2) Calculate the angle of the proximal phalanx of each finger using the marker shown on the camera. (3) Calculate the range of candidate fingertip positions based on the angle information of the proximal phalanx. (4) Calculate the contact position from the relationship between the position/shape information of the object and the candidate fingertip position.
  • each part of the fixed situation determination system has been described as an independent functional block.
  • the configuration of the fixed situation determination system does not have to be the configuration of the embodiment described above.
  • the functional blocks of the fixed situation determination system may have any configuration as long as they can realize the functions described in the embodiments described above.
  • the fixed situation determination system may be a system composed of a plurality of devices instead of one device.
  • a plurality of parts of Embodiment 1 may be combined and implemented.
  • one part of this embodiment may be implemented.
  • this embodiment may be implemented in any combination, either in whole or in part. That is, in Embodiment 1, it is possible to freely combine each embodiment, to modify any component of each embodiment, or to omit any component in each embodiment.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Engineering & Computer Science (AREA)
  • Textile Engineering (AREA)
  • Signal Processing (AREA)
  • Human Computer Interaction (AREA)
  • User Interface Of Digital Computer (AREA)
PCT/JP2022/020458 2022-05-17 2022-05-17 固定状況判定システム、グローブ、情報処理装置、固定状況判定方法、および、固定状況判定プログラム Ceased WO2023223403A1 (ja)

Priority Applications (5)

Application Number Priority Date Filing Date Title
JP2023579706A JP7450837B1 (ja) 2022-05-17 2022-05-17 固定状況判定システム、グローブ、情報処理装置、固定状況判定方法、および、固定状況判定プログラム
DE112022006819.1T DE112022006819B4 (de) 2022-05-17 2022-05-17 Fixierzustand-bestimmungssystem, handschuh, informationsverarbeitungseinrichtung, fixierzustand-bestimmungsverfahren und fixierzustand-bestimmungsprogramm
CN202280095967.5A CN119173833A (zh) 2022-05-17 2022-05-17 固定状况判定系统、手套、信息处理装置、固定状况判定方法及固定状况判定程序
PCT/JP2022/020458 WO2023223403A1 (ja) 2022-05-17 2022-05-17 固定状況判定システム、グローブ、情報処理装置、固定状況判定方法、および、固定状況判定プログラム
US18/931,581 US20250061603A1 (en) 2022-05-17 2024-10-30 Fixing-state determination system and glove

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2022/020458 WO2023223403A1 (ja) 2022-05-17 2022-05-17 固定状況判定システム、グローブ、情報処理装置、固定状況判定方法、および、固定状況判定プログラム

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/931,581 Continuation US20250061603A1 (en) 2022-05-17 2024-10-30 Fixing-state determination system and glove

Publications (1)

Publication Number Publication Date
WO2023223403A1 true WO2023223403A1 (ja) 2023-11-23

Family

ID=88834822

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/020458 Ceased WO2023223403A1 (ja) 2022-05-17 2022-05-17 固定状況判定システム、グローブ、情報処理装置、固定状況判定方法、および、固定状況判定プログラム

Country Status (5)

Country Link
US (1) US20250061603A1 (https=)
JP (1) JP7450837B1 (https=)
CN (1) CN119173833A (https=)
DE (1) DE112022006819B4 (https=)
WO (1) WO2023223403A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2026035418A1 (en) * 2024-08-09 2026-02-12 Tacta Systems Inc. Sensing gloves for performing tasks

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102022132061A1 (de) * 2022-12-02 2024-06-13 Workaround Gmbh Handschuh sowie tragbare Vorrichtung
DE102023118306A1 (de) 2023-07-11 2025-01-16 Workaround Gmbh Elektrische Schaltungsbaugruppe für einen Handschuh

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010221359A (ja) * 2009-03-24 2010-10-07 Toyota Industries Corp ロボットハンド用フィンガ
US20160174897A1 (en) * 2014-12-22 2016-06-23 Caterpillar Inc. Sensor Embedded Wearable Technology Glove
JP2019128940A (ja) * 2018-01-24 2019-08-01 シー．アール．エフ． ソシエタ コンソルティレ ペル アツィオニ 特に作業者の人間工学的解析のためのシステム及び方法
US20200117271A1 (en) * 2018-10-12 2020-04-16 Aurora Flight Sciences Corporation Manufacturing Monitoring System

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010221359A (ja) * 2009-03-24 2010-10-07 Toyota Industries Corp ロボットハンド用フィンガ
US20160174897A1 (en) * 2014-12-22 2016-06-23 Caterpillar Inc. Sensor Embedded Wearable Technology Glove
JP2019128940A (ja) * 2018-01-24 2019-08-01 シー．アール．エフ． ソシエタ コンソルティレ ペル アツィオニ 特に作業者の人間工学的解析のためのシステム及び方法
US20200117271A1 (en) * 2018-10-12 2020-04-16 Aurora Flight Sciences Corporation Manufacturing Monitoring System

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2026035418A1 (en) * 2024-08-09 2026-02-12 Tacta Systems Inc. Sensing gloves for performing tasks
US12602110B2 (en) 2024-08-09 2026-04-14 Tacta Systems Inc. Sensing gloves for performing tasks

Also Published As

Publication number Publication date
DE112022006819T5 (de) 2025-01-09
JP7450837B1 (ja) 2024-03-15
JPWO2023223403A1 (https=) 2023-11-23
DE112022006819B4 (de) 2026-05-07
CN119173833A (zh) 2024-12-20
US20250061603A1 (en) 2025-02-20

Similar Documents

Publication Publication Date Title
JP7450837B1 (ja) 固定状況判定システム、グローブ、情報処理装置、固定状況判定方法、および、固定状況判定プログラム
US6744420B2 (en) Operation input apparatus using sensor attachable to operator's hand
US11481029B2 (en) Method for tracking hand pose and electronic device thereof
US11175729B2 (en) Orientation determination based on both images and inertial measurement units
JP2004157850A (ja) 運動検出装置
US10776657B2 (en) Template creation apparatus, object recognition processing apparatus, template creation method, and program
US10895628B2 (en) Tracking system, tracking device and tracking method
US20210278898A1 (en) Ring device having an antenna, a touch pad, and/or a charging pad to control a computing device based on user motions
US20130158946A1 (en) Modelling of hand and arm position and orientation
US20210089162A1 (en) Calibration of inertial measurement units in alignment with a skeleton model to control a computer system based on determination of orientation of an inertial measurement unit from an image of a portion of a user
Choi et al. Development of a low-cost wearable sensing glove with multiple inertial sensors and a light and fast orientation estimation algorithm
GB2597378A (en) Fatigue determination device, fatigue determination method, and fatigue determination program
US20210109606A1 (en) Automatic Switching between Different Modes of Tracking User Motions to Control Computer Applications
CN116909384A (zh) 头戴式显示器及点击输入信号产生方法
JP7254262B2 (ja) 作業推定装置、作業推定方法、及び、作業推定プログラム
KR102334543B1 (ko) 가상현실 기반 주행형 농업기계의 안전교육을 위한 손 동작 추적 시스템 및 방법
Fang et al. A novel data glove for fingers motion capture using inertial and magnetic measurement units
CN118092636A (zh) 头戴式显示器、控制方法及其非暂态电脑可读取存储介质
US20240003766A1 (en) Contact information obtaining apparatus, contact information obtaining system, contact information obtaining method, and computer readable medium
US12073020B2 (en) Head-mounted display, unlocking method and non-transitory computer readable storage medium thereof
CN113467599B (zh) 手指与手掌间屈伸自由度解算方法、装置及数据手套
CN113268136B (zh) 拇指与手掌间自由度解算方法、装置及数据手套
CN120579052A (zh) 实时多人拳击分类方法、装置、设备及存储介质
WO2025197687A1 (ja) 遠隔制御システム、遠隔制御装置、遠隔制御方法、およびプログラム
CN117621063A (zh) 外力确定方法、装置、机器人及计算机可读存储介质

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2023579706

Country of ref document: JP

Kind code of ref document: A

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22942608

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 202447080092

Country of ref document: IN

WWE Wipo information: entry into national phase

Ref document number: 112022006819

Country of ref document: DE

WWE Wipo information: entry into national phase

Ref document number: 2401007448

Country of ref document: TH

122 Ep: pct application non-entry in european phase

Ref document number: 22942608

Country of ref document: EP

Kind code of ref document: A1