WO2024144035A1 - Système d'affichage d'hologramme et procédé de commande associé - Google Patents

Système d'affichage d'hologramme et procédé de commande associé Download PDF

Info

Publication number
WO2024144035A1
WO2024144035A1 PCT/KR2023/020835 KR2023020835W WO2024144035A1 WO 2024144035 A1 WO2024144035 A1 WO 2024144035A1 KR 2023020835 W KR2023020835 W KR 2023020835W WO 2024144035 A1 WO2024144035 A1 WO 2024144035A1
Authority
WO
WIPO (PCT)
Prior art keywords
hand
image
holographic
user
control
Prior art date
Application number
PCT/KR2023/020835
Other languages
English (en)
Korean (ko)
Inventor
김희연
Original Assignee
에이아이다이콤(주)
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from KR1020230009940A external-priority patent/KR102559138B1/ko
Priority claimed from KR1020230179097A external-priority patent/KR102701038B1/ko
Application filed by 에이아이다이콤(주) filed Critical 에이아이다이콤(주)
Publication of WO2024144035A1 publication Critical patent/WO2024144035A1/fr

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
    • B60K35/10—Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03H—HOLOGRAPHIC PROCESSES OR APPARATUS
    • G03H1/00—Holographic processes or apparatus using light, infrared or ultraviolet waves for obtaining holograms or for obtaining an image from them; Details peculiar thereto
    • G03H1/04—Processes or apparatus for producing holograms
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03H—HOLOGRAPHIC PROCESSES OR APPARATUS
    • G03H1/00—Holographic processes or apparatus using light, infrared or ultraviolet waves for obtaining holograms or for obtaining an image from them; Details peculiar thereto
    • G03H1/26—Processes or apparatus specially adapted to produce multiple sub- holograms or to obtain images from them, e.g. multicolour technique
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R1/00—Details of transducers, loudspeakers or microphones
    • H04R1/02—Casings; Cabinets ; Supports therefor; Mountings therein

Definitions

  • the present invention relates to a holographic display system and its control method. More specifically, the present invention relates to a holographic display system and a control method that can control a holographic image displayed through a holographic display device in a non-contact manner by detecting or recognizing a user's hand movements or even a voice.
  • control devices such as touch screens, computer monitors, and mice are controlled by directly touching the 3D medical images implemented in such holographic systems, there is a risk of infectious diseases due to germs being spread through direct contact. This exists.
  • a holographic display is capable of controlling the holographic image in a non-contact manner by detecting or recognizing the user's hand movements to be controlled and furthermore, the voice.
  • a system and its control method are urgently needed.
  • the present invention aims to solve the problem of providing a holographic display system and a control method that can control a holographic image displayed through a holographic display device in a non-contact manner by detecting or recognizing the user's hand movements and further voice. .
  • the present invention includes a display device that projects and displays a holographic image;
  • a photographing device that acquires an image by photographing the user's hand movements; And recognize the user's hand movement from the image acquired from the imaging device using a pre-trained artificial intelligence algorithm, obtain three-dimensional spatial information about the hand movement from the recognized hand movement, and obtain three-dimensional spatial information about the hand movement from the obtained three-dimensional spatial information.
  • a holographic display system comprising a control device for generating a control signal of the display device and controlling a holographic image displayed on the display device according to the control signal.
  • the control device includes a 3D image acquisition unit that acquires a 3D image from the image transmitted from the photographing device; a hand motion recognition unit that recognizes the user's hand motion by acquiring hand shape information and hand motion information from the 3D spatial information of the 3D image; a hand positioning unit that specifies the 3D position coordinates of a hand making a gesture to control the holographic image to a virtual 3D panel on which the holographic image is displayed; and a control signal generator that generates a control signal using 3D coordinate information specified in the 3D panel.
  • the imaging device may include an RGB-D camera for acquiring RGB images and depth images of the user's hand movements.
  • the artificial intelligence algorithm in the hand motion recognition unit may be trained to acquire a plurality of feature points forming the skeleton of the hand from the 3D image and to obtain hand shape information based on the acquired plurality of feature points.
  • the three-dimensional position coordinates of the hand making the control operation gesture in the hand position designation unit are specified on a connection line connecting the first sub-position coordinates and the second sub-position coordinates specified on the three-dimensional panel. It can be.
  • the artificial intelligence algorithm recognizes the hand area 30h from the 3D image 30 and extracts the end points (B1 to B5) of each finger through the contour. Then, a finger vector is extracted based on the end point (B1 to B5) of each finger. Specifically, the artificial intelligence algorithm draws a first reference circle (K1) of the first radius (r1) centering on the finger tip points (B1 to B5), and creates a pair at the border of the finger that intersects the first reference circle (K1). The first boundary points (E1, E2) are extracted. Then, the first reference point (F1) is extracted from the point that bisects the line segment connecting the pair of first boundary points (E1, E2).
  • the voice signal analysis device 400 recognizes and analyzes the user’s voice, transmits a control command, and outputs the sound of the holographic image output through the speaker 100s. You can increase the volume.
  • the holographic display system can control holographic images by recognizing not only the user's hand movements but also the user's voice, thereby improving convenience of use.
  • the holographic image control method includes the step of acquiring a three-dimensional image 30 from an image captured by the photographing device 200 (S100); Recognizing hand movements by obtaining hand shape information and hand movement information of the user using the 3D image 30 (S200); Specifying the 3D position coordinates of the point where the 3D panel 40 and the hand of the user making a control operation gesture on the 3D panel 40 on which the holographic image is displayed through the display device 100 intersect.
  • Step (S300) and generating a control signal based on the position coordinates specified on the three-dimensional panel 40 and controlling the holographic image according to the control signal (S400).
  • the present invention detects a gesture in which the user's index finger moves and recognizes the position of the hand, and a holographic image (H) displayed on the display device, for example, a medical image such as the heart, is displayed on the finger. It can be controlled to move along the direction of .
  • the display device 100 may be provided with a speaker 100s near the frame unit 110 for outputting the sound of the holographic image H.
  • the present invention controls the sound volume of a holographic image by recognizing a user's specific hand movement or voice, and the sound signal of the controlled holographic image is output to the outside through the speaker (100s).

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • General Engineering & Computer Science (AREA)
  • Acoustics & Sound (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Mechanical Engineering (AREA)
  • Transportation (AREA)
  • Combustion & Propulsion (AREA)
  • Chemical & Material Sciences (AREA)
  • Signal Processing (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Computational Linguistics (AREA)
  • Social Psychology (AREA)
  • Psychiatry (AREA)
  • General Health & Medical Sciences (AREA)
  • Holo Graphy (AREA)

Abstract

La présente invention concerne un système d'affichage d'hologramme et un procédé de commande associé, le système d'affichage d'hologramme pouvant commander sans contact une image holographique affichée par l'intermédiaire d'un dispositif d'affichage holographique, et ce en détectant ou en reconnaissant des mouvements de la main de l'utilisateur, et même sa voix.
PCT/KR2023/020835 2022-12-28 2023-12-15 Système d'affichage d'hologramme et procédé de commande associé WO2024144035A1 (fr)

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
KR20220187558 2022-12-28
KR10-2022-0187558 2022-12-28
KR1020230009940A KR102559138B1 (ko) 2022-12-28 2023-01-26 차량용 비 접촉식 제어 시스템
KR10-2023-0009940 2023-01-26
KR1020230179097A KR102701038B1 (ko) 2023-12-11 2023-12-11 홀로그램 디스플레이 시스템 및 그 제어방법
KR10-2023-0179097 2023-12-11

Publications (1)

Publication Number Publication Date
WO2024144035A1 true WO2024144035A1 (fr) 2024-07-04

Family

ID=91718571

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2023/020835 WO2024144035A1 (fr) 2022-12-28 2023-12-15 Système d'affichage d'hologramme et procédé de commande associé

Country Status (1)

Country Link
WO (1) WO2024144035A1 (fr)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20150073857A (ko) * 2013-12-20 2015-07-01 임머숀 코퍼레이션 햅틱 피드백을 구비하는 차량 내 제스처 기반 입력 시스템
JP2016524262A (ja) * 2013-07-10 2016-08-12 リアル ビュー イメージング リミテッド ３次元ユーザインタフェース
KR20180043216A (ko) * 2018-04-09 2018-04-27 전자부품연구원 공간터치장치 및 이를 포함하는 디스플레이 장치
KR20210057358A (ko) * 2019-11-12 2021-05-21 주식회사 에스오에스랩 제스처 인식 방법 및 이를 수행하는 제스처 인식 장치
KR20220036146A (ko) * 2020-09-15 2022-03-22 광운대학교 산학협력단 손 제스처 인식에 기초한 인터랙션 홀로그램 디스플레이 방법 및 시스템
KR102559138B1 (ko) * 2022-12-28 2023-07-25 에이아이다이콤 (주) 차량용 비 접촉식 제어 시스템

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016524262A (ja) * 2013-07-10 2016-08-12 リアル ビュー イメージング リミテッド ３次元ユーザインタフェース
KR20150073857A (ko) * 2013-12-20 2015-07-01 임머숀 코퍼레이션 햅틱 피드백을 구비하는 차량 내 제스처 기반 입력 시스템
KR20180043216A (ko) * 2018-04-09 2018-04-27 전자부품연구원 공간터치장치 및 이를 포함하는 디스플레이 장치
KR20210057358A (ko) * 2019-11-12 2021-05-21 주식회사 에스오에스랩 제스처 인식 방법 및 이를 수행하는 제스처 인식 장치
KR20220036146A (ko) * 2020-09-15 2022-03-22 광운대학교 산학협력단 손 제스처 인식에 기초한 인터랙션 홀로그램 디스플레이 방법 및 시스템
KR102559138B1 (ko) * 2022-12-28 2023-07-25 에이아이다이콤 (주) 차량용 비 접촉식 제어 시스템

Similar Documents

Publication Publication Date Title
US20140049465A1 (en) Gesture operated control for medical information systems
WO2018048000A1 (fr) Dispositif et procédé d'interprétation d'imagerie tridimensionnelle basés sur caméra unique, et support lisible par ordinateur enregistré avec un programme pour une interprétation d'imagerie tridimensionnelle
US9798381B2 (en) Hands-free pointer system
US11678950B2 (en) Multiple-viewpoint video image viewing system and camera system
JP2021194539A (ja) 外科手術中のコンピュータ支援ナビゲーション用のカメラ追跡バー
JPH1031551A (ja) ヒューマンインターフェースシステムおよびこれを使用した高速移動物体位置検出装置
WO2019132244A1 (fr) Procédé de génération d'informations de simulation chirurgicale et programme
US20230000419A1 (en) Electrogram Annotation System
WO2022035110A1 (fr) Terminal utilisateur pour fournir une image médicale à réalité augmentée et procédé pour fournir une image médicale à réalité augmentée
CN114730454A (zh) 场景感知系统和方法
JP2020516355A (ja) ウェアラブルデバイスの配置のための案内を与えるための方法及び装置
WO2021177674A1 (fr) Procédé et système de prise en charge de commande d'objet en utilisant une caméra bidimensionnelle, et support d'enregistrement non transitoire lisible par ordinateur
WO2024144035A1 (fr) Système d'affichage d'hologramme et procédé de commande associé
WO2021162287A1 (fr) Procédé de vérification de la mise en correspondance d'une cible de chirurgie, appareil associé, et système le comprenant
WO2023013832A1 (fr) Système de commande de robot chirurgical utilisant une technologie de suivi de la main sans contact basée sur un casque
JP2002323956A (ja) マウス代替方法、マウス代替プログラム、および同プログラムを記録した記録媒体
Jain et al. Human computer interaction–Hand gesture recognition
WO2019022464A1 (fr) Appareil et procédé permettant de commander un système de visualisation chirurgical à l'aide de la détection de mouvement
KR102701038B1 (ko) 홀로그램 디스플레이 시스템 및 그 제어방법
De Paolis A touchless gestural platform for the interaction with the patients data
WO2023063671A1 (fr) Procédé et dispositif d'agencement d'images de balayage provenant d'un scanner 3d, et support d'enregistrement sur lequel sont enregistrées des commandes
WO2023200128A1 (fr) Procédé, appareil et support d'enregistrement stockant des commandes pour traiter une image balayée d'un scanner intra-buccal
WO2023014107A1 (fr) Procédé et dispositif de filtrage de bruit dans le traitement d'image à balayage de scanner tridimensionnel
WO2024019374A1 (fr) Procédé et dispositif de fourniture d'une image médicale en coupe transversale
WO2022250403A1 (fr) Dispositif électronique et son procédé de traitement d'image

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23912710

Country of ref document: EP

Kind code of ref document: A1