WO2020122375A1 - Système d'apprentissage basé sur la réalité virtuelle (rv) pour un enfant ayant un trouble du développement - Google Patents

Système d'apprentissage basé sur la réalité virtuelle (rv) pour un enfant ayant un trouble du développement Download PDF

Info

Publication number
WO2020122375A1
WO2020122375A1 PCT/KR2019/012174 KR2019012174W WO2020122375A1 WO 2020122375 A1 WO2020122375 A1 WO 2020122375A1 KR 2019012174 W KR2019012174 W KR 2019012174W WO 2020122375 A1 WO2020122375 A1 WO 2020122375A1
Authority
WO
WIPO (PCT)
Prior art keywords
unit
hand
user
image
radio wave
Prior art date
Application number
PCT/KR2019/012174
Other languages
English (en)
Korean (ko)
Inventor
최요철
이향균
송창환
김지수
Original Assignee
주식회사 홀로웍스
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 주식회사 홀로웍스 filed Critical 주식회사 홀로웍스
Publication of WO2020122375A1 publication Critical patent/WO2020122375A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
    • G06Q50/10—Services
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G02B27/0176—Head mounted characterised by mechanical features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
    • G06Q50/10—Services
    • G06Q50/22—Social work
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/003—Navigation within 3D models or images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/107—Static hand or arm

Definitions

  • the present invention relates to a VR-based training system, and more specifically, provides a virtual space based on VR for learning training or vocational training of a child with developmental disabilities, and after acquiring an image of the hand of a child with developmental disabilities, the VR-based virtualization It is a technical field related to a training system that coordinates overlapping into the space of a child, so that children with developmental disabilities can be taught, presented and evaluated.
  • VR Virtual Reality
  • VR technology is active in various fields, among which, in the healthcare field, it has been developed through medical objectives such as remote consultation of doctors and surgical guides and virtual reality technology.
  • VR mental therapy is emerging as a new mechanism in the treatment of mental disorders.
  • VR is effective as a psychotherapeutic tool and is particularly effective when using exposure therapy.
  • VR technology can provide experiences that immersiveness and realism rely on counseling between the therapist and patient or recreate a problem environment in VR, providing experiences that are full of VR, and the most actively used cognitive behavioral therapy in recent years.
  • Exposure Therapy is a patient's imaginative approach, and when applying VR technology, it can increase the therapeutic effect to patients who have difficulty in imagining their patient.
  • VR technology is effective in conducting clinical and experimental research that requires precise control because the amount of exposure and stimulation can be controlled.
  • Patent Document 1 Everybody's daily life education method using a virtual reality based on user environment and a device therefor (Publication No. 10-2018- 0028033, hereinafter referred to as Patent Document 1).
  • Patent Document 2 there is also a “virtual reality-based user-customized training system (Publication No. 10-2017-0079135, hereinafter referred to as Patent Document 2)".
  • Patent Document 2 relates to a computer-based virtual reality training system, and a problem to be solved is to provide a virtual reality-based user-customized training system.
  • the present invention includes a user layer that allows a user to perform computer-based training using an input device and an output device; A client layer that provides a training module to train the user by applying virtual reality technology and user-customized technology to the computer-based training; And a server layer that collects the user's behavior from the training module and reconfigures or generates the training module to match the user's level and behavior pattern.
  • Patent Document 3 there is also "intercommunication between the head-mounted display and the real-world object (Publication No. 10-2017-0116121, hereinafter referred to as Patent Document 3)".
  • Patent Document 4 there is a “cognitive and behavioral rehabilitation system using motion tracking technology and augmented reality technique (Publication No. 10-2008-0005798, hereinafter referred to as Patent Document 4)".
  • interactive rehabilitation support system is introduced that utilizes interactive HCI technology such as gesture recognition or motion tracking to rehabilitate patients with cognitive and motor disorders, and also introduces game techniques through video.
  • the present invention extracts the color of interest and applies the CAMSHIFT algorithm to solve the inconvenience of manipulation of the keyboard or mouse and implement a cognitive and behavioral disorder rehabilitation system that is not boring, and extracts the color of interest by converting it to the HSV color space In order to reduce other noise, only the color of interest was used.
  • a robust CAMSHIFT algorithm to light changes or overlapping of other objects, the center of the object to be tracked in the image received from the image camera was found, and the width and height of the object and the degree of rotation were found.
  • the system of the present invention includes a Visual Reaction, Visual Differential Reaction, Visual Reaction and Move cognitive disorder rehabilitation support system that measures attention and reaction time, and a Visual Track and measures attention and hand movement control. It is composed of a behavioral rehabilitation support system that performs a Visual Spatial and Motor function that measures target and attention and visual perception motor skills.
  • the VR-based training system for children with developmental disabilities has been devised to solve the conventional problems as described above, and presents the following problems to be solved.
  • the image of the user's hand is reflected in the virtual space provided to the user, that is, the child with developmental disabilities, so that the training effect can be maximized.
  • the user can maximize the effectiveness of learning and training by providing various guides for the targeted mission.
  • the VR-based training system for children with developmental disabilities has the following task solving means for the above-mentioned tasks.
  • a VR-based training system for a child with developmental disabilities In the case of a VR-based training system for a child with developmental disabilities according to the present invention, it is mounted on a user's face and displays virtual space information with both eyes of the user (virtual reality unit); A hand detecting unit (hand) provided on the VR unit to detect a 3D spatial shape of the user's hand (the "3D spatial shape of the user's hand is referred to as a "hand shape”.
  • Mounting unit is mounted on the bottom of the VR unit, and the upper part is equipped with the hand detecting unit, so that the hand detecting unit is capable of adjusting an angle in a direction to detect the shape of the user's hand ( mounting unit); And providing the virtual space information to the VR unit, receiving the hand shape from the hand detecting unit, overlapping on the virtual space information, and presenting a mission to the user through the VR unit.
  • it may be characterized in that it comprises a training unit (training unit) for evaluating the training degree of the user according to the hand-shaped operation information.
  • the mounting unit of the VR-based training system for a child with developmental disabilities comprises: a holding unit for holding the hand detecting unit and adjusting the front and rear angles; A first bar having one end hinged to the holding portion to allow angle adjustment of the holding portion; One end is hinged to the other end of the first bar, the second bar to make the angle adjustment of the first bar (bar); And by connecting the other end of the second bar hinge to one side, so that the angle adjustment of the second bar is made, it may be characterized in that it comprises a docking portion fixedly attached to the upper surface of the VR unit.
  • the hand detecting unit of the VR-based training system for children with developmental disabilities includes: a radio wave transmitting unit for transmitting radio waves having a predetermined wavelength band toward the user's hand; A radio wave receiving unit which is arranged to be spaced apart from the radio wave transmitting unit and is transmitted by the radio wave transmitting unit to receive the reflected wave reflected from the hand of the user; And a structure recognition unit receiving the reflected wave from the radio wave receiving unit, recognizing the distance of the reflected wave, and recognizing the shape of the hand of the user.
  • the radio wave transmitting part of the VR-based training system for a child with developmental disabilities sets the predetermined wavelength band of the radio wave to a range of 780 nm to 1 mm, and the range of photon energy of the radio wave ranges from 1.24 meV to 1.7 eV. It is formed of electromagnetic waves, and the radio wave receiver may include at least two or more radio wave recognition modules, so that the radio wave transmitter transmits the reflected wave reflected from the hand of the user through the radio wave recognition module. .
  • the hand detecting unit of the VR-based training system for a child with developmental disabilities further comprises an image acquisition unit facing the user's hand and acquiring a visible light image reflected from the user's hand. Can be done with
  • the hand detecting unit of the VR-based training system for a child with developmental disabilities acquires the hand shape from the structure recognition unit, and the visible light image reflected from the user's hand from the image acquisition unit By mapping the hand shape, only the portion of the visible light image that is mapped to the hand image (the "part of the visible light image that is mapped to the hand image" is referred to as a "three-dimensional hand image”.
  • a hand mapping unit acquired with; And a movement transmitter for transmitting motion information of the 3D hand image from the hand mapping unit to the training unit.
  • the training unit of the VR-based training system for children with developmental disabilities includes: a background transmitting unit that transmits the virtual space information to the VR unit; A movement receiving unit receiving the 3D hand image from the movement transmitting unit; And an overlapping portion overlapping the 3D hand image with the virtual space information transmitted by the background transmitting portion.
  • the training unit of the VR-based training system for children with developmental disabilities includes: a mission presenting unit that presents a preset action to be performed by the user on the virtual space information; And an evaluation measurement for receiving the motion information of the 3D hand image according to the preset action, and presenting the motion information of the 3D hand image to a preset evaluation criteria table to evaluate the motion information of the 3D hand image. It may be characterized in that it further comprises a wealth.
  • the training unit of the VR-based training system for children with developmental disabilities further comprises a guide presenting unit presenting a predetermined guide to the user so as to perform the preset action presented by the mission presenting unit can do.
  • the guide presenting unit of the VR-based training system for a child with developmental disabilities presents a virtual path in which the user's 3D hand image moves for the preset action, and the virtual path is the 3D A route presentation unit to be formed from the hand shape to the target region; An outline presenting unit displaying an outline of the target region to be reached by the 3D hand shape and presenting it to the user; And it may be characterized in that it comprises a target presentation unit to inform the user by providing a predetermined mark in the target area to be reached by the three-dimensional hand shape.
  • the VR-based training system for children with developmental disabilities having the above configuration provides the following effects.
  • VR Virtual Reality goggles are provided to users with developmental disabilities, and through VR goggles, commercial facilities, which are difficult for children with developmental disabilities to receive realistic training, provide realism, such as field trips, to provide education and vocational training. It provides a sense of realism.
  • the movement of the user's hand is acquired in 3D, so that the 3D image of the user's hand is overlapped with the VR image in real time. To make this happen.
  • the mission is provided to children with developmental disabilities at the same time as voice and text, but after receiving the voice of the administrator, the voice frequency band of the entertainer who likes the voice. To modulate.
  • FIG. 1 is a conceptual diagram of a VR-based training system for children with developmental disabilities according to an embodiment of the present invention.
  • FIG. 2 is a block diagram illustrating a hand detecting unit of a VR-based training system for a child with developmental disabilities according to an embodiment of the present invention.
  • FIG. 3 illustrates that the hand detecting unit of the VR-based training system for a child with developmental disabilities according to an embodiment of the present invention recognizes the hand shape (a) and visible light image (b) from the hand.
  • the hand mapping unit of the hand detecting unit of the VR-based training system for a child with developmental disabilities acquires a 3D hand image.
  • Figure 5 is a perspective view showing the operational relationship of each component of the mounting unit of the VR-based training system for children with developmental disabilities according to an embodiment of the present invention.
  • FIG. 6 is a block diagram showing each configuration of a training unit of a VR-based training system for a child with developmental disabilities according to an embodiment of the present invention.
  • FIG. 7 is a block diagram showing each configuration of the guide presentation unit of the training unit of the VR-based training system for children with developmental disabilities according to an embodiment of the present invention.
  • FIG. 8 shows that a VR unit of a VR-based training system for a child with developmental disabilities according to an embodiment of the present invention presents virtual space information and a mission to a user, and shows that a 3D hand image overlaps with virtual space information. It is a VR screen.
  • FIG. 9 is a VR screen showing that the movement of the hand works in conjunction with the virtual space information when the user presented with the mission in FIG. 8 moves to the screen of the kitchen and then moves the hand to lower the coffee.
  • FIG. 10 is a VR screen illustrating that the outline presentation unit of the guide presentation unit of FIG. 7 presents an outline at a target position.
  • FIG. 11 is a VR screen showing that the target presentation unit of the guide presentation unit of FIG. 7 shows an identification mark at the target position.
  • FIG. 12 is a VR screen showing that the route presentation unit of the guide presentation unit of FIG. 7 presents a route to a target location.
  • FIG. 13 is an evaluation measurement unit of a VR-based training system for a child with developmental disabilities according to an embodiment of the present invention, showing a good contact point of a target area.
  • FIG. 14 is a view showing a good contact point and a bad contact point of a target area after the evaluation measurement unit of the VR-based training system for a child with developmental disabilities increases the evaluation level compared to the target area of FIG. 13 according to an embodiment of the present invention.
  • FIG. 15 is a view showing contact points of a 3D hand image, which should contact the good contact points of FIGS. 13 and 14.
  • FIG. 16 is a conceptual diagram showing an operational relationship between a voice recognition unit, a text extraction unit, and a frequency modulation unit of a mission presentation unit of a VR-based training system for a child with developmental disabilities according to an embodiment of the present invention.
  • the VR-based training system for children with developmental disabilities can be modified in various ways and can have various embodiments, and specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all modifications, equivalents, and substitutes included in the technical spirit and technical scope of the present invention.
  • FIG. 1 is a conceptual diagram of a VR-based training system for children with developmental disabilities according to an embodiment of the present invention.
  • 2 is a block diagram illustrating a hand detecting unit of a VR-based training system for a child with developmental disabilities according to an embodiment of the present invention.
  • FIG. 3 illustrates that the hand detecting unit of the VR-based training system for a child with developmental disabilities according to an embodiment of the present invention recognizes the hand shape (a) and visible light image (b) from the hand.
  • 4 illustrates that the hand mapping unit of the hand detecting unit of the VR-based training system for a child with developmental disabilities according to an embodiment of the present invention acquires a 3D hand image.
  • FIG. 5 is a perspective view showing the operational relationship of each component of the mounting unit of the VR-based training system for children with developmental disabilities according to an embodiment of the present invention.
  • 6 is a block diagram showing each configuration of a training unit of a VR-based training system for a child with developmental disabilities according to an embodiment of the present invention.
  • 7 is a block diagram showing each configuration of the guide presentation unit of the training unit of the VR-based training system for children with developmental disabilities according to an embodiment of the present invention.
  • FIG. 8 shows that a VR unit of a VR-based training system for a child with developmental disabilities according to an embodiment of the present invention presents virtual space information and a mission to a user, and shows that a 3D hand image overlaps with virtual space information. It is a VR screen.
  • FIG. 9 is a VR screen showing that the movement of the hand works in conjunction with the virtual space information when the user presented with the mission in FIG. 8 moves to the screen of the kitchen and then moves the hand to lower the coffee.
  • FIG. 10 is a VR screen illustrating that the outline presentation unit of the guide presentation unit of FIG. 7 presents an outline at a target position.
  • FIG. 11 is a VR screen showing that the target presentation unit of the guide presentation unit of FIG. 7 shows an identification mark at the target position.
  • FIG. 12 is a VR screen showing that the route presentation unit of the guide presentation unit of FIG. 7 presents a route to a target location.
  • 13 is an evaluation measurement unit of a VR-based training system for a child with developmental disabilities according to an embodiment of the present invention, showing a good contact point of a target area.
  • FIG. 10 is a VR screen illustrating that the outline presentation unit of the guide presentation unit of FIG. 7 presents an outline at a target position.
  • FIG. 11 is a VR screen showing that the target presentation unit of the guide presentation
  • FIG. 14 is a view showing a good contact point and a bad contact point of a target area after the evaluation measurement unit of the VR-based training system for a child with developmental disabilities increases the evaluation level compared to the target area of FIG. 13 according to an embodiment of the present invention.
  • 15 is a view showing contact points of a 3D hand image, which should contact the good contact points of FIGS. 13 and 14.
  • FIG. 16 is a conceptual diagram showing an operational relationship between a voice recognition unit, a text extraction unit, and a frequency modulation unit of a mission presentation unit of a VR-based training system for a child with developmental disabilities according to an embodiment of the present invention.
  • a child with a developmental disorder that is, a user is provided with a specific virtual environment based on VR, and education and training are provided in the virtual environment.
  • a system that enables quantitative evaluation of education and training.
  • VR-based training system for children with developmental disabilities is a VR unit (Virtual Reality unit, 300), a hand detecting unit (hand detecting unit 100), a mounting unit (mounting unit) 200), and a training unit (training unit 400).
  • VR unit Virtual Reality unit, 300
  • hand detecting unit hand detecting unit 100
  • mounting unit mounting unit
  • training unit training unit 400
  • the VR unit 300 it is provided and mounted on the user's face, and is configured to display virtual space information with both eyes of the user.
  • the VR unit 300 is a type of virtual reality goggles that are generally commercially available, and is typically a product of Oculus, but VR-based goggles of Samsung Electronics, Sony, HTC, Pymax, and Max Steel. Since it is sufficient, detailed description of the principle or function will be omitted.
  • the hand detecting unit 100 As shown in FIG. 1, it is provided and mounted on the top of the VR unit 300 to detect the shape of the three-dimensional space of the user's hands, and the user thus detected
  • the 3D space shape of the hand is transmitted to the training unit 400, so that the 3D space shape of the user's hand is ultimately overlapped with the virtual space information provided by the VR unit 300.
  • the configuration is mounted on the upper surface of the VR unit 300, the upper portion of the mounting unit 200, the hand detecting unit 100 as described above It is to be mounted, and the lower part of the mounting unit 200 is a configuration that is fixedly attached to the VR unit 300 as described above.
  • the front and rear, upper and lower angles are adjusted so that the angle of the hand detecting unit 100 can be adjusted so that the hand detecting unit 100 mounted on the upper portion faces the user's hand. It is provided to be.
  • the body from the cuff to the elbow covers the hand and the finger according to the angle of the arm and the hand, so the height of the mounting unit 200 is mounted. And accordingly, the position of the height of the hand detecting unit 100 is important.
  • the training unit 400 As well as providing virtual space information to the VR unit 300 as described above, receiving the hand shape recognized by the hand detecting unit 100, and providing it to the VR unit 300 By overlapping with the virtual space information, the user exists in the virtual space information so that the user feels like using his or her hand in the virtual space information as the user's hand actually moves.
  • the mission is presented to the user who wears the VR unit 300 through the VR unit 300, and the motion of the user's hand shape is recognized by performing the mission, and the motion information of the hand shape According to this, the user's training information is evaluated.
  • the mounting unit 200 of the VR-based training system for a child with developmental disabilities is a holding unit 210, a first bar 220, a second bar 230, and docking as shown in FIG. It may include a portion 240.
  • the holding unit 210 As shown in FIG. 5, while holding the hand detecting unit 100, the angles of the front and rear are adjusted, and the handy detecting unit 100 looks at the user's hand. To make it possible.
  • one end thereof is hingedly connected to the holding portion, and is configured to control the angle of the holding portion 210.
  • first bar 220 In the case of the first bar 220, it is connected to the first link 211 of the holding unit 210.
  • the first link 211 is connected to the first bar 220 through the first hinge 212.
  • the second link 221 is provided, and the second hinge 222 may be provided.
  • one end thereof is hinged to the other end of the first bar 220 so that the angle adjustment of the first bar 220 is made.
  • a third hinge 231 may be provided to be connected to the docking part 240.
  • the other end of the second bar 230 is hinged to one side thereof, and the angle of the second bar 230 is adjusted, and is fixedly attached to the upper surface of the VR unit 300.
  • the radio wave transmitting unit 110, the radio wave receiving unit 120, and the structure recognition unit 130 may be included.
  • the hand detecting unit 100 transmits radio waves having a predetermined wavelength band in the user's hand direction from the top of the VR unit 300. .
  • the radio wave transmitting unit 110 should transmit radio waves in a predetermined wavelength band, as described above, the mounting unit 200 should be present on the top of the VR unit 300, and the VR unit The hand detecting unit 100 should be able to transmit radio waves toward both hands of the user in a mounted state of the hand detecting unit 100 at the upper portion of the 300.
  • the radio wave having a predetermined wavelength band is preferably set to have a wavelength in the range of 780 nm to 1 mm, so that it is not recognized by the human eye.
  • the energy When the energy is lower than 1.24 meV, it is difficult to recognize the reflected wave coming back, and thus it is difficult to recognize the three-dimensional shape of both hands of the user.
  • it When it is greater than 1.7 eV, the user can recognize the energy of the skin, that is, heat energy.
  • the photon energy region as described above will be set.
  • the radio wave receiving unit 120 it is arranged to be spaced apart from the radio wave transmitting unit 110, and after being transmitted by the radio wave transmitting unit 110, the reflected wave reflected from the user's hand is received.
  • radio wave receiver 120 at least two or more radio wave recognition modules, such as CMOS, are provided with a module capable of recognizing electromagnetic waves in the above-described wavelength range, thereby obtaining reflected waves reflected from the hand.
  • CMOS complementary metal-oxide-semiconductor
  • the structure recognition unit 130 As shown in FIG. 3(a), receiving the reflected wave received by the radio wave receiving unit 120, the mutual distance and phase of the reflected wave are recognized to recognize the shape of the user's hand. Is done.
  • the hand detecting unit 100 in order to add realism in addition to the hand shape recognized by the user, in order to reflect and map the actual image of the user's hand on the hand shape, an image for acquiring the user's hand image is acquired.
  • the unit 140 may be further included.
  • the hand mapping unit 150 and the movement transmitting unit 160 may be further included.
  • the structure recognition unit 130 acquires the recognized hand shape, and the image acquisition unit 140 reflects the image of visible light reflected from the user's hand.
  • mapping to the shape only the portion of the visible light image that is mapped to the hand shape is selectively acquired.
  • the movement transmitting unit 160 transmits a change of the 3D hand image over time from the hand mapping unit 150, that is, a motion unit 400 of the operation information of the 3D hand image. Is done.
  • the background transmitting unit 410, the movement receiving unit 420, and the overlapping unit 430 It can contain.
  • virtual background information that is, virtual space information
  • virtual space information to provide an experience to a user wearing the VR unit 300
  • the VR unit 300 the user It makes you feel like you are in virtual space information.
  • a three-dimensional hand image is received from the movement transmitting unit 160 of the hand detecting unit 100 as described above.
  • the virtual space information that the user equipped with the VR unit 300 feels as belonging by overlapping the 3D hand image with the virtual space information transmitted by the background transmitting unit 430 You feel like you are experiencing something by using your own hands.
  • the mission presentation unit 440 and the evaluation measurement report 450 may be further included.
  • the user may go to a kitchen viewed from virtual space information and perform an action of dripping coffee in the virtual space through his 3D hand image.
  • the motion information of the 3D hand image is received, and then, the motion information of the 3D hand image is presented in a preset evaluation criteria table, and the 3D hand image operation is performed. Information will be evaluated.
  • the good contact (green point) of the target area is set, and as shown in FIG. 14 in which the level is increased, the good contact (green point) and the bad contact (red point) of the target area are respectively set.
  • the contact point is set as a green point in the user's 3D hand image.
  • the operation information is quantitatively evaluated by evaluating whether the contact of the user's 3D hand image as shown in FIG. 15 touches the good or bad contact in FIGS. 13 and 14 as shown below. Will be evaluated.
  • Thumb tip Index finger Stop tip Ring finger tips Possession tips Height 1 (good contact point) 5 5 4 -One -3 Height 2 (good contact point) 5 4 5 0 -2 Height 3 (good contact point) 5 3 4 5 0 Height 4 (good contact point) 2 0 3 4 3 Height 5 (good contact point) 0 -2 0 3 5
  • the mission exceeds a certain threshold and passes, the target area is visually flashed or flashed, etc. Reward is given.
  • a penalty may be given, such as shaking of the target area.
  • the mission presentation unit 440 may present the next mission to pass the mission.
  • a guide presentation unit 460 may be further included.
  • the guide presenting unit 460 it is a configuration that presents a predetermined guide to the user to perform a preset action presented by the mission presenting unit 440.
  • the guide presentation unit 460 may include a route presentation unit 461, an outline presentation unit 462, and a target presentation unit 463, as shown in FIG. 7. have.
  • a virtual path (indicated by a blue dotted line in FIG. 12) that the user needs to move the 3D hand image for a preset action is shown in FIG. 300), and the virtual path is formed from the 3D hand image to the target position.
  • the contact contact as shown in FIG. 15 to the good contact of the target region, as shown in FIG. 13, are continued.
  • the virtual path is linked according to the user's motion information, and the virtual path is also moved.
  • FIG. 14 the defective contact in FIG. 14 is avoided from the contact contact in FIG. 15, and a dotted line continues to the good contact.
  • the outline presentation unit 462 the outline of the target region to be reached by the 3D hand image is displayed to visually identify the target region to the user.
  • a predetermined mark for example, a red star, is displayed on the target region to which the 3D hand image is to reach, so as to inform the user.
  • the mission presenting unit 440 of the VR-based training system for a child with developmental disabilities may include a voice recognition unit 441 to recognize the voice of the manager.
  • the mission presentation unit 440 may further include a character extraction unit 442 and a frequency modulation unit 442.
  • the voice of the manager recognized by the speech recognition unit 441 is converted into text, that is, text through speech recognition, and displayed on the screen as shown in FIG.
  • the voice recognition unit After acquiring this, 441, the frequency modulator 443 modulates the voice into a specific frequency band of the entertainer preferred by the user, as described above.
  • the frequency modulator 443 modulates into a frequency band of a frequency band of the entertainer, passes a band pass filter (BPF) through which only the corresponding frequency band passes, and then passes the modulated voice signal through a certain section. After amplifying as much as possible, it is heard by the user. If this happens, the user with a developmental disability increases the concentration and goal consciousness to perform the mission, and helps to perform the mission in a fun way.
  • BPF band pass filter

Abstract

La présente invention concerne un système d'apprentissage basé sur la réalité virtuelle (RV). La présente invention concerne une idée technique comprenant : une unité de RV pour afficher des informations d'espace virtuel ; une unité de détection de main pour détecter une forme de main d'un utilisateur ; une unité de montage conçue pour permettre un ajustement d'angle dans une direction dans laquelle l'unité de détection de main détecte la forme de main de l'utilisateur ; et une unité d'apprentissage qui reçoit une forme de main, superpose la forme de main reçue sur les informations d'espace virtuel, suggère une tâche à l'utilisateur par l'intermédiaire de l'unité de RV, puis évalue le degré d'apprentissage de l'utilisateur selon des informations relatives à un mouvement de la forme de main.
PCT/KR2019/012174 2018-12-14 2019-09-19 Système d'apprentissage basé sur la réalité virtuelle (rv) pour un enfant ayant un trouble du développement WO2020122375A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR10-2018-0161969 2018-12-14
KR1020180161969A KR102016676B1 (ko) 2018-12-14 2018-12-14 발달장애아를 위한 vr기반의 트레이닝 시스템

Publications (1)

Publication Number Publication Date
WO2020122375A1 true WO2020122375A1 (fr) 2020-06-18

Family

ID=67776279

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2019/012174 WO2020122375A1 (fr) 2018-12-14 2019-09-19 Système d'apprentissage basé sur la réalité virtuelle (rv) pour un enfant ayant un trouble du développement

Country Status (2)

Country Link
KR (1) KR102016676B1 (fr)
WO (1) WO2020122375A1 (fr)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102016676B1 (ko) * 2018-12-14 2019-08-30 주식회사 홀로웍스 발달장애아를 위한 vr기반의 트레이닝 시스템
KR102142062B1 (ko) 2019-11-22 2020-08-07 주식회사 유니브이알 Vr 콘텐츠 연동 홈 트레이닝 장치
KR20210148764A (ko) 2020-06-01 2021-12-08 한국전자통신연구원 가상 콘텐츠 제공 장치 및 방법
KR102430066B1 (ko) * 2020-06-22 2022-08-08 주식회사 휴메닉 비대면형 인지 및 운동 재활 훈련 시스템 및 방법
KR102430067B1 (ko) * 2020-06-22 2022-08-08 주식회사 휴메닉 운동 병합형 가상현실 기반 인지재활 훈련 시스템 및 방법
KR102594249B1 (ko) 2020-11-18 2023-10-26 한국전자통신연구원 실감 인터랙션 기반의 가상 훈련 방법 및 장치
KR102569857B1 (ko) * 2021-01-27 2023-08-23 주식회사 시온네트웍 Hmd 핸드트래킹을 이용한 실기술 트레이닝 제공방법

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101044901B1 (ko) * 2011-01-14 2011-06-29 탐투스 주식회사 증강현실 촬영을 위한 실물 화상기를 구비한 디지털 학습장치
KR20110116422A (ko) * 2010-04-19 2011-10-26 서경대학교 산학협력단 마커와 훈련자 손 인식을 통한 증강 현실 상황 훈련 시스템
US20110270135A1 (en) * 2009-11-30 2011-11-03 Christopher John Dooley Augmented reality for testing and training of human performance
KR20180028033A (ko) * 2016-09-05 2018-03-15 고려대학교 산학협력단 사용자 환경 기반 가상현실을 이용한 장애인 일상생활 교육 방법 및 이를 위한 장치
KR102016676B1 (ko) * 2018-12-14 2019-08-30 주식회사 홀로웍스 발달장애아를 위한 vr기반의 트레이닝 시스템

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20080005798A (ko) 2006-07-10 2008-01-15 (주)유리시스 모션트랙킹 기술과 증강현실 기법을 활용한 인지 및행동장애 재활 지원시스템
US20170061700A1 (en) 2015-02-13 2017-03-02 Julian Michael Urbach Intercommunication between a head mounted display and a real world object
KR20170079135A (ko) 2015-12-30 2017-07-10 한국생산기술연구원 가상 현실 기반 사용자 맞춤형 훈련 시스템

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110270135A1 (en) * 2009-11-30 2011-11-03 Christopher John Dooley Augmented reality for testing and training of human performance
KR20110116422A (ko) * 2010-04-19 2011-10-26 서경대학교 산학협력단 마커와 훈련자 손 인식을 통한 증강 현실 상황 훈련 시스템
KR101044901B1 (ko) * 2011-01-14 2011-06-29 탐투스 주식회사 증강현실 촬영을 위한 실물 화상기를 구비한 디지털 학습장치
KR20180028033A (ko) * 2016-09-05 2018-03-15 고려대학교 산학협력단 사용자 환경 기반 가상현실을 이용한 장애인 일상생활 교육 방법 및 이를 위한 장치
KR102016676B1 (ko) * 2018-12-14 2019-08-30 주식회사 홀로웍스 발달장애아를 위한 vr기반의 트레이닝 시스템

Also Published As

Publication number Publication date
KR102016676B1 (ko) 2019-08-30

Similar Documents

Publication Publication Date Title
WO2020122375A1 (fr) Système d'apprentissage basé sur la réalité virtuelle (rv) pour un enfant ayant un trouble du développement
Yuan et al. Is the rubber hand illusion induced by immersive virtual reality?
Slater et al. Towards a digital body: the virtual arm illusion
Youngblut et al. Review of virtual environment interface technology
US20200118456A1 (en) Secure Testing Device With Combiner
Frey et al. Teegi: tangible EEG interface
Barfield et al. Comparison of human sensory capabilities with technical specifications of virtual environment equipment
Souza et al. Measuring presence in virtual environments: A survey
Dargar et al. Towards immersive virtual reality (iVR): a route to surgical expertise
Plopski et al. The eye in extended reality: A survey on gaze interaction and eye tracking in head-worn extended reality
US20050255434A1 (en) Interactive virtual characters for training including medical diagnosis training
MXPA06011083A (es) Sistema de cristal de aumento de entidad digital tridimensional, con funciones de instruccion visual tridimensional.
US20200301150A1 (en) Secure testing device with liquid crystal shutter
WO2013133583A1 (fr) Système et procédé de réhabilitation cognitive par interaction tangible
Caporusso et al. A wearable device supporting multiple touch-and gesture-based languages for the deaf-blind
O'Regan et al. Acting out our sensory experience
Katzakis et al. Stylo and handifact: Modulating haptic perception through visualizations for posture training in augmented reality
CN108446011A (zh) 一种基于增强现实的医疗辅助方法及设备
US11287971B1 (en) Visual-tactile virtual telepresence
Ariza et al. Inducing body-transfer illusions in VR by providing brief phases of visual-tactile stimulation
Richards et al. Analysis of peripheral vision and vibrotactile feedback during proximal search tasks with dynamic virtual entities in augmented reality
Ryan et al. The existence of internal visual memory representations
WO2020138671A1 (fr) Système d'évaluation d'actes chirurgicaux basé sur la réalité virtuelle, utilisant un simulateur, pour l'oto-laryngologie et la neurochirurgie
Katzakis et al. Evaluation of 3D pointing accuracy in the fovea and periphery in immersive head-mounted display environments
Zhu et al. An investigation into the effectiveness of using acoustic touch to assist people who are blind

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19896516

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 29/09/2021)

122 Ep: pct application non-entry in european phase

Ref document number: 19896516

Country of ref document: EP

Kind code of ref document: A1