KR101810039B1 - Visual Aid System - Google Patents

Visual Aid System Download PDF

Info

Publication number
KR101810039B1
KR101810039B1 KR1020150190794A KR20150190794A KR101810039B1 KR 101810039 B1 KR101810039 B1 KR 101810039B1 KR 1020150190794 A KR1020150190794 A KR 1020150190794A KR 20150190794 A KR20150190794 A KR 20150190794A KR 101810039 B1 KR101810039 B1 KR 101810039B1
Authority
KR
South Korea
Prior art keywords
camera
image
user
interest
unit
Prior art date
Application number
KR1020150190794A
Other languages
Korean (ko)
Other versions
KR20170079818A (en
Inventor
박준
박진현
박한
Original Assignee
홍익대학교 산학협력단
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 홍익대학교 산학협력단 filed Critical 홍익대학교 산학협력단
Priority to KR1020150190794A priority Critical patent/KR101810039B1/en
Publication of KR20170079818A publication Critical patent/KR20170079818A/en
Application granted granted Critical
Publication of KR101810039B1 publication Critical patent/KR101810039B1/en

Links

Images

Classifications

    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G02B27/0172—Head mounted characterised by optical features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/014—Hand-worn input/output arrangements, e.g. data gloves
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0101—Head-up displays characterised by optical features
    • G02B2027/014—Head-up displays characterised by optical features comprising information/image processing systems
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0179—Display position adjusting means not related to the information to be displayed
    • G02B2027/0187—Display position adjusting means not related to the information to be displayed slaved to motion of at least a part of the body of the user, e.g. head, eye

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Optics & Photonics (AREA)
  • Human Computer Interaction (AREA)
  • User Interface Of Digital Computer (AREA)
  • Studio Devices (AREA)

Abstract

The present invention relates to an auxiliary visual system, and more particularly, to an auxiliary visual system which does not obtain through the eyes of a user but enlarges or tracks visual information desired by the user through the image and control of the PTZ camera to generate additional visual information, It is not possible to obtain the information through the eyes of the user but the visual information desired by the user is additionally provided to the user so that the user can not gaze or zoom continuously because of the user's eyes, Vision system that overcomes the limitations of vision, distance, and gaze of the eye, and monitors the broad scope and uses the workforce more efficiently in tasks that require two simultaneous gazes.

Description

Visual Aid System}

The present invention relates to an auxiliary visual system, and more particularly, to an auxiliary visual system which does not obtain through the eyes of a user but enlarges or tracks visual information desired by the user through the image and control of the PTZ camera to generate additional visual information, And an auxiliary visual system which can not obtain the visual information through the user's eyes but additionally provides the user with desired visual information.

In general, existing assistive vision systems have been developed for the convenience of their lives for people with disabilities.

However, the eyes of ordinary people without disabilities as well as those with disabilities also have various limitations, and an auxiliary visual system to overcome these limitations will bring many improvements for the convenience of the general public.

In particular, the human eye has the problem that it is impossible to obtain visual information that is out of sight or at a long distance, and it is difficult to simultaneously observe two distances beyond a certain distance.

Patent Document 10-2009-0084399 Engineering patent 10-2015-0023702

The problem to be solved by the present invention is to overcome the above-described problems and to allow the user to continuously gaze or magnify the information desired by the user although the user can not continuously gaze or zoom in on the user's eyes.

Another problem to be solved by the present invention is to allow both the military and security field to observe a wide range of information and to observe and operate the two visual information simultaneously.

[0001] The present invention relates to an auxiliary visual system, and in an auxiliary visual system (100), a camera (110) having a Pan - Tilt - Zoom (PTZ) function for controlling a time of a camera; An HMD (120) which allows a user to see the outside when worn and outputs additional time information provided by the auxiliary time system (100) to the user; An operation sensing unit 130 recognizing the operation of the user and allowing the user to control the auxiliary visual system 100; And analyzing information of the operation of the user through the motion sensing unit 130, stabilizing an image through the camera 110, tracking an object of interest of the image, And a processing unit (140) for controlling an output screen of the HMD (120).

At this time, the processing unit 140 may control the camera and the screen management unit 142d through the motion detection unit 130 to track the object of interest by the processing unit 140 and control the camera and screen management unit 142d, An operation processing unit (141) for analyzing the hand operation information of the user; And an auxiliary visual processing unit for stabilizing the shaking of the image through the camera 110 and for tracking the user's interest in the image and controlling the operation of the camera 110 and the output screen of the HMD 120 142).

Also, the auxiliary time processing unit 142 may include an image processing unit 142a for stabilizing the shaking of the image through the camera 110, and analyzing the image to track the user's interest. Determines the next operation of the camera 110 according to the analysis result of the user's hand operation through the operation processing unit 141 and the result of the target tracking through the image processing unit 142a and then transmits the determination to the camera control unit 142c An auxiliary time setting unit 142b for transmitting the time information; A camera controller 142c for controlling the camera 110 according to contents of a next operation of the camera 110 received from the auxiliary time setting unit 142b; And a screen management unit 142d for providing the user with the user's interest on the screen of the HMD 120 through the HMD 120. [

In addition, the image processing unit 142a may include a stabilization unit 142a1 for stabilizing the shaking of the image through the camera 110 through low-level image shake correction. And a tracking unit 142a2 for analyzing the image through a 2D tracking technique in order of cursing and tracking the user's interest.

Also, at this time, the stabilization unit 142a1 stabilizes the image by comparing the reference image and the current image using robust feature extraction technology and optical flow technology.

Also, at this time, the tracking unit 142a2 uses the low-level 2D tracking technique through comparison between the image of interest extracted from the image of the camera 110 and the current image.

According to the present invention, the user can not continuously gaze or zoom in due to the constraints of the user's eyes, but the user can continuously gaze or magnify the desired information, thereby overcoming restrictions on the user's eyesight, distance, It is effective.

In addition, according to the present invention, it is possible to monitor a wide range through overcoming restrictions on eyesight, distance, and gazing of the user's eyes, and to use the workforce more efficiently in an operation in which two eyes must be simultaneously examined.

1 is a block diagram illustrating a configuration of an auxiliary visual system according to an exemplary embodiment of the present invention.
2 is a block diagram for explaining a configuration of a processing unit in an auxiliary visual system according to an embodiment of the present invention.
3 is a block diagram for explaining a configuration of an auxiliary visual processing unit in a processing unit in the auxiliary visual system according to an embodiment of the present invention.
4 is a block diagram for explaining a configuration of an image processing unit in an auxiliary visual processing unit in a processing unit in an auxiliary visual system according to an exemplary embodiment of the present invention.
FIG. 5 is an example of enlarging an object of interest through the HMD of the auxiliary visual system according to an embodiment of the present invention.
FIG. 6 is an example of tracking an object of interest through the HMD of the auxiliary visual system according to an embodiment of the present invention.

Before describing the specific details for the practice of the invention, terms and words used in the specification and claims should be construed to enable the inventor to properly define the concept of a term in order to best describe its invention The present invention is not limited thereto.

It is to be noted that the detailed description of known functions and constructions related to the present invention is omitted when it is determined that the gist of the present invention may be unnecessarily blurred.

Hereinafter, an auxiliary visual system according to a preferred embodiment of the present invention will be described in detail with reference to FIGS. 1 to 5. FIG.

1 is a block diagram illustrating a configuration of an auxiliary visual system according to an exemplary embodiment of the present invention.

1, the auxiliary visual system 100 according to the preferred embodiment of the present invention includes a camera 110 having a Pan-Tilt-Zoom (PTZ) function for controlling the viewing angle of a camera, An HMD 120 for allowing a user to see the outside and outputting additional time information provided by the auxiliary time system 100 to the user, a controller for recognizing the operation of the user and controlling the auxiliary time system 100 And an operation unit 130 for analyzing the operation of the user through the motion sensing unit 130 and stabilizing an image through the camera 110 and tracking an object of interest of the image, And a processing unit 140 for controlling the operation of the HMD 120 and the output screen of the HMD 120.

In addition, a preferred embodiment of the motion detector 130 for analyzing the hand operation information of the user analyzes whether the user has performed a click or sweep operation.

2 is a block diagram for explaining a configuration of a processing unit in an auxiliary visual system according to an embodiment of the present invention.

2, the processing unit 140 tracks the object of interest by the processing unit 140 through the hand operation information of the user and controls the camera and screen management unit 142d to control the camera and screen management unit 142d. An operation processing unit 141 for analyzing the hand operation information of the user through the camera 130, and a controller 140 for stabilizing the shaking of the image through the camera 110, tracking the user's interest in the image, And an auxiliary visual processing unit 142 for controlling the operation of the HMD 120 and the output screen of the HMD 120.

3 is a block diagram for explaining a configuration of an auxiliary visual processing unit in a processing unit in the auxiliary visual system according to an embodiment of the present invention.

3, the auxiliary time processing unit 142 includes an image processing unit 142a for stabilizing the shaking of the image through the camera 110 and analyzing the image to track the user's interest, Determines the next operation of the camera 110 according to the analysis result of the user's hand operation through the operation processing unit 141 and the tracking result of the object of interest through the image processing unit 142a and transmits the determination to the camera control unit 142c A camera controller 142c for controlling the camera 110 according to contents of a next operation of the camera 110 transmitted from the auxiliary time setting unit 142b and a camera controller 142c for controlling the camera 110, And a screen management unit 142d for providing the user with the interest of the user on the screen of the HMD 120 through the display unit 120. [

FIG. 5 is an example of enlarging an object of interest through the HMD of the auxiliary visual system according to an embodiment of the present invention.

5, the screen management unit 142d may provide the user with an interest of the user on the screen of the HMD 120 through the HMD 120. In this case, If the object is far away from the position of the user, the camera 110 enlarges an object of interest and provides it to the screen of the HMD 120.

FIG. 6 is an example of tracking an object of interest through the HMD of the auxiliary visual system according to an embodiment of the present invention.

6, the screen management unit 142d may provide the user with a subject of interest to the screen of the HMD 120 through the HMD 120. For example, The camera 110 keeps track of an object of interest so that the user can continue to gaze through the camera 110 with the object of interest.

4 is a block diagram for explaining a configuration of an image processing unit in an auxiliary visual processing unit in a processing unit in an auxiliary visual system according to an exemplary embodiment of the present invention.

4, the image processing unit 142a includes a stabilization unit 142a1 for stabilizing the shaking of the image through the camera 110 through low-level image stabilization, And a tracking unit 142a2 for tracking the user's interest.

Also, at this time, the stabilization unit 142a1 stabilizes the image by comparing the reference image and the current image using robust feature extraction technology and optical flow technology.

In addition, the stabilization unit 142a1 stabilizes the shaking of the image by comparing the reference image and the current image using the feature point extraction technique and the optical flow technique, which are robust to low-level image blurring correction, Thereby correcting the shake when there is a shake in the image.

Also, at this time, the tracking unit 142a2 uses the low level 2D tracking technology through comparison between the image of interest extracted from the image of the camera 110 and the current image.

In addition, a preferred embodiment in which the tracking unit 142a2 tracks the object of interest is a method for tracking the object of interest by comparing the image of the object of interest extracted from the image of the camera 110, which is a low- (PAN) and vertical (Tilt) movement of the camera 110 are controlled through the camera controller 142c such that a target of interest is located at the center of the image when an object of interest is detected in the image.

While the present invention has been particularly shown and described with reference to preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. It will be appreciated by those skilled in the art that numerous changes and modifications can be made without departing from the invention. Accordingly, all such modifications and variations are intended to be included within the scope of the present invention.

100: Auxiliary Visual System
110: camera
120: Optical see-through head-mounted display (HMD)
130:
140:
141:
142: auxiliary time processing unit
142a:
142a1:
142a2:
142b:
142c:
142d:

Claims (6)

In the auxiliary visual system 100,
A camera 110 having a Pan - Tilt - Zoom (PTZ) function for controlling the time of the camera;
An HMD (120) which allows a user to see the outside when worn and outputs additional time information provided by the auxiliary time system (100) to the user;
An operation detecting unit 130 recognizing a click or a swipe operation of the user and allowing the user to control the auxiliary visual system 100; And
The operation of the camera 110 is analyzed by analyzing information of the operation of the user through the motion sensing unit 130, stabilizing the image through the camera 110, tracking an object of interest of the image, A processing unit 140 for controlling an output screen of the display unit 120;
, ≪ / RTI &
The processing unit (140)
The hand operation information of the user through the motion detection unit 130 is analyzed through the hand operation information of the user to track the object of interest by the processing unit 140 and to control the camera and screen management unit 142d An operation processing unit 141 for executing the operation; And
An auxiliary vision processor 142 for stabilizing the shaking of the image through the camera 110 and for tracking the user's interest in the image and for controlling the operation of the camera 110 and the output screen of the HMD 120, );
, ≪ / RTI &
The auxiliary time processing unit (142)
An image processor 142a for stabilizing the shaking of the image through the camera 110 and analyzing the image to track the user's interest;
Determines the next operation of the camera 110 according to the analysis result of the user's hand operation through the operation processing unit 141 and the result of the target tracking through the image processing unit 142a and then transmits the determination to the camera control unit 142c An auxiliary time setting unit 142b for transmitting the time information;
A camera controller 142c for controlling the camera 110 according to contents of a next operation of the camera 110 received from the auxiliary time setting unit 142b; And
A screen management unit 142d for providing the user with an interest of the user on the screen of the HMD 120 through the HMD 120;
, ≪ / RTI &
The screen management unit 142d
If the object of interest is distant from the position of the user, the object of interest is enlarged through the enlargement function of the camera 110 and provided to the screen of the HMD 120. Even if the user gazes at a portion The camera 110 keeps track of an object of interest so that the user can continue to gaze through the camera 110 on the screen of the HMD 120,
The image processing unit 142a,
A stabilization unit 142a1 for stabilizing a shake of an image through the camera 110 through a low-level image shake correction; And
A tracking unit (142a2) for analyzing the image through a 2D tracking technique in order of curses and tracking the user's interest;
, ≪ / RTI &
The stabilizing unit 142a1 includes:
Stabilizing the image by comparing the reference image and the current image using a robust feature extraction technique and an optical flow technique by low-level image blur correction,
The tracking unit 142a2,
When an object of interest is detected in the image of the camera 110 through comparison between the image of the object of interest extracted from the image of the camera 110 and the current image, rotation of the camera 110 such that the object of interest is located in the center of the image (PAN) and vertical (Tilt) movement is controlled through the camera controller 142c to use low level 2D tracking technology. delete delete delete delete delete
KR1020150190794A 2015-12-31 2015-12-31 Visual Aid System KR101810039B1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020150190794A KR101810039B1 (en) 2015-12-31 2015-12-31 Visual Aid System

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020150190794A KR101810039B1 (en) 2015-12-31 2015-12-31 Visual Aid System

Publications (2)

Publication Number Publication Date
KR20170079818A KR20170079818A (en) 2017-07-10
KR101810039B1 true KR101810039B1 (en) 2018-01-18

Family

ID=59356525

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020150190794A KR101810039B1 (en) 2015-12-31 2015-12-31 Visual Aid System

Country Status (1)

Country Link
KR (1) KR101810039B1 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3690627A1 (en) * 2019-01-30 2020-08-05 Schneider Electric Industries SAS Graphical user interface for indicating off-screen points of interest

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000101898A (en) * 1998-09-21 2000-04-07 Fuji Photo Film Co Ltd Electronic camera
JP2015057659A (en) * 2014-10-24 2015-03-26 ソニー株式会社 Display device, display method, and program
JP2015061200A (en) * 2013-09-18 2015-03-30 コニカミノルタ株式会社 Image display device, image display method, and image display program

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000101898A (en) * 1998-09-21 2000-04-07 Fuji Photo Film Co Ltd Electronic camera
JP2015061200A (en) * 2013-09-18 2015-03-30 コニカミノルタ株式会社 Image display device, image display method, and image display program
JP2015057659A (en) * 2014-10-24 2015-03-26 ソニー株式会社 Display device, display method, and program

Also Published As

Publication number Publication date
KR20170079818A (en) 2017-07-10

Similar Documents

Publication Publication Date Title
US10984756B2 (en) Adaptive parameters in image regions based on eye tracking information
KR101663452B1 (en) Screen Operation Apparatus and Screen Operation Method
CN107272904B (en) Image display method and electronic equipment
US20210350762A1 (en) Image processing device and image processing method
JP2019531782A (en) Sensor fusion system and method for eye tracking applications
US11487354B2 (en) Information processing apparatus, information processing method, and program
KR20150101612A (en) Head Mounted Display with closed-view and Method for controlling the same
JP2019125215A (en) Information processing apparatus, information processing method, and recording medium
US10347050B1 (en) Augmented reality device to warp images
JP2016115965A (en) Medical spectacle type display device, information processing device, and information processing method
JP2013148599A (en) Display device
US11443719B2 (en) Information processing apparatus and information processing method
US11343420B1 (en) Systems and methods for eye-based external camera selection and control
EP3547079B1 (en) Presenting images on a display device
WO2016157923A1 (en) Information processing device and information processing method
KR101810039B1 (en) Visual Aid System
US11474595B2 (en) Display device and display device control method
US20210314557A1 (en) Information processing apparatus, information processing method, and program
US11270451B2 (en) Motion parallax in object recognition
JP2017091190A (en) Image processor, image processing method, and program
KR20160144245A (en) Projector apparatus
CN111886564B (en) Information processing device, information processing method, and program
US20240119643A1 (en) Image processing device, image processing method, and computer-readable storage medium
US9986138B2 (en) Eyetracker mounts for use with handheld devices
WO2022208595A1 (en) Wearable terminal device, program, and notification method

Legal Events

Date Code Title Description
A201 Request for examination
E902 Notification of reason for refusal
E90F Notification of reason for final refusal
E701 Decision to grant or registration of patent right
GRNT Written decision to grant