KR20190108909A - System and method for measurement of visual acuity based on the analysis of user eye movements - Google Patents

Abstract

Disclosed are a sight measurement method using movement of user′s eyes, which displays a sight measurement mark on a screen of a smartphone held by a user and naturally measures user′s sight when the user′s eyes follow the measurement mark without user′s input, and a system thereof. According to the present invention, the sight measurement system using movement of user′s eyes comprises: a camera image input unit photographing a user′s face to input a captured facial image; an eye image extraction unit extracting the position, area, and information of eyes from the facial image and extracting a binocular distance value based on the position of the eyes; a distance calculation unit calculating a distance from the user based on the binocular distance value; a screen generation unit for each sight generating a screen displayed while moving a measurement mark corresponding to each sight value in accordance with the distance from the user; a screen display unit displaying and outputting the generated screen; a gaze position extraction unit extracting a gaze position of a pupil moving along the moving measurement mark; a gaze matching detection unit detecting whether the moving position of the measurement mark matches with the gaze position of the pupil; and a sight determination unit determining a sight value based on whether the moving position of the measurement mark matches with the gaze position of the pupil.

Description

System and method for measurement of visual acuity based on the analysis of user eye movements}

The present invention relates to a method and system for measuring vision using a user's eye movements, and more particularly, to display an eye measurement marker on a screen of a smart phone carried by a user, and the user's eyes are displayed on the screen of the smart phone without a user's input. The present invention relates to a method and system for measuring eyesight using a user's gaze movement, which enables a user to naturally measure eyesight only by following a moving measurement mark.

The range that human eyes can see without moving is called vision. The visual field of view of the eye is 58 degrees above, 73 degrees below, 65 degrees inside the medulla, and 100 degrees outside. The central part where the left and right eye's field of view overlaps is a range where the stereoscopic field of view seen by the eye is possible.

In order to measure the eyesight of the eyes having such a vision, the user should visit a hospital or an optician who can directly measure eyesight.

The principle of measuring eyesight in such a place is to guide the user to stare at the sight plate marked with an image or text on a paper plate as shown in FIG. 1 according to the manager's guidance. The visual acuity value of the user is to recognize the visual acuity of the user. 1 is a view showing a general vision measurement principle.

The principle of vision measurement is that visual acuity refers to the 'minimum' of the ability of the brain to recognize these two points quickly when two points are close, as shown in FIG. The visual acuity is 1.0 when the minimum time that distinguishes two points is 1 minute, and the visual acuity is 2.0 when the minimum time is 0.5 minute.

However, in order to measure the eyesight, there is a problem in that the user has to go to the place where the eyesight can be measured.

In order to solve this problem, a technology for measuring eyesight of a user is provided by installing an eyesight measurement application (App) on a smart phone carried by a user.

However, when measuring the eyesight of the user through a conventional smart phone, there was an inconvenience in that the user directly inputs a value using a measurement marker or the like and measures by touching the screen.

Korean Laid-Open Patent Publication No. 10-2017-0085309 (published date: July 24, 2017)

An object of the present invention for solving the above-mentioned problems is to display an eye measurement marker on a screen of a smart phone carried by the user, and to follow the measurement marker that the user's eyes move on the screen of the smart phone without the user's input. The present invention provides a method and system for measuring eyesight using a user's gaze movement, which enables a user to naturally measure eyesight alone.

According to an aspect of the present invention, there is provided a vision measuring system using a user's gaze movement, including: a camera image input unit configured to input a face image captured by capturing a face of a user; An eye image extracting unit extracting eye position, area, and information from the facial image, and extracting a binocular distance value based on the eye position; A distance calculator configured to calculate a distance to a user based on a binocular distance value; A vision-specific screen generation unit configured to generate a screen to be displayed while moving a measurement mark corresponding to each visual acuity value according to a distance from the user; A screen display unit which displays and displays the generated screen; A gaze position extraction unit for extracting a gaze position of a pupil moving along a moving measurement mark; A gaze matching detector configured to detect whether a moving position of a measurement mark matches a gaze position of a pupil; And a visual acuity determining unit configured to determine a visual acuity value according to whether the movement position of the measurement mark matches the eye position of the eye.

In addition, the measurement marker may include an image or text having a size corresponding to each visual acuity value for determining a visual acuity value of the user.

The eye image extractor may extract the eyeline direction of the left or right eye of the user based on the position of the eye.

In addition, the eye image extractor may determine whether the user's gaze is maintained based on the position of the eye.

The distance calculator may calculate a distance from the user based on a triangular ratio based on a binocular distance value using a pixel value between two eyes.

In addition, the screen generation unit for each visual acuity may generate a screen for displaying a measurement marker corresponding to the initial reference visual acuity value according to the calculated distance from the user, or may display a screen for displaying a measurement marker for each visual acuity value for each measurement step. Can be generated.

In addition, the screen generation unit for each visual acuity may generate a screen for continuously outputting the measurement markers corresponding to each visual acuity value sequentially with a time difference according to the visual acuity value.

In addition, the screen generation unit for each visual acuity starts with a low visual acuity value, and gradually increases the visual acuity value, and continuously outputs a measurement marker corresponding to each visual acuity value sequentially with a time difference according to the visual acuity value. Can be generated.

In addition, the gaze matching detector may compare the coordinate value and the size value of the moving position of the measurement marker with the coordinate value and the size value of the gaze position to detect whether the match is performed.

The visual acuity determination unit may determine a visual acuity value of a measurement mark that was previously matched with a gaze position of a pupil as a visual acuity value of the user when the movement position of the measurement marker and the eye gaze position of the pupil do not match.

On the other hand, the eyesight measurement method using the user's eye gaze motion according to an embodiment of the present invention for achieving the above object, the camera image input unit, comprising the steps of inputting a facial image captured by capturing the user's face; Extracting, by the eye image extractor, the position of the eye from the facial image and extracting a binocular distance value based on the position of the eye; Calculating, by the distance calculator, a distance value with the user based on the binocular distance value; Generating a screen for displaying a measurement marker for each vision corresponding to a distance value from the user; Inputting, by the camera image input unit, a face image photographing a face of a user who moves eyes according to a vision-specific measurement mark moving on a screen; A gaze position extracting unit, extracting gaze positions of the eyes according to eye movements and gaze directions in the facial image; Detecting, by the gaze matching detector, whether a movement position of a measurement mark and a gaze position of a pupil match; And determining the visual acuity value of the user based on whether the moving position of the measurement marker matches the eye gaze position of the pupil.

In addition, the step of extracting the binocular distance value, the eye image extraction unit may extract the eye gaze direction of the left or right eye of the user based on the position of the eye.

In addition, in the step of extracting the binocular distance value, the eye image extractor may determine whether to maintain the user's gaze based on the position of the eye.

In the calculating of the distance value with the user, the distance calculator may calculate the distance with the user based on the triangular ratio based on the binocular distance value using the pixel value between the two eyes.

The generating of the screen to be displayed may include generating a screen for displaying a measurement mark corresponding to an initial reference visual acuity value in accordance with the distance from the calculated user, or for each visual acuity value per measurement step. You can create a screen that displays the corresponding measurement markers.

In the generating of the screen to be displayed, the screen generating unit for each sight may generate a screen for continuously outputting the measurement markers corresponding to each sight value sequentially while having a time difference depending on the sight value.

In addition, the step of generating the screen to be displayed, the screen-by-sight screen generation unit initially starts to lower the visual acuity value and gradually increase the visual acuity value, the measurement markers corresponding to each visual acuity value according to the visual acuity value sequentially You can create a screen that outputs continuously while moving to.

In addition, the step of detecting whether the eye gaze position of the pupil is matched, the gaze matching detection unit detects a match by comparing the coordinate value and the size value for the movement position of the measurement marker with the coordinate value and the size value for the eye position. can do.

The determining of the visual acuity value of the user may include: when the visual acuity determination unit does not match the movement position of the measurement mark and the eye gaze position of the pupil, the visual acuity value of the measurement mark that was previously matched with the eye gaze position of the eye. It can be judged by the value.

According to the present invention, when measuring the eyesight of the user through a smart phone, it is possible to solve the inconvenience that the user must input directly, and inaccuracies that occur when the user's input is wrong.

In addition, since the user can automatically measure the user's vision simply by staring at and following the change in the screen displayed on the smartphone can increase the convenience and accuracy.

In addition, it is possible to determine whether the user correctly recognizes the previous mark by outputting the currently displayed mark with a correlation with the previously output mark.

In addition, it is possible to accurately find out whether the gaze of the user's gaze and the direction of the gaze are directed to determine whether it matches the area currently displayed on the smartphone screen.

In addition, the image and text for vision measurement are continuously displayed on the smartphone at a time difference. The currently displayed image and text are displayed in association with the previously displayed image and text. It is easy to see if it is properly recognized.

1 is a view showing a general vision measurement principle.
2 is a configuration diagram schematically showing the overall configuration of the vision measurement system according to an embodiment of the present invention.
3 is a block diagram schematically illustrating an internal configuration of an eye vision measuring system according to an exemplary embodiment of the present invention.
4 is a diagram illustrating an example of measuring a distance between a smart phone and a user according to an exemplary embodiment of the present invention.
5 is a flowchart illustrating an operation of measuring a vision using a user's gaze movement according to an exemplary embodiment of the present invention.
6 is a diagram illustrating an example of continuously outputting a measurement marker on the screen of a smart phone according to the visual acuity value with a difference of 1 second in sequence.
7 is a diagram illustrating an example of scanning eye movement and a gaze direction according to an exemplary embodiment of the present invention.
8 illustrates an example of staring at a measurement mark corresponding to 0.5 of visual acuity moving on a screen according to an exemplary embodiment of the present invention.
9 is a diagram illustrating a gaze of a user's gaze when a measurement marker according to an exemplary embodiment of the present invention moves and displays a position according to a change of a vision stage.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present invention. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention.

In order to clearly describe the present invention, parts irrelevant to the description are omitted, and like reference numerals designate like elements throughout the specification.

Throughout the specification, when a part is "connected" to another part, this includes not only "directly connected" but also "electrically connected" with another element in between. . In addition, when a part is said to "include" a certain component, which means that it may further include other components, except to exclude other components unless otherwise stated.

When a portion is referred to as being "above" another portion, it may be just above the other portion or may be accompanied by another portion in between. In contrast, when a part is mentioned as "directly above" another part, no other part is involved between them.

Terms such as first, second, and third are used to describe various parts, components, regions, layers, and / or sections, but are not limited to these. These terms are only used to distinguish one part, component, region, layer or section from another part, component, region, layer or section. Accordingly, the first portion, component, region, layer or section described below may be referred to as the second portion, component, region, layer or section without departing from the scope of the invention.

The terminology used herein is for reference only to specific embodiments and is not intended to limit the invention. As used herein, the singular forms “a,” “an,” and “the” include plural forms as well, unless the phrases clearly indicate the opposite. As used herein, the meaning of "comprising" embodies a particular characteristic, region, integer, step, operation, element and / or component, and the presence of other characteristics, region, integer, step, operation, element and / or component It does not exclude the addition.

Terms indicating relative space such as "below" and "above" may be used to more easily explain the relationship of one part to another part shown in the drawings. These terms are intended to include other meanings or operations of the device in use with the meanings intended in the figures. For example, turning the device in the figure upside down, some parts described as being "below" of the other parts are described as being "above" the other parts. Thus, the exemplary term "below" encompasses both up and down directions. The device can be rotated 90 degrees or at other angles, the terms representing relative space being interpreted accordingly.

Unless defined otherwise, all terms including technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. Commonly defined terms used are additionally interpreted to have a meaning consistent with the related technical literature and the presently disclosed contents, and are not interpreted in an ideal or very formal sense unless defined.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present invention. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention.

2 is a configuration diagram schematically showing the overall configuration for explaining a vision measurement method using a user's eye gaze motion according to an embodiment of the present invention.

Referring to FIG. 2, the method for measuring eyesight using a user's eye gaze motion according to an embodiment of the present invention is applied to a smart phone 100 carried by a user, and the eyesight measurement according to the present invention is performed in the smart phone 100. Application for application is installed, and the vision measurement can be started with the execution.

In an embodiment of the present invention, the vision measuring system using the user's gaze movement is applied to the smart phone 100. However, the present invention is not limited thereto, and any terminal provided with a camera that can be carried by the user and photographs the user's face can be used. It is possible.

The vision measuring system 100 using the user's gaze movement according to an exemplary embodiment of the present invention may be applied to, for example, a PDA terminal, a mobile phone, a tablet PC, a portable notebook, a portable camera, and the like in addition to a smart phone.

In the exemplary embodiment of the present invention, the vision measuring system 100 is applied to the smart phone 100 as an example, and both the vision measuring system 100 and the smart phone 100 will be described using the same reference numerals.

When the vision measurement system 100 is executed, the vision measurement system 100 may be displayed on the screen while moving with a time difference sequentially from the measurement marker corresponding to the low vision as shown in FIG. 2. have.

Here, the measurement marker may be implemented as an image and text corresponding to each visual acuity value.

At this time, the user gazes at the measurement marker displayed on the screen of the smart phone 100, and continues to stare while following the measurement marker corresponding to each visual acuity value displayed while moving sequentially according to the vision measurement step. I can keep it.

The smartphone 100 operates the front camera to capture the user's face and analyzes the user's face and main features in real time, and analyzes the movement of the user's eyes recognized by the camera in detail to measure the corresponding vision value. You can find out the user's awareness of.

The smart phone 100 may determine whether the user recognizes the UI (User Interface) of the smart phone screen by grasping the movement, stop, and movement of the user's eyes. The smartphone 100 may determine whether the user is recognized by displaying an image or text corresponding to the distance with the current user and a vision value to be measured on the screen.

In this case, the smartphone 100 continuously displays images and texts with a time difference in order to accurately determine whether the user is recognized, and displays the currently displayed images and texts with a correlation between the previously displayed images and texts. This allows you to determine if the user has correctly recognized the previous image and text.

The smart phone 100 measures whether the user's gaze is properly followed by a measurement mark of the corresponding visual acuity value, which starts with a low value of the visual acuity value initially and gradually increases the visual acuity value, and the visual acuity value is the current visual acuity of the user. If the value is higher than the user's eyes will not follow, the smart phone 100 will be able to check the eyesight value of the user based on this time point.

3 is a block diagram schematically illustrating an internal configuration of an eye vision measuring system according to an exemplary embodiment of the present invention.

1 to 3, the vision measuring system 100 according to an exemplary embodiment of the present invention may include a camera image input unit 110, an eye image extractor A and 120, a distance calculator B and 130, and a visual acuity. Each of the screen generating unit (D, 140), the gaze position extraction unit (C, 150), the gaze matching detection unit (E, 160), the screen display unit 170 and the like.

The camera image input unit 110 may include a camera and input a captured facial image by capturing a face of a user in real time through the camera. Here, the facial image may include a still image with no motion on the user's face, or may include a video including eye movement, facial movement, and movement of two eyes.

In addition, the camera image input unit 110 may input a facial image including the stop and movement of the two pupils obtained by photographing a user's face gazing at the measurement marker displayed with two eyes while moving on the screen display unit 170. Can be.

Here, the measurement marker may include an image or text having a size corresponding to each visual acuity value for determining a visual acuity value of the user. For example, the measurement marker may include an image of a bird, an airplane, a car, a butterfly, a fish, etc., having a size corresponding to each visual acuity value, and text such as numbers or letters as shown in FIG. 1.

The eye image extractor 120 may extract the position, area, and information of the eye from the facial image. That is, the eye image extractor 120 may extract the eye image or the pupil image of the user according to a feature distinguished by color difference by comparing each other in a pixel unit or pixel area unit with respect to the face image.

In addition, the eye image extractor 120 may extract the binocular distance based on the eye position extracted from the facial image (a-1).

That is, the eye image extractor 120 may derive a pixel value between the two eyes with respect to the extracted eyes and convert the same into a physical distance value. To this end, the eye image extractor 120 may further include a matching table in which the actual physical distance value per pixel is matched according to the distance ratio between the user and the smart phone. In addition, the matching table may store, as data, pixel values at which the measurement markers move per pixel of the user's eyes that correspond to the measurement markers moving on the screen according to the gaze of the user.

In addition, the eye image extractor 120 may extract a gaze direction of the left or right eye of the user based on the position of the eye in the facial image in which the user's pupil moves along the measurement marker on the screen (a-2). .

That is, the eye image extractor 120 extracts a gaze direction according to which direction the pupil moves based on the pupil image in a stopped state based on the eye image or the pupil image extracted from the facial image. Can be.

For example, depending on whether the pupil moves in the east, west, south, or north direction with respect to the origin at the center stop, or in the northeast, southeast, northwest, southwest direction, the left eye or The direction of the eye of the right eye can be extracted.

In addition, the eye image extractor 120 may determine whether the user's gaze is maintained based on the position of the eye in the extracted eye image or the pupil image (a-3).

That is, the eye image extracting unit 120 gazes at the front of the eye based on the eye image or the pupil image extracted from the facial image, and maintains the stopped state or in any one of east, west, south and north directions. It is possible to determine whether the user keeps his eyes, such as whether to maintain the moving state.

As shown in FIG. 4, the distance calculator 130 may calculate a distance between the smart phone 100 and the user based on a triangular ratio based on a binocular distance value between two eyes. 4 is a diagram illustrating an example of measuring a distance between a smart phone and a user according to an exemplary embodiment of the present invention. As shown in FIG. 4, the distance calculator 130 may calculate the distance between the smartphone 100 and the user based on the triangular ratio based on a physical distance value using a pixel value between two eyes. have.

That is, the smartphone 100 may detect the positions of the two pupils of the user by using the front camera, and measure pixel values between the two pupils within the camera resolution through the detected positions of the two pupils. Can be. In the present invention, using the fact that the distance between the eyes of a person maintains a constant value, the pixel value between the eyes is derived as a physical distance value and based on this, using the triangular ratio between the two eyes of the user and the smartphone 100 By measuring the actual distance between the user and the smartphone (100). Since the proportional expression in the case of using a triangular ratio is calculated on the premise that the actual user and the smartphone 100 are horizontal, errors due to the tilt of the smartphone 100 and the tilt of the user's eye may occur in the actual use environment. have. The error of the smartphone 100 may increase the accuracy of the distance between the user and the smartphone 100 by measuring the degree of inclination using a tilt sensor (not shown) and compensating for the inclination accordingly. In this case, the user distance may use mm or Cm.

The vision-specific screen generation unit 140 may generate an image or text of a screen corresponding to the visual acuity value corresponding to the distance to the user.

That is, the screen display unit 140 for each vision generates a screen displaying a measurement mark corresponding to the initial reference visual acuity value according to the measurement distance between the user and the smart phone 100, or for each visual acuity value for each measurement step. It is possible to create a screen displaying the measurement marker corresponding to. Here, the initial reference visual acuity value may use a previously measured visual acuity value or may use a default visual acuity value set by default.

At this time, the image or text on the screen that matches the visual acuity value can be generated according to the distance-based discern resolution of the standard EN ISO 8596: 2017 (6/6 acuity, 20/20 acuity).

In addition, the screen display unit 140 for each visual acuity may generate a screen for continuously outputting the measurement markers corresponding to each visual acuity value sequentially with a time difference depending on the visual acuity value.

In addition, the visual display unit 140 for each visual acuity starts with a low level of visual acuity at first, and gradually increases the visual acuity value for each step, and sequentially measures a measurement marker corresponding to each visual acuity value with a time difference according to the visual acuity value. You can create a screen that outputs continuously while moving.

In addition, the screen display unit 140 for each visual acuity may transmit a measurement mark corresponding to each visual acuity value, that is, a coordinate value and a size value of an image or text, to the gaze matching detection unit 160.

The gaze position extractor 150 may extract eye movement and gaze direction. That is, the gaze position extractor 150 may extract the gaze position based on the gaze direction and gaze maintenance information of the left or right eye extracted from the eye image extractor 120.

The gaze position extracting unit 150 is a pixel where the gaze of the current user stays in the facial image where the pupil moves based on the origin, and stops without moving after the pupil moves according to the gaze of the measurement marker of the user. The eye gaze position can be extracted in terms of units. In this case, the extracted gaze position may include X and Y coordinate values of a pupil and a size of a pixel area occupied by the pupil. Therefore, the gaze position extractor 150 may transmit the gaze information value defined by the extracted X and Y coordinate values of the pupil and the size value of the region to the gaze matching detector 160.

The gaze position extracting unit 150 uses a new card (Saccades) method that distinguishes the fixation and movement of the gaze, and a smooth pursuit method that follows the movement of an object or the screen, and moves and stops and moves the user's gaze. The direction can be extracted.

The gaze matching detector 160 may detect whether the movement position of the measurement mark matches the eye position of the pupil.

That is, the gaze matching detector 160 receives a coordinate value and a size value of a moving position of a measurement marker corresponding to each visual acuity value received from the visual acuity screen generation unit 140, and the visual gaze position extracting unit 150. Coordinate values and size values of eye gaze positions may be compared with each other to detect a match.

The smartphone 100 may check whether the user is currently viewing the vision measurement screen according to the matching detection result of the gaze matching detection unit 160. That is, when the user gazes at the screen, when the measurement marker moving on the screen is visible, the pupil is continuously moved along the measurement marker, which corresponds to a case where the movement position of the measurement marker is matched with the eye position of the pupil.

The screen display unit 170 may display and output a screen to be displayed while moving the measurement mark corresponding to each visual acuity value according to the distance from the user.

The visual acuity determining unit 180 may determine the visual acuity value according to whether the movement position of the measurement marker matches the eye gaze position of the pupil.

In other words, when the measurement marker is moved while displaying the size of the vision by gradually increasing the size of the vision from low to high vision, when the user's eyesight and the eyesight of the measurement markers are the same, the user's eyes continuously move along the measurement markers. In other words, the movement position of the measurement marker and the eye position of the pupil are identically matched, and if the user's eyesight and the eyesight of the measurement marker are different, the measurement marker is not visible to the user's eyes and the pupil cannot follow the movement of the measurement marker. The position of the marker's movement and the pupil's line of sight do not match.

Therefore, when the moving position of the measurement mark and the eye gaze position of the pupil do not match, the visual acuity determiner 180 may determine the visual acuity value of the measurement mark which has been previously matched with the eye gaze position of the user as the visual acuity value of the current user. have.

And, although not shown in Figure 3, the vision measurement system 100 using the user's eye gaze motion according to an embodiment of the present invention, a separate measurement for measuring the distance between the user and the smartphone to measure the vision using infrared or ultrasonic It may include a distance measuring sensor.

5 is a flowchart illustrating an operation of measuring a vision using a user's gaze movement according to an exemplary embodiment of the present invention.

1 to 5, in the vision measuring system 100 of the present invention, first, the camera image input unit 110 may input a face image captured by capturing a user's face (S510).

Here, the facial image may include images of the user's face, two eyes and two eyes, etc. Since the image is obtained by first photographing the user's face for eyesight measurement, the facial image may include a still image without movement of both eyes. Can be.

Subsequently, the eye image extractor 120 may extract the position of the eye from the facial image, and extract a binocular distance value based on the position of the eye (S520).

That is, the eye image extractor 120 compares each other in a pixel unit or a pixel region unit with respect to the facial image, and extracts the positions of two eyes separated by color differences as X, Y coordinate values and region size values.

In addition, the eye image extractor 120 may determine the distance between the two eyes, that is, the X and Y coordinate values of the left eye and the X and Y coordinate values of the right eye, based on the X and Y coordinate values of the left eye and the X and Y coordinate values of the right eye. The binocular distance value according to the difference can be extracted.

In addition, the eye image extractor 120 may extract a gaze direction of the left or right eye of the user based on the position of the eye, and determine whether the user maintains the gaze based on the position of the eye.

Subsequently, the distance calculator 130 may calculate a distance value between the smart phone 100 and the user based on the triangular ratio based on the binocular distance value (S530).

That is, the distance calculating unit 130 is based on the pixel value between the two pupils in the camera resolution through the position of the two pupils detected using the camera as shown in FIG. The actual distance between the 100 can be measured. The proportional expression in the case of using the triangle ratio can be calculated on the premise that the actual user and the smart phone 100 is horizontal.

Subsequently, the vision screen generation unit 140 may generate a screen displaying a measurement marker for each vision (image or text on the screen corresponding to the visual acuity value) corresponding to the distance to the user and output the screen to the screen display unit 170. (S540).

That is, the screen display unit 140 for each vision generates a screen displaying a measurement mark corresponding to the initial reference visual acuity value according to the measurement distance between the user and the smart phone 100, or for each visual acuity value for each measurement step. A screen displaying a measurement mark corresponding to the screen may be generated and output to the screen display unit 170.

" 이미지를 1초간 표시하고 사라지는 화면을 생성할 수 있다. 이어, 시력별 화면 생성부(140)는 표시 위치를 두번째 위치로부터 우측 아래로 이동하여 세번째 표시로 시력 0.3에 해당하는 "

" 이미지를 1초간 표시하고 사라지는 화면을 생성할 수 있다. 이어, 시력별 화면 생성부(140)는 표시 위치를 세번째 위치로부터 좌측 아래로 이동하여 네번째 표시로 시력 0.4에 해당하는 "2"라는 숫자를 1 초간 표시하고 사라지는 화면을 생성할 수 있다. 도 6의 경우에는 본 발명의 실시 예에 따른 스마트 폰의 화면 상에 측정 표식을 시력 값에 따라 순차적으로 1초간의 차이를 두고 이동하면서 연속적으로 출력하는 예를 나타낸 도면이다. For example, as shown in FIG. 6, the vision screen generating unit 140 displays a number “4” corresponding to 0.1 for visual acuity on the upper right of the screen for 1 second with respect to the measurement marker as shown in FIG. 6. Can be generated. Subsequently, the screen generating unit 140 according to the eyesight moves the display position from the first position to the lower left side, and corresponds to the eyesight 0.2 as the second display.

"The image may be displayed for 1 second and a disappearing screen may be generated. Then, the screen display unit 140 for each sight moves the display position from the second position to the lower right to correspond to the vision 0.3 as the third display.

"The image may be displayed for 1 second and a disappearing screen may be generated. Then, the screen display unit 140 for each sight moves the display position from the third position to the lower left, and the number" 2 "corresponding to the vision 0.4 as the fourth display. 6 and generate a disappearing screen In the case of Fig. 6, the measurement marker is continuously moved with a difference of 1 second sequentially according to the visual acuity value on the screen of the smart phone according to the embodiment of the present invention. It is a figure which shows the example which outputs.

In this case, the display interval of the measurement marker is illustrated as an interval of 1 second in FIG. 6, but the present invention is not limited thereto. The display may be output at 0.5 second intervals or at 1.5 second intervals.

In addition, although the display position of the measurement marker is also displayed alternately left and right in FIG. 6, the present invention is not limited thereto, but the display position of the measurement marker is displayed on the display position set by the user through menu setting or by a program or a screen generation unit for each sight. 140 may be set to display the display position of the measurement marker while changing or specifying.

Accordingly, the screen display unit 170 displays and outputs a screen that sequentially displays measurement markers while changing and moving a position with a predetermined time difference for each visual acuity.

In this case, the screen display unit 170 starts with a low value of the visual acuity value as initially 0.1 according to the operation of the visual acuity measuring application or the screen-specific screen generation unit 140, and gradually increases the visual acuity value for each step. As shown in FIG. 1, the measurement markers corresponding to each visual acuity value may be moved and displayed on the screen continuously with a time difference sequentially according to the visual acuity value.

In addition, although the screen display unit 170 changed the interval for the visual acuity value in FIG. 4 to display a measurement mark corresponding thereto, the screen display unit 170 is not limited thereto. For example, the measurement marker may be displayed by changing the unit by 0.3, or the measurement marker may be displayed by variously changing the interval 0.1, 0.3, 0.4, etc. according to the random number algorithm.

At this time, the user gazes at the measurement mark displayed on the screen display unit 170 of the smart phone 100 spaced apart from the user, and the two eyes move the same along the movement of the measurement mark whenever the measurement mark is moved. As you progress, you will continue to watch the measurement markers.

Subsequently, the camera image input unit 110 may input a face image photographing a face of a user moving eyes according to a measurement marker for each eye moving on the screen (S550).

Here, the facial image input through the camera image input unit 110 is an operation image such as a user's face and a stop and movement of two eyes, and may include an image obtained by synthesizing a still image of each frame unit. It may include a real-time video image that changes the motion of the two eyes according to the flow.

In addition, the facial image may include still and motion images of two eyes of a user staring while following a measurement mark displayed while moving with a time difference on the screen of the screen display unit 170.

Subsequently, the gaze position extractor 150 may extract the gaze position of the pupil according to the eye movement and the gaze direction from the facial image (S560).

That is, the gaze position extractor 150 scans the movement and the gaze direction of the pupil of the user as shown in FIG. 7 in response to the stop and the movement of the measurement marker displayed on the screen display unit 1170 according to the visual acuity value. ) can do. 7 is a diagram illustrating an example of scanning eye movement and a gaze direction according to an exemplary embodiment of the present invention. As shown in FIG. 7, it can be seen that all of the pupils of the user are actively moved when the measurement markers corresponding to 0.5, 0.8, 1.0 and 1.2 are displayed. That is, since the corresponding measurement mark is clearly visible to the eyes of the user, the eye movement is actively performed from the measurement mark corresponding to visual acuity 0.5 to the measurement mark corresponding to visual acuity 1.2 while maintaining the gaze state.

At this time, the gaze position extraction unit 150 uses a new card (Saccades) method for discriminating the fixation and movement of the gaze, and a smooth pursuit method that follows the movement of the object or the screen, The direction of stop and movement can be extracted.

The user looks at the measurement marker displayed as shown in FIG. 8 while changing the position and changing the position according to the visual acuity value with time difference on the screen display unit 170, and the pupil is also the same along the moving position of the measurement marker. Moved. 8 illustrates an example of staring at a measurement mark corresponding to 0.5 of visual acuity moving on a screen according to an exemplary embodiment of the present invention.

Subsequently, the gaze matching detector 160 may detect whether the movement position of the measurement marker and the eye gaze position of the pupil match (S570).

That is, the gaze matching detector 160 may detect a match by comparing the coordinate value and the size value of the moving position of the measurement marker with the coordinate value and the size value of the gaze position.

At this time, the gaze matching detection unit 160 has a ratio of the movement position and direction of the measurement marker to the movement position and direction of the pupil based on a matching table in which the pixel value at which the measurement marker is moved per pixel of the user's eyes is stored. Can be detected.

" 측정 표식을 응시하다가 다음 시력 단계로 전환되어 시력 1.1에 해당하는 "

" 측정 표식을 응시하게 되는 경우를 예로 들 수 있다. 도 9는 본 발명의 실시 예에 따른 측정 표식이 시력 단계의 전환에 따라 위치를 이동하여 디스플레이할 때 사용자의 시선 응시를 나타낸 도면이다. 도 9에서, 시력 전환에 따라 측정 표식은 시력 1.0에 해당하는 "

" 측정 표식의 X, Y 좌표가 (300, 50)이고, 다음에 디스플레이되는 시력 1.1에 해당하는 "

" 측정 표식의 X, Y 좌표가 (30, 400)인 경우, X 좌표가 좌측(-) 방향으로 10배로 이동했고, Y 좌표가 아래(+) 방향으로 8 배로 이동했음을 알 수 있다. 이 때, 시력 1.0에 해당하는 "

" 측정 표식을 응시하는 눈동자의 X, Y 좌표(10, 1)를 기준으로 시력 1.1에 해당하는 "

" 측정 표식을 응시하는 눈동자의 X, Y 좌표가 (1, 8)로 이동한 경우에, 시선 매칭 검출부(160)는 측정 표식을 응시하는 눈동자의 X 좌표가 좌측 방향으로 10배 이동하고, 눈동자 Y 좌표가 아래 방향으로 8배로 이동한 것으로 인식하여, 측정 표식의 이동 위치와 눈동자의 시선 위치가 매칭되는 상태로 검출할 수 있다.For example, in the matching table, the moving position of the measurement marker per pixel of the eye is matched to 50 pixels, and the user corresponds to the visual acuity of 1.0 as shown in FIG. 9 on the screen.

"Stare at the measurement marker and switch to the next stage of vision, corresponding to 1.1

For example, the gaze of the measurement marker may be viewed. FIG. 9 illustrates a gaze of the user when the measurement marker according to an exemplary embodiment of the present invention moves and displays a position according to a change of vision stage. FIG. At 9, the marker measured according to visual acuity conversion corresponds to visual acuity 1.0 "

"The X, Y coordinates of the measurement markers are (300, 50) and correspond to the visual acuity 1.1 displayed next"

"If the X, Y coordinates of the measurement marker are (30, 400), then the X coordinate has moved 10 times in the left (-) direction and the Y coordinate has moved 8 times in the down (+) direction. , Equivalent to eyesight 1.0

"Corresponds to a visual acuity of 1.1 based on the X, Y coordinates (10, 1) of the pupil gazing at the measurement marker"

"When the X and Y coordinates of the pupil gazing at the measurement marker are moved to (1, 8), the gaze matching detector 160 moves the X coordinate of the pupil gazing at the measurement marker to the left by 10 times, and the pupil By recognizing that the Y coordinate has moved 8 times in the downward direction, it is possible to detect the state where the movement position of the measurement marker matches the eye position of the pupil.

That is, when the user gazes at the screen area where the measurement mark is changed, the gaze matching detection unit 160 recognizes that the screen of the current measurement mark is recognized, and the vision-specific screen generation unit 140 increases the size of the eyesight. As shown in FIG. 9, a screen of the next measurement marker may be generated at a position different from the previous screen, and displayed on the screen display unit 170.

When the screen of the measurement marker is changed, the gaze position extractor 150 may again measure the gaze position of the user and check whether the gaze is changed to the gaze position of the screen. This screen may display a screen including information that is not related to information linked to the previous screen. When the user gazes at the information linked with the previous screen normally, the user recognizes the contents of the previous screen normally, and if the user does not gaze, it does not recognize it.

Subsequently, the visual acuity determiner 180 may determine the visual acuity value of the user according to whether the moving position of the measurement marker matches the eye gaze position of the pupil (S580).

That is, the visual acuity determination unit 180 may determine the visual acuity value of the measurement marker that was previously matched with the eye gaze position of the pupil when the moving position of the measurement marker and the eye gaze position of the pupil do not match. .

" 측정 표식은 응시하고, 측정 표식이 전환되어 시력 1.1에 해당하는 "

" 측정 표식을 응시하지 못한 경우에, 시력 1.1에 대한 측정 표식의 이동 위치와 눈동자의 시선 위치가 매칭되지 않으므로, 바로 이전에 눈동자의 시선 위치와 매칭되었던 1.0 측정 표식의 시력 값을 사용자의 시력 값으로 판단할 수 있다.For example, in FIG. 9, the pupil of the user moves in the same direction along the movement of the measurement marker, and corresponds to the visual acuity of 1.0.

"The measurement marker is staring and the measurement marker is switched to correspond to 1.1 eyesight."

"If the eye was not staring at the measurement marker, the eye position of the 1.0 measurement marker that was previously matched to the eye position of the pupil was not matched since the movement position of the measurement marker for visual acuity 1.1 did not match. Judging by

As described above, the user continues to stare at the measurement marker displayed on the display screen unit 170 of the smart phone 100, changes the visual acuity value of the measurement marker at each step, and repeats the series of processes. The visual acuity value can be measured and confirmed.

The smart phone 100 of the present invention can measure the user's vision degree by checking the gaze of the user chasing this screen to the screen change.

In addition, the smart phone 100 of the present invention can clearly distinguish the change of the gaze and gaze of the user from the previous screen, by creating a correlation between the previous screen and the current screen, the user was surely recognized the previous screen. Can be judged.

In addition, the smart phone 100 of the present invention can measure the visual acuity without the user's input through the user's reaction detection. That is, it is possible to accurately find out whether the gaze of the user's gaze and the direction of the gaze are directed to determine whether it matches the area currently displayed on the screen of the smart phone 100.

As described above, according to the present invention, by simply displaying a marker for measuring the vision on the screen of the smart phone that the user carries, and following the measurement marker that the user's eyes move on the screen of the smart phone without the user's input, A vision measuring method and system using a user's gaze movement to measure vision can be realized.

As those skilled in the art to which the present invention pertains may implement the present invention in other specific forms without changing the technical spirit or essential features, the embodiments described above should be understood as illustrative and not restrictive in all aspects. Should be. The scope of the present invention is shown by the following claims rather than the detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalents should be construed as being included in the scope of the present invention. do.

100: vision measurement system
110: camera image input unit
120: eye image extraction unit
130: distance calculation unit
140: gaze position extraction unit
150: screen generation unit for each sight
160: line matching detection unit
170: screen display unit

Claims (19)

In an eye measurement system using the user's eye movements, the user displays the eye measurement markers, so that the user's eyes can naturally measure the user's eyes just by following the measurement markers,
A camera image input unit configured to input a face image captured by capturing a user's face;
An eye image extractor extracting a position, an area, and information of an eye from the face image, and extracting a binocular distance value based on the position of the eye;
A distance calculator configured to calculate a distance to the user based on the binocular distance value;
A vision-specific screen generation unit configured to generate a screen to be displayed while moving a measurement mark corresponding to each visual acuity value according to a distance from the user;
A screen display unit which displays and displays the generated screen;
A gaze position extraction unit for extracting a gaze position of a pupil moving along the moving measurement mark;
A gaze matching detector configured to detect whether a movement position of the measurement mark matches a gaze position of the pupil; And
A visual acuity determiner configured to determine a visual acuity value according to whether a movement position of the measurement mark is matched with a gaze position of the pupil;
Vision measurement system using user eye movement.
The method of claim 1,
The measurement marker may include an image or text having a size corresponding to each visual acuity value for determining a visual acuity value of the user.
Vision measurement system using user eye movement.
The method of claim 1,
The eye image extracting unit extracts a gaze direction of the left or right eye of the user based on the position of the eye,
Vision measurement system using user eye movement.
The method of claim 1,
The eye image extracting unit determines whether to maintain the user's gaze based on the position of the eye,
Vision measurement system using user eye movement.
The method of claim 1,
The distance calculator calculates a distance from the user based on a triangular ratio based on a binocular distance value using a pixel value between two eyes,
Vision measurement system using user eye movement.
The method of claim 1,
The screen generation unit for each visual acuity generates a screen for displaying a measurement mark corresponding to an initial reference visual acuity value according to the calculated distance from the user, or displays a screen for displaying a measurement mark corresponding to each visual acuity value for each measurement step. Produced,
Vision measurement system using user eye movement.
The method of claim 6,
The screen generation unit for each visual acuity generates a screen for continuously outputting a measurement mark corresponding to each visual acuity value while sequentially moving with a time difference according to the visual acuity value.
Vision measurement system using user eye movement.
The method of claim 6,
The screen generation unit for each sight generates a screen for continuously outputting the measurement markers corresponding to each sight value sequentially with a time difference according to the sight value while gradually starting to lower the sight value and gradually increasing the sight value. doing,
Vision measurement system using user eye movement.
The method of claim 1,
The gaze matching detector detects a match by comparing a coordinate value and a size value of a moving position of the measurement marker with a coordinate value and a size value of the gaze position.
Vision measurement system using user eye movement.
The method of claim 1,
The visual acuity determining unit may determine a visual acuity value of a measurement mark that was previously matched with a gaze position of an eye as the visual acuity value of the user when the movement position of the measurement mark and the eye position of the eye do not match.
Vision measurement system using user eye movement.
In the eye measurement method using the user's eye movements, the user to display the eye measurement markers, so that the user's eyes can naturally measure the user's eyes just by following the measurement markers,
Inputting, by a camera image input unit, a face image captured by capturing a face of a user;
An eye image extracting unit, extracting a position of an eye from the face image and extracting a binocular distance value based on the position of the eye;
Calculating, by a distance calculator, a distance value with the user based on the binocular distance value;
Generating a screen for displaying a measurement marker for each sight corresponding to a distance value with the user;
Inputting, by a camera image input unit, a face image photographing a face of a user moving an eye according to a vision-specific measurement mark moving on the screen;
A gaze position extracting unit, extracting gaze positions of pupils according to eye movements and gaze directions from the facial image;
Detecting, by the line of sight matching detector, whether a movement position of the measurement mark and a line of sight of the pupil match; And
Determining, by the visual acuity determiner, the visual acuity value of the user according to whether the movement position of the measurement mark matches the eye position of the eye;
Vision measurement method using a user's gaze movement comprising a.
The method of claim 11,
Extracting the binocular distance value,
The eye image extracting unit extracts a gaze direction of the left or right eye of the user based on the position of the eye,
Vision measurement method using the user's eye movement.
The method of claim 11,
Extracting the binocular distance value,
The eye image extractor determines whether to maintain the user's gaze based on the position of the eye,
Vision measurement method using the user's eye movement.
The method of claim 11,
Computing the distance value with the user,
The distance calculator calculates a distance from the user based on a triangular ratio based on a binocular distance value using a pixel value between two eyes,
Vision measurement method using the user's eye movement.
The method of claim 11,
Generating the screen to be displayed,
The screen generation unit for each sight generates a screen displaying a measurement mark corresponding to an initial reference vision value according to the calculated distance from the user, or a screen for displaying a measurement mark corresponding to each vision value for each measurement step. doing,
Vision measurement method using the user's eye movement.
The method of claim 15,
Generating the screen to be displayed,
The screen generation unit for each sight generates a screen for continuously outputting a measurement mark corresponding to each sight value sequentially moving with a time difference according to the sight value,
Vision measurement method using the user's eye movement.
The method of claim 15,
Generating the screen to be displayed,
The screen generation unit for each sight generates a screen that continuously outputs the measurement markers corresponding to each sight value sequentially with a time difference according to the sight value while gradually starting to lower the sight value and gradually increasing the sight value. doing,
Vision measurement method using the user's eye movement.
The method of claim 11,
Detecting whether the eye gaze position of the pupil is matched,
The gaze matching detection unit detects a match by comparing a coordinate value and a size value with respect to the movement position of the measurement mark and a coordinate value and a size value with respect to the gaze position.
Vision measurement method using the user's eye movement.
The method of claim 11,
Determining the visual acuity value of the user,
When the visual acuity determination unit does not match the movement position of the measurement marker and the eye gaze position of the pupil, the visual acuity value of the measurement marker that was matched with the eye gaze position of the eye immediately before, as the eyesight value of the user,
Vision measurement method using the user's eye movement.

Images