KR20200063789A - Ar glass and method of providing augmented reality service using the same - Google Patents

Abstract

Disclosed are an AR glass and an augmented reality service providing method using the same. The AR glass comprises: a vision correction device for correcting the vision of a user in accordance with the vision of the user input from the outside; a communication device receiving data corresponding to a specialized service which can be provided to the user; and a service providing device providing the received specialized service as an augmented reality (AR) service in response to the vision of the user which is corrected.

Description

AR glass and augmented reality service using it {AR GLASS AND METHOD OF PROVIDING AUGMENTED REALITY SERVICE USING THE SAME}

The present invention relates to AR glass and a method for providing augmented reality service using the same, and relates to a technology for providing augmented reality service based on a user's eyesight corrected through a vision correction device.

Recently, a demand for an augmented reality device or a mixed reality device that provides various types of information by overlaying a virtual image on a real-world image or background is rapidly increasing. Demand for such augmented reality devices or mixed reality devices continues to increase not only in the entertainment area, but also in special business and vehicle devices.

In particular, in the case of augmented reality devices for vehicles, the technology provided by matching the information provided by the vehicle to the actual image in front of the driver's field of view to minimize the driver's physical cognitive load and achieve driver safety and convenience The situation is developing around HUD (Head Up Display).

In the case of such a HUD, there is an advantage that information can be transmitted in the same position as the driver's field of view, but the size of the display panel is small, so there is a limitation in realizing various information and realizing augmented reality. .

Accordingly, it is necessary to develop a wearable augmented reality device or a mixed reality device that can solve the inconvenience of the driver's reaction force or lack of visibility caused by physical conditions and provide driving conditions, driving directions, and surrounding information through various UI/UX. Situation.

The present invention can provide an apparatus and method for providing an augmented reality service based on a user's eyesight corrected through a vision correction device.

In addition, the present invention can provide an apparatus and method for displaying a virtual image at the same depth location as a real object.

AR glass according to an embodiment of the present invention comprises an eye correction device for correcting the user's eyesight according to the user's eyesight input from the outside; A communication device receiving data corresponding to a specialized service available to the user; And a service providing device that provides the received specialized service as an Augmented Reality (AR) service in response to the corrected user's eyesight.

The vision correction device may correct the user's eyesight by adjusting the refractive index of a variable Fresnel lens according to the user's eyesight input from the outside.

The service providing device tracks the user's gaze or head direction through a sensor disposed on the AR glass, and the image data of a blind spot received through the communication device is aligned with the tracked user's gaze or head direction. It can be provided in conjunction.

The service providing apparatus may provide image data of the blind spot by matching the first coordinates according to the user's gaze or head direction and the second coordinates according to the received blind spot image data.

When the illuminance and visibility of the front region corresponding to the user's field of view are below a certain level, the service providing apparatus may provide improved image data through an image processing process.

The service providing device may provide the augmented reality service to the user by tracking the pupil of the user based on the user's eyesight corrected by the vision correction device and outputting a virtual image through the pupil of the tracked user. .

A method of providing an augmented reality (AR) service using an AR glass according to an embodiment of the present invention comprises the steps of correcting the user's eyesight according to the user's eyesight input from the outside; The method may include receiving data corresponding to a specialized service available to the user and providing the received specialized service as an augmented reality service in response to the corrected user's eyesight.

The step of correcting the eyesight may correct the eyesight of the user by adjusting the refractive index of a variable Fresnel lens according to the user's eyesight input from the outside.

The providing of the augmented reality service may include tracking the user's gaze or head direction through a sensor disposed on the AR glass; And providing image data of a blind spot received through the communication device in association with the tracked user's gaze or head direction.

The providing of the augmented reality service provides image data of the blind spot to the user by matching first coordinates according to the user's gaze or head direction and second coordinates according to the received blind spot image data. can do.

In the step of providing the augmented reality service, when the illuminance and visibility of the front region corresponding to the user's field of view are below a certain level, improved image data may be provided through an image processing process.

The providing of the augmented reality service may include tracking the pupil of the user based on the eyesight of the user corrected by the vision correction device; And providing an augmented reality service to the user by outputting a virtual image to the tracked user's pupil.

According to an embodiment of the present invention, augmented reality services with clean image quality may be provided by correcting a user's eyesight through a vision correction device.

In addition, according to an embodiment of the present invention, it is possible to increase the immersive feeling of the user by displaying the virtual image at the same depth position as the object of reality based on the eyesight of the user corrected through the vision correction device.

1 is a diagram illustrating a conceptual diagram for providing augmented reality service using an AR glass according to an embodiment of the present invention.
2 is a view showing the configuration of an AR glass according to an embodiment of the present invention.
FIG. 3 is a diagram illustrating an example of providing a first specialized service as an augmented reality service using an AR glass according to an embodiment of the present invention.
4 is a view showing an example of providing a second specialized service as an augmented reality service using an AR glass according to an embodiment of the present invention.
5 is a diagram illustrating an example of providing a third specialized service as an augmented reality service using an AR glass according to an embodiment of the present invention.
6 is a flowchart illustrating a method of providing an augmented reality service according to an embodiment of the present invention as a flowchart.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, various changes may be made to the embodiments, and the scope of the patent application right is not limited or limited by these embodiments. It should be understood that all modifications, equivalents, or substitutes for the embodiments are included in the scope of rights.

The terms used in the examples are used for illustrative purposes only and should not be construed as limiting. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, the terms "include" or "have" are intended to indicate the presence of features, numbers, steps, actions, components, parts or combinations thereof described herein, one or more other features. It should be understood that the existence or addition possibilities of fields or numbers, steps, operations, components, parts or combinations thereof are not excluded in advance.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person skilled in the art to which the embodiment belongs. Terms, such as those defined in a commonly used dictionary, should be interpreted as having meanings consistent with meanings in the context of related technologies, and should not be interpreted as ideal or excessively formal meanings unless explicitly defined in the present application. Does not.

In addition, in the description with reference to the accompanying drawings, the same reference numerals are assigned to the same components regardless of reference numerals, and redundant descriptions thereof will be omitted. In describing the embodiments, when it is determined that detailed descriptions of related known technologies may unnecessarily obscure the subject matter of the embodiments, detailed descriptions thereof will be omitted.

1 is a diagram illustrating a conceptual diagram for providing an augmented reality service using an AR glass according to an embodiment of the present invention.

For users who use glasses due to the existing near/native perspective, the scene in the real world may differ depending on the individual's eyesight, because the wearer must take off the glasses and wear an augmented reality device or a mixed reality device. Is generated. And these problems can be a stumbling block to the universal commercialization of augmented reality devices or mixed reality devices.

In addition, the virtual image displayed on the augmented reality device or the mixed reality device may be displayed as if it appears at the same depth location as the real thing to increase the immersion of the user. However, in the case of an existing augmented reality device or a mixed reality device, a problem occurs that a virtual image is displayed at a designated same depth position, thereby inhibiting the immersion of the user.

To solve this problem, the AR glass 100, which is the augmented reality device or the mixed reality device of the present invention, may include a function of correcting the user's eyesight according to the user's eyesight input from the outside. That is, when the user checks his/her eyesight and inputs the confirmed visual acuity value to the AR glass 100, the AR glass 100 adjusts the refractive index of the variable lens according to the inputted visual acuity value of the user. The user's vision can be corrected.

Through this, the user can be provided with a clean quality augmented reality service or a mixed reality service even if the AR glasses 100 is worn off the existing glasses.

In addition, the AR glass 100 of the present invention can increase the immersion of the user by displaying the virtual image at the same depth as the real object based on the corrected user's eyesight by adjusting the refractive index of the variable lens. That is, the AR glass 100 can adjust a user's focus on a real object through vision correction, and superimpose a virtual image on a real object corresponding to the adjusted focus, thereby providing a more realistic augmented reality service, thereby immersing the user. Can increase.

The AR glass 100 of the present invention can effectively provide various specialized services by correcting a user's eyesight and displaying a virtual image at the same depth as a real object.

As an example, the AR glass 100 is a safety service that provides vehicle driving information providing special service, night driving information, and fog removal, providing blind spot information and vehicle status information during driving/parking based on visual acuity correction of low vision. A user can be provided with a service for providing real-time surrounding information based on augmented reality data, including a service for providing driving information and a navigation function.

2 is a view showing the configuration of an AR glass according to an embodiment of the present invention.

Referring to FIG. 2, the AR glass 100 of the present invention may be composed of a vision correction device 110, a communication device 120, and a service providing device 130. Specifically, the vision correction device 110 may include a variable focus module based on a liquid crystal lens. For example, the variable focus module based on the liquid crystal lens may be configured with a variable Fresnel lens. Accordingly, when the eyesight is input from the user, the vision correction device 110 may change the focal length by adjusting the refractive power of the variable Fresnel lens according to the inputted vision.

)로 전체 초점거리 가변 범위를 6 Dptr로 구현할 수 있다. 이와 같이 세밀한 정밀도를 기반으로 시력 보정 장치(110)는 다양한 사용자들의 시력에 따라 가변 프레넬 렌즈의 굴절력을 조정함으로써 다양한 사용자들 각각에 대한 초점을 보다 뚜렷하게 보정할 수 있다.In one example, the vision correction device 110 is a step 0.25 Dptr (precision according to the refractive power of the variable Fresnel lens

), the entire focal length variable range can be implemented with 6 Dptr. Based on such precise precision, the vision correction device 110 may more accurately correct focus for each of various users by adjusting the refractive power of the variable Fresnel lens according to the vision of various users.

The communication device 120 may receive various data for providing a specialized service from a plurality of sensors mounted inside and outside the vehicle. At this time, the communication device 120 may use short-range wireless signals such as Bluetooth and Wi-Fi, as well as long-range wireless signals such as 3G, 4G, and 5G.

As an example, the communication device 120 may receive image data corresponding to a blind spot for providing a specialized service corresponding to blind spot information during driving/parking through a camera sensor mounted inside and outside the vehicle.

As another example, the communication device 120 may receive various vehicle information for providing specialized services corresponding to vehicle state information during driving/parking through vehicle sensors mounted on the inside and outside of the vehicle.

The service providing device 130 may provide various data related to a specialized service received through the communication device 120 as an augmented reality service in response to a user's eyesight corrected through the vision correcting device 110. At this time, the service providing device 130 may provide a more realistic augmented reality service by superimposing a weather image on a real object based on the adjusted focus in response to the corrected user's eyesight.

Specifically, the AR glass 100 of the present invention provides information on a blind area during driving/parking and status information of a vehicle based on vision correction of a low vision person (1) Special service for providing vehicle driving information, night driving information and fog removal It can provide users with (2) specialized services for providing safe driving information, and real-time services for providing real-time information based on existing navigation and augmented reality data (3).

3 is a view showing an example of providing a first specialized service as an augmented reality service using an AR glass according to an embodiment of the present invention.

Referring to FIG. 3, the AR glass 100 of the present invention may provide blind spot information and vehicle status information during driving/parking based on vision correction of a low vision person. For example, the AR glass 100 may receive image data corresponding to a blind spot through a camera sensor mounted on the outside of the vehicle.

For example, image data received through the camera sensor may include preset coordinate information. In addition, the AR glass 100 may track the user's gaze or head direction through various sensors disposed therein, and may extract coordinate information about the tracked user's gaze or head direction.

The AR glass 100 may provide blind spot information to the user by using the extracted coordinate information about the user's gaze or head direction and coordinate information preset in the image data received through the camera sensor.

That is, the AR glass 100 provides the user with a variety of information necessary for driving by providing the user's blind spot information, which is obscured by the door of the vehicle, through coordinate matching as shown in FIG. 3, in conjunction with image data received through the camera sensor. Can provide.

As another example, the AR glass 100 may receive location data related to an obstacle existing in a traveling direction of the vehicle through various distance sensors mounted on the inside and outside of the vehicle. For example, the AR glass 100 may receive position data of obstacles existing in the front and rear of the vehicle as well as the left and right sides of the vehicle through a radar sensor, a lidar sensor, etc. mounted on the inside and outside of the vehicle, and receive the position data of the received obstacle. The information necessary for driving can be provided by superimposing and displaying the user's field of view.

As another example, the AR glass 100 may receive current state information of the vehicle through various vehicle sensors mounted on the inside and outside of the vehicle. For example, the AR glass 100 may receive the current state of the tire air pressure through the tire air pressure sensor mounted on the vehicle, and if the tire air pressure becomes lower than appropriate, the warning signal is superimposed on the user's field of view to operate. Can provide the necessary information.

4 is a view showing an example of providing a second specialized service as an augmented reality service using an AR glass according to an embodiment of the present invention.

Referring to FIG. 4, the AR glass 100 of the present invention may provide night driving information and fog removal for safe driving of a user while driving. For example, when the illuminance of the front area corresponding to the user's field of view is below a certain level, the AR glass 100 operates at night by performing an image processing process on image data received through a camera sensor mounted in front of the vehicle. Information can be provided to the user.

That is, the AR glass 100 may assist the user to perform safe driving by processing the image data received as shown in FIG. 4(a) and highlighting and displaying a moving object in front.

As another example, when the visibility of the front area corresponding to the user's field of view is below a certain level, the AR glass 100 provides the user with driving information suitable for a road having a short viewing distance through an image processing process. Can provide

That is, the AR glass 100 may help the user to perform a safe operation by removing and displaying fog in the front through an image processing process as shown in FIG. 4(b).

5 is a diagram illustrating an example of providing a third specialized service as an augmented reality service using an AR glass according to an embodiment of the present invention.

Referring to FIG. 5, the AR glass 100 of the present invention may provide real-time surrounding information based on augmented reality data by including a navigation function. At this time, the AR glass 100 may provide a more realistic augmented reality service by correcting a user's eyesight and displaying a virtual image at the same depth as a real object.

To this end, the AR glass 100 tracks the pupil of the user based on the eyesight of the user corrected by the vision correction device 110 and separately provides an augmented reality service by directly outputting a virtual image to the tracked user's pupil It may include a video transmission module. That is, the AR glass 100 uses a variable lens for correcting a user's eyesight and a lens of an image transmission module that outputs a virtual image by tracking a user's pupil, so that a virtual image is superimposed on a real object to provide a more realistic augmented reality service. By providing, it is possible to increase the immersion of the user.

6 is a flowchart illustrating a method of providing an augmented reality service according to an embodiment of the present invention as a flowchart.

Referring to FIG. 6, in step 610, the AR glass 100 may correct the user's eyesight according to the user's eyesight input from the outside. That is, when the user checks his/her eyesight and inputs the confirmed vision value into the AR glass 100, the AR glass 100 adjusts the refractive index of the variable lens according to the inputted user's vision value. The user's vision can be corrected.

In step 620, the AR glass 100 may receive data corresponding to a specialized service that can be provided to a user. To this end, the AR glass 100 may receive various data for providing a specialized service from a plurality of sensors mounted inside and outside the vehicle. At this time, the AR glass 100 may use short-range wireless signals such as Bluetooth and Wi-Fi, as well as long-range wireless signals such as 3G, 4G, and 5G.

In step 630, the AR glass 100 may provide various data related to the specialized service as an augmented reality service in response to the corrected user's eyesight. At this time, the AR glass 100 may provide a more realistic augmented reality service by superimposing a weather image on a real object based on the adjusted focus in response to the corrected user's eyesight.

Specifically, the AR glass 100 of the present invention provides information on a blind area during driving/parking and status information of a vehicle based on vision correction of a low vision person (1) Special service for providing vehicle driving information, night driving information and fog removal It can provide users with (2) specialized services for providing safe driving information, and real-time services for providing real-time information based on existing navigation and augmented reality data (3).

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, or the like alone or in combination. The program instructions recorded in the medium may be specially designed and configured for the embodiments or may be known and usable by those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs, DVDs, and magnetic media such as floptical disks. -Hardware devices specifically configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language code that can be executed by a computer using an interpreter, etc., as well as machine language codes produced by a compiler. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

The software may include a computer program, code, instruction, or a combination of one or more of these, and configure the processing device to operate as desired, or process independently or collectively You can command the device. Software and/or data may be interpreted by a processing device, or to provide instructions or data to a processing device, of any type of machine, component, physical device, virtual equipment, computer storage medium or device. , Or may be permanently or temporarily embodied in the transmitted signal wave. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

As described above, although the embodiments have been described with limited drawings, those skilled in the art can apply various technical modifications and variations based on the above. For example, the described techniques are performed in a different order than the described method, and/or the components of the described system, structure, device, circuit, etc. are combined or combined in a different form from the described method, or other components Alternatively, even if replaced or substituted by equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

100: AR glass
110: vision correction device
120: communication device
130: service providing device

Claims (12)

A vision correction device that corrects the user's eyesight according to the user's eyesight input from the outside;
A communication device receiving data corresponding to a specialized service available to the user; And
A service providing device that provides the received specialized service as an Augmented Reality (AR) service in response to the corrected user's vision
AR glass comprising a.
According to claim 1,
The vision correction device,
AR glass that corrects the user's eyesight by adjusting the refractive index of a variable Fresnel lens according to the user's eyesight input from the outside.
According to claim 1,
The service providing device,
AR that tracks the user's gaze or head direction through a sensor disposed on the AR glass, and provides image data of a blind spot received through the communication device in association with the tracked user's gaze or head direction. Glass.
According to claim 3,
The service providing device,
AR glass providing image data of the blind spot by matching first coordinates according to the user's gaze or head direction and second coordinates according to the received blind spot image data.
According to claim 1,
The service providing device,
AR glass that provides improved image data through an image processing process when the illuminance and visibility of the front region corresponding to the user's field of view are below a certain level.
According to claim 1,
The service providing device,
AR glass that provides augmented reality service to the user by tracking the pupil of the user based on the eyesight of the user corrected by the vision correction device and outputting a virtual image to the pupil of the tracked user.
In the method of providing augmented reality (AR) service using AR glass,
Correcting the user's eyesight according to the user's eyesight input from the outside;
Receiving data corresponding to a specialized service available to the user, and
Providing the received specialized service as an augmented reality service in response to the corrected user's vision
Augmented reality service providing method comprising a.
The method of claim 7,
The step of correcting the eyesight,
Method for providing augmented reality service to correct the user's eyesight by adjusting the refractive index of a variable Fresnel lens according to the user's eyesight input from the outside.
The method of claim 7,
The step of providing the augmented reality service,
Tracking the direction of the user's gaze or head through a sensor disposed on the AR glass; And
Providing image data of a blind spot received through the communication device in association with the tracked user's gaze or head direction
Augmented reality service providing method comprising a.
The method of claim 9,
The step of providing the augmented reality service,
A method for providing augmented reality service by providing image data of the blind spot to the user by matching the first coordinate according to the user's gaze or head direction with the second coordinate according to the received blind spot image data.
The method of claim 7,
The step of providing the augmented reality service,
A method for providing augmented reality service that provides improved image data through an image processing process when the illuminance and visibility of a front region corresponding to the user's field of view are below a certain level.
The method of claim 7,
The step of providing the augmented reality service,
Tracking the pupil of the user based on the user's vision corrected by the vision correction device; And
Providing an augmented reality service to the user by outputting a virtual image through the pupil of the tracked user
Augmented reality service providing method comprising a.

Images