KR102655191B1 - Smart glass and controlling method tereof - Google Patents

Abstract

The present invention includes the steps of checking the size and resolution of the display mounted on smart glasses; Providing an enlarged AR screen using the optical device of the smart glasses; Running a 3D modeling application; Measuring the size of the AR screen and the distance from the smart glasses; and measuring the viewing angle based on the size and distance.

Description

Smart glasses and control method{SMART GLASS AND CONTROLLING METHOD TEREOF}

The present invention relates to glasses and a viewing angle measurement method. More specifically, the present invention provides smart glasses and a control method that allow a user to easily measure the viewing angle without a separate device.

Augmented reality (AR) is a field of virtual reality (VR) and is a computer graphics technique that synthesizes virtual objects or information into an actual environment to make them appear as if they existed in the original environment. Recently, various devices using AR platforms using augmented reality techniques have been developed.

VR (Virtual Reality)/AR (Augmented Reality) devices can use complex optical modules to implement enlarged virtual images on a display very close to the human eye.

In order to implement accurate images and control the screen using a controller, accurate information about the viewing angle of the AR screen is required.

Registration number 10-22401900, “Lens and optical system optical characteristics measurement system for VR or AR devices” This is about how to separate lenses, fix them to equipment, and measure them for VR/AR device lens and optical system optical characteristic measurement system. Although it is possible to precisely measure the characteristics of lenses for VR/AR, the lens must be separated from the AR/VR device and fixed to the facility, a large space is required for the facility, and the number of samples is small or the lens is attached to the device. If so, measurement is limited.

Registration number 10-1866291, “Optical measurement device and method for three-dimensional display,” relates to an optical measurement device and method for a 3D display that can measure the main viewing distance and viewing angle of a 3D display. An optical instrument equipped with a polarizing member is used to measure 3D display. Disclosed is a method of measuring viewing distance and viewing angle using an optical meter placed horizontally with the upper part of a display panel.

However, there is a problem in that physical space is needed to fix the display and install measuring instruments, and it is difficult to film the small display device used for AR/VR display and the content restored in the virtual space of the AR/VR device.

In the case of this measurement method, it is difficult to accurately measure the viewing angle of smart glasses with various resolutions, and the measurement value of the viewing angle may vary depending on the user's estimation. Additionally, it is difficult to measure the vertical viewing angle, horizontal viewing angle, and diagonal viewing angle, and the viewing angle value may vary depending on the content location of the smart glasses.

The purpose of the present invention is to provide smart glasses and a viewing angle measurement method that allow users to easily measure the viewing angle without a separate device.

Checking the display size and resolution mounted on smart glasses; Providing an enlarged AR screen using the optical device of the smart glasses; Running a 3D modeling application; Measuring the size of the AR screen and the distance from the smart glasses; and measuring the viewing angle based on the size and distance.

The step of measuring the size and the distance includes calculating coordinates of the smart glasses and coordinates of a vertex and center point of the AR screen; And it may include calculating the size and the distance through the difference between each coordinate.

In the step of calculating the coordinates, the coordinates of the smart glasses can be set as zero and the coordinates of the vertex and center point of the AR screen can be calculated.

In the coordinate calculation step, the position of the smart glasses in a space recognized through a stereo camera mounted on the smart glasses can be obtained, and the position of the AR screen can be calculated using the position of the smart glasses as a reference point.

The method may further include calculating pixels per angle based on the resolution of the display and the viewing angle.

If the size or location of the AR screen changes, the step of measuring the size and distance may be performed again.

A frame mounted on the user's head; A pair of front lenses located on the front of the frame; A display unit mounted on the frame; An optical module that magnifies the image output from the display unit; and a control unit that controls the AR screen to be output by controlling the display unit and the optical module, wherein the control unit measures the size of the AR screen and the distance to the AR screen by running a 3D modeling application, and measures the size of the AR screen. And it provides smart glasses, characterized in that the viewing angle is measured based on the distance.

The control unit may calculate the coordinates of the smart glasses and the vertex and center point of the AR screen, and calculate the size and distance through the difference between each coordinate.

It further includes a stereo camera that recognizes the surrounding space, and the control unit can obtain the position of the smart glasses in the space recognized through the stereo camera and calculate the position of the AR screen using the position of the smart glasses as a reference point. there is.

The control unit may calculate pixels per angle based on the resolution of the display and the viewing angle.

According to an embodiment of the present invention, the vertical viewing angle, horizontal viewing angle, and diagonal viewing angle of smart glasses including various resolutions and optical devices can be accurately measured using augmented reality content.

In addition, it can help measure the viewing angle of smart glasses regardless of location without using viewing angle measurement equipment that is bulky, expensive to build, and difficult to move.

In addition, after obtaining information about the viewing angle of the smart glasses, the size of the content can be adjusted to match the resolution of the smart glasses to provide the user with images of optimal quality.

Figure 1 is a block diagram showing the configuration of glass according to an embodiment of the present invention.
Figure 2 is a perspective view showing glass according to an embodiment of the present invention.
Figure 3 is a diagram illustrating a virtual reality screen visible on glasses according to an embodiment of the present invention.
Figure 4 is a diagram for explaining the principles of a glass display and an optical module according to an embodiment of the present invention.
Figure 5 is a flowchart explaining a method of measuring the viewing angle of glass according to an embodiment of the present invention.
Figure 6 is a diagram for explaining the execution of viewing angle measurement software according to an embodiment of the present invention.
7 to 9 are diagrams for explaining a viewing angle measurement method according to an embodiment of the present invention.

Hereinafter, embodiments disclosed in the present specification will be described in detail with reference to the attached drawings. However, identical or similar components will be assigned the same reference numbers regardless of reference numerals, and duplicate descriptions thereof will be omitted. The suffixes “module” and “part” for components used in the following description are given or used interchangeably only for the ease of preparing the specification, and do not have distinct meanings or roles in themselves.

Additionally, in describing the embodiments disclosed in this specification, if it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in this specification, the detailed descriptions will be omitted. In addition, the attached drawings are only for easy understanding of the embodiments disclosed in this specification, and the technical idea disclosed in this specification is not limited by the attached drawings, and all changes included in the spirit and technical scope of the present invention are not limited. , should be understood to include equivalents or substitutes.

Terms containing ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. When a component is said to be "connected" or "connected" to another component, it is understood that it may be directly connected to or connected to the other component, but that other components may exist in between. It should be.

On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between.

Singular expressions include plural expressions unless the context clearly dictates otherwise.

In this application, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

FIG. 1 is a block diagram showing the configuration of smart glasses 100 according to an embodiment of the present invention, and FIG. 2 is a perspective view showing the glasses 100 according to an embodiment of the present invention. Smart glasses 100 include a communication unit 110, a microphone 120, a sensing unit 130, a camera 140, a display 150, an optical module 155, a speaker 160, a control unit 180, and a power supply unit. It may include (190).

It can be connected to an external terminal 300 that provides image information to the smart glasses 100 through the communication unit 110, and can be connected to the controller 300 that controls the smart glasses 100. The communication unit 110 may include both wired communication and wireless communication.

In the case of wired communication, it may include a cable 111 connected to an external terminal 300 or a controller, or a connector into which the cable 111 is inserted. Referring to FIG. 2, external data can be received through the cable 111 connected to the leg portion 102 of the smart glasses 100, and the screen displayed on the smart glasses 100 can be controlled.

In the case of a wired connection, not only data but also power can be supplied through the cable 111, and the cable 111 can serve as the power supply unit 190. In the case of wired use, it may be inconvenient, but it has the advantage of making the smart glasses 100 lighter because there is no need to provide a separate battery for power supply.

When performing wireless communication, wireless signals can be transmitted and received with the external terminal 300 through the wireless communication module and antenna. Bluetooth™, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra Wideband), ZigBee, NFC (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Wireless communication can be supported using at least one of Direct and Wireless USB (Wireless Universal Serial Bus) technologies.

The myrophone 120 processes external acoustic signals into electrical voice data. The processed voice data can be transmitted through the communication unit 110 to the external terminal 300 that provides image information to the smart glasses 100. The sound signal collected by the microphone 120 can be used in various ways depending on the function being performed (or application program being executed).

Meanwhile, various noise removal algorithms can be implemented in the microphone 120 to remove noise generated in the process of receiving external sound signals, and external sound signals are collected from different locations to remove external noise. To do this, a plurality of microphones 120 may be provided.

The sensing unit 130 is a device that senses at least one of environmental information around the smart glasses 100 and status information of the smart glasses 100 themselves and generates a corresponding sensing signal. The control unit 180 can control the smart glasses 100 based on these sensing signals and drive AR content provided by the smart glasses 100 and perform data processing, functions, or operations of the application.

Examples of the sensing unit 130 may include a proximity sensor for detecting nearby objects, an illumination sensor for detecting surrounding brightness, and a touch sensor for detecting user input. It may further include a temperature sensor for detecting the state of the smart glasses 100 or a camera 140 for detecting the user's gaze.

The sensing unit 130, which detects the user's active movement, can serve as a user input unit and can control the smart glasses 100 by connecting to a separate controller 300 as a user input unit.

The camera 140 is a device for acquiring image information. It may include a front-facing camera 140 to capture images of the surroundings of the glasses 100 and obtain information about the surrounding environment therefrom, and may be located inside the smart glasses 100 to detect the movement of the user's eyes. Additional cameras may be included.

The display 150 is an ultra-small display device measuring less than 1 inch. In the case of a general liquid crystal display 150 or OLED, the pixel size is large, so it is difficult to apply it to the smart glass 100, and it is difficult to apply it to the smart glass 100, and it is difficult to apply it to the smart glass 100. (150) can be used. The silicon liquid crystal display device is a projection type display device that uses a silicon (Si) wafer instead of the glass at the bottom of the existing liquid crystal display (LCD) and forms an electronic circuit on it.

The surface of the silicon chip was painted with crystal, and the reflective surface of the electronic circuit that forms the video image was made of aluminum. Since this technology can be implemented from small screens to large screens, the range of applications can be very diverse.

Light entering from outside is reflected through the mirror and provides a screen to the user. At this time, by using a silicon wafer instead of the lower glass, it is possible to integrate circuits such as a drive IC and controller on the substrate, making it possible to miniaturize and lower the price of the LCD cell.

The optical module 155 is a device that provides images created on the ultra-small display 150 to the user's eyes. It may include a mirror or beam splitter to adjust the angle of light, or a lens to enlarge or reduce the image. .

The speaker 160 is a device that provides sound to the user and may include an earbud inserted into the user's ear. However, since the leg portion 102 of the glass 100 is in close contact with the user's head, a bone conduction speaker is installed at the portion in close contact. By mounting (160), sound can be provided to the user.

The control unit 180 can control each of the above-described components to provide AR content to the smart glasses 100. The control unit 180 detects the surroundings in the sensing unit 130.

The power supply unit 190 is a device that supplies power to drive each component. It can be equipped with a battery in the smart glasses 100 and run wirelessly, or it can omit the battery and use an external terminal 300 or an auxiliary battery by wire. Power can be supplied through.

The former has the advantage of being easy to use because there are no wires, while the latter has the advantage of being lightweight. When receiving data by wire, it is advantageous to select the latter method and receive power by wire, and when receiving data wirelessly, it is advantageous to have its own power supply unit 190.

Referring to FIG. 2, the smart glasses 100 of the present invention are a glasses-type electronic device, include frames 101 and 102 for mounting on the face, and are displayed on the front of the user's face through a transparent front lens 105. You can watch the image output from the display 150 at the same time.

The frames 101 and 102 may be equipped with a speaker 160, a microphone, a control unit 180, a sensing unit 130, etc., and a camera 140 may be located between the front lenses 105 on both sides. The frames 101 and 102 may include a front part 101 where the front lens 105 is located and a leg part 102 that hangs over the ear, and a cable 111 connected to the external terminal 300 is attached to the leg part 102. ) can be located.

FIG. 3 is a diagram illustrating a virtual reality screen visible on the glass 100 according to an embodiment of the present invention. As shown in (a), the user's front is visible through the glass 100 and a screen is displayed overlapping it. You can adjust the transparency of the screen so that the user's front side is visible through the screen, and you can adjust the size of the screen. Multitasking is also possible by opening multiple screens, as shown in (b).

It differs from VR devices in that the screen is viewed while the front is visible, and the screen can move or display a different screen by detecting the movement of the user's eyes.

FIG. 4 is a diagram for explaining the principles of the display 150 and the optical module 155 of the glass 100 according to an embodiment of the present invention. As shown in FIG. 4, the image output from the micro display 150 of 1 inch or less is small in size and is enlarged through the optical module 155.

The light emitted from the light source 151 is provided to a polarizing beam splitter (1552, PBS) through the fly eye lens 1551. The polarizing beam splitter (1552) is an optical device that transmits the P-wave component of the light source and reflects the S-wave component. The light provided from the light source passes through and is provided to a liquid crystal display (Liquid Crystal on Silicon).

The silicon liquid crystal display 150 is a display 150 device that synthesizes an image by reflecting a light source. The light reflected from the silicon liquid crystal display 150 is synthesized into image information and supplied back to the polarizing beam splitter 1552. .

The waveform of the light reflected from the silicon liquid crystal display device 150 changes and is reflected from the polarizing beam splitter 1552. Light containing image information is reflected and expanded through the lens system 1554 and supplied to the user's eyes, allowing the user to view the image superimposed on the front glass 100.

In this way, the light emitted from the light source 151 is synthesized in the silicon liquid crystal display device 150, expanded through the optical module 155 (1551, 1552, 1553, and 1554), and delivered to the user, thereby creating dozens of TVs of similar size. You can enjoy AR content on a large inch screen.

At this time, when the display 150 screen of the smart glasses 100 is enlarged using the optical module 155, the viewing angle cannot be increased beyond a certain value due to the physical limitations of the optical module 155 and the smart glasses 100. It becomes one of the quality factors of the glass 100.

In a conventional method of measuring the viewing angle of smart glasses 100 before the present invention, smart glasses 100 are fixed at the center of a facility for measuring viewing angle, and augmented reality content of the smart glasses 100 is executed.

The viewing angle of the smart glasses 100 is determined by estimating the viewing angle of the smart glasses 100 while the user looks at a viewing angle wall or steel plate at 10-degree intervals with one resolution at a distance of 1 m or 2 m from the fixed smart glasses 100. Measured.

In the case of this measurement method, it is difficult to accurately measure the viewing angle of smart glasses 100 with various resolutions, and the measurement value of the viewing angle may vary depending on the user's estimation.

In addition, it is difficult to measure the vertical viewing angle, horizontal viewing angle, and diagonal viewing angle, and the viewing angle value may vary depending on the content location of the smart glasses 100.

FIG. 5 is a flowchart explaining a method of measuring the viewing angle of smart glasses 100 according to an embodiment of the present invention, and FIG. 6 is a diagram explaining the concept of viewing angle measurement software according to an embodiment of the present invention.

The method of measuring the viewing angle of the smart glasses 100 of this embodiment is to adjust the size of the content to match the resolution of the smart glasses 100 through an augmented reality content-based app, and use a controller 300 so that the augmented reality content can be viewed intact. ) so that you can check the values of the vertical viewing angle, horizontal viewing angle, and diagonal viewing angle based on the actual screen of the smart glasses 100 by displaying them in the augmented reality content.

First, check the actual horizontal, vertical, and diagonal size and resolution values of the actual display 150 mounted on the smart glasses 100 (S110). This can be confirmed through the data sheet of the display 150 used in manufacturing the smart glasses 100.

Next, after driving the smart glasses 100, check the virtual AR screen 210 that is enlarged and displayed by the optical device (polarizing beam splitter, lens system, etc.) of the smart glasses 100 (S120).

Next, the 3D modeling application is run to output the virtual AR screen 210 (S130). The viewing angle application is an application for measuring the viewing angle and displays a virtual AR screen 210 at a specific distance (for example, 3m) to the user.

3D modeling applications can be implemented through a 3D implementation platform such as Unity3D software. In other words, it is possible to recognize the space in which the user is located, implement it in 3D, extract coordinate values within the space implemented in 3D, and measure the viewing angle of the AR screen 210 based on this.

Next, as shown in FIG. 6, the center point of the virtual AR screen 210, each vertex value, and the distance value between the virtual screen and the smart glasses 100 are displayed using the smart glasses 100 controller 300, Measure (S140).

The controller 300 of the smart glasses 100 provides a cursor like a computer mouse, and using this cursor, the user can designate the vertex of the image for measuring the viewing angle.

The controller 300 that controls the smart glasses 100 may be a terminal 300 connected by wire or a remote control connected to the smart glasses 100 and driven wirelessly.

The smart glasses 100 are equipped with sensors such as acceleration sensors, geomagnetic sensors, and inertial sensors to provide a 3D effect to the user, and are also equipped with a stereo camera 140 to recognize the surrounding space, so the sensors and cameras ( 140), you can check the user's location information in the current space.

The AR screen 210 can be output to provide a sense of distance using space and user location information collected through these sensors. The 3D modeling application can render a desired object at a specific location based on the zero point of the AR screen 210, so that the location information of the glasses 100 confirmed from the sensors and camera 140 of the smart glasses 100 is zero point. It is possible to provide the AR screen 210 by rendering it at a specific distance.

The control unit can receive data from the sensing unit 130 and input it into a 3D modeling application to check the user's location and the location of the AR screen 210 and extract coordinate values. That is, the user's location within the recognized space and the location of the AR screen 210 may have coordinate values.

The location information of the AR screen 210 guides the user to select the horizontal vertex of the content and the center point of the content using the controller 300 of the smart glasses 100, and the virtual device of the controller 300 connected to the smart glasses 100 When a point is selected by clicking with the pointer 230, the corresponding coordinate value is displayed on the screen so that the user can easily check the information.

The user can change the position and size of the AR screen 210, and at this time, the changed screen can be re-measured to measure the viewing angle for each content currently being provided to the user (S150).

The distance for each pixel displayed on the screen can be calculated using the ratio of the size of the display 150 mounted on the smart glasses 100 and the size of the AR screen 210, and the viewing angle shown in Figures 7 to 9 is measured. Using the method, the number of pixels displayed on the virtual AR screen 210 per 1° can be calculated. The values of each vertex, screen size, calculated viewing angle, and number of pixels per angle (PPD; Pixel per Degree) can be calculated and provided (S160).

7 to 9 are diagrams for explaining a viewing angle measurement method according to an embodiment of the present invention. Based on the coordinates of the vertex and center point of the AR screen 210 and the coordinates of the smart glasses 100 obtained in step S140, the horizontal and vertical sizes of the AR screen 210 and the distance to the user (smart glasses 100) can be obtained. , Based on this, the field of view (FoV) can also be measured.

Figure 7 is a diagram showing a method of measuring the horizontal viewing angle. Half of the horizontal viewing angle can be obtained by using half the horizontal size of the AR screen 210, which is the distance from the AR screen 210.

Using the distance (D) between the smart glass and the AR screen 210 and the half width size (w/2) of the AR screen 210, half the horizontal viewing angle (θ _h = tan ^-1 {(|x ₂ - Obtain x ₁ |/2)/|y ₁ -y ₀ |}) and use this to calculate the horizontal viewing angle (2θ _h ).

Figure 8 is a diagram showing a method of measuring the vertical viewing angle. Similar to the method of measuring the horizontal viewing angle, the vertical size (h) of the AR screen 210 is acquired using a 3D model application, and the horizontal viewing angle is calculated using this. . It can be calculated using the upper and lower vertices and the content center point selected by the user using the controller 300.

The measurement equation is the distance between the smart glass and the AR screen 210 (D) and the half vertical viewing angle of the AR screen 210 (h/2) (θ _v = tan ^-1 {(|y ₂ -y ₁ |/2)/|z ₁ -z ₀ |}) can be obtained and the vertical viewing angle can be calculated using this.

Figure 9 is a diagram showing a method of measuring the diagonal viewing angle. Similar to the horizontal and vertical viewing angle measurement method, the diagonal size of the AR screen 210 is acquired using the 3D content model SW, and the diagonal viewing angle is calculated using this. do.

The measurement equation is the distance (D) between the smart glasses 100 and the AR screen 210, and the half diagonal viewing angle (θ _d = tan ^-1 [{ (|y ₂ -y ₁ | ² +|x ₂ -x ₁ | ² ) ^0.5 /2}/|z ₁ -z ₀ |]) can be obtained, and the diagonal viewing angle (2θ _d ) can be calculated using this. .

The pixel per angle can be calculated based on the pixel information of the display 150 and the viewing angle calculated above.

According to one embodiment of the present invention, the vertical viewing angle, horizontal viewing angle, and diagonal viewing angle of smart glasses 100 including various resolutions and optical devices can be accurately measured using augmented reality content.

In addition, it can help measure the viewing angle of the smart glasses 100 regardless of location without using viewing angle measurement equipment that is bulky, expensive to build, and difficult to move.

In addition, after obtaining information about the viewing angle of the smart glasses 100, the size of the content can be adjusted to match the resolution of the smart glasses to provide the user with an image of optimal quality.

Since the AR screen 210 is not a screen displayed on the actual screen, if information about the viewing angle is obtained in the above manner, the viewing angle between the actual original image and the AR screen 210 can be compared to output an accurate image.

Additionally, when the user increases or decreases the size of the AR screen 210 within the space recognized by the user, the AR screen 210 can be adjusted to an accurate ratio based on the user's viewing position.

By re-measuring according to the shape of the AR screen 210 changed by the user, an additional viewing angle can be provided to the user on the currently changed screen. At this time, sensor data that changes when the virtual AR screen 210 moves or changes size can be used to help with measurement by initially using values such as vertex designation.

Although the glass 100 and the viewing angle measuring method according to the embodiment of the present invention have been described above as specific embodiments, this is only an example and the present invention is not limited thereto, and has the widest scope according to the basic idea disclosed in the present specification. It should be interpreted as having. A person skilled in the art may combine and substitute the disclosed embodiments to implement embodiments not specified, but this also does not deviate from the scope of the present invention. In addition, a person skilled in the art can easily change or modify the embodiments disclosed based on the present specification, and it is clear that such changes or modifications also fall within the scope of the present invention.

100: smart glasses
101: frame
105: Front lens
103: Nose pad
110: Department of Communications
111: cable
120: microphone
130: Sensing unit
140: camera
150: display
155: Optical module
160: speaker
180: control unit
190: Power supply unit
210: AR screen
230: pointer

Claims (10)

Checking the display size and resolution mounted on smart glasses;
Obtaining the position of the smart glasses in a space recognized through a stereo camera mounted on the smart glasses and setting the coordinates of the smart glasses to zero;
Providing an enlarged AR screen at a specific distance from the smart glasses using the optical device of the smart glasses;
Running a 3D modeling application to calculate the coordinates of the vertex and center point of the AR screen;
Measuring a viewing angle based on a specific distance at which the AR screen is displayed, coordinates of a vertex and a center point of the AR screen; and
Comprising the step of comparing the viewing angle of the original image and the AR screen and outputting an image with accurate resolution,
Smart glass control method. delete delete delete According to claim 1,
Characterized in that it further comprises calculating pixels per angle based on the resolution of the display and the viewing angle,
Smart glass control method. delete A frame mounted on the user's head;
A pair of front lenses located on the front of the frame;
A display unit mounted on the frame;
Stereo camera that recognizes the surrounding space;
An optical module that provides an AR screen by enlarging the image output from the display unit; and
It includes a control unit that controls the display unit and the optical module to control the output AR screen,
The control unit,
Obtain the location of the smart glasses in the space recognized through the stereo camera,
Running a 3D modeling application, the AR screen is displayed at a specific distance using the location of the smart glasses as a reference point,
Calculate the coordinates of the vertex and center point of the AR screen,
Measure the viewing angle based on the specific distance at which the AR screen is displayed, the coordinates of the vertex and the center point of the AR screen,
Characterized by comparing the viewing angle of the original image and the AR screen to output an image with accurate resolution,
Smart glasses. delete delete According to claim 7,
The control unit
Characterized in calculating pixels per angle based on the resolution of the display and the viewing angle,
Smart glasses.