KR20230069680A - Electronic device and method for displaying of curve indicating target point in augmented reality - Google Patents

Abstract

An electronic device and method for displaying a curve pointing to a target point in augmented reality that allows an intuitive understanding of an augmented reality environment and is easy to use is disclosed. The electronic device for displaying a curve pointing to a target point in augmented reality according to an embodiment comprises: a plane recognition unit that acquires a video image of the real environment through a camera and recognizes a plane existing in the real environment using the video image; a target point setting unit that fires a ray toward a portion of the plane and sets a point where the plane and the ray meet as a target point; a curve calculation unit that calculates a curve connecting the target point from the camera using the location information of the camera, the location information of the target point, and the location information of the target point with the height value of the camera; and a display unit that synthesizes a curve into the video image and provides a result; wherein the curve calculation unit uses the Bezier curve function to calculate the curve connecting the camera and target point.

Description

Electronic device and method for displaying a curve pointing to a target point in augmented reality

The Disclosure relates to an electronic device and method for displaying a curve indicating a target point in augmented reality.

Korean Patent Publication No. 10-2019-0028238 discloses a pointer control method and electronic device in virtual reality.

The electronic device disclosed in this publication includes a display; communication circuit; a processor electrically connected to the display and the communication circuit; and a memory electrically connected to the processor, wherein the memory, when executed, causes the processor to receive a control signal from an external controller through the communication circuit and to set a direction based at least on the received control signal. and determining a first point located in a designated first distance range in the checked direction from a first reference position related to the external controller in a virtual space displayed through the display, and determining the virtual space based on the first point. It is characterized by storing instructions for controlling the display to display a pointer on the screen of the space.

The aforementioned electronic device derives coordinate information from the movement and direction of an external controller and indicates a point in a virtual space displayed on a display as a point. For this reason, in the case of a user who is not familiar with and lacks understanding of virtual reality or augmented reality, there is a problem in that the pointer display method is not intuitive and is inconvenient to use.

KR 10-2019-0028238 A (2019. 03. 18. Public)

An object of the present disclosure is to provide an electronic device and method for intuitively understanding an augmented reality environment and displaying a curve indicating a target point in augmented reality, which is easy to use.

An electronic device displaying a curve pointing to a target point in augmented reality according to an embodiment includes a plane recognition unit that acquires a video image of a real environment through a camera and recognizes a plane existing in the real environment by using the video image; A target point setting unit for launching a light beam toward a part of a plane and setting a point where the plane and the light beam meet as a target point; a curve calculation unit that calculates a curve connecting the target points in the camera using the position information of the camera, the position information of the target point, and the position information of the target point having the height value of the camera; and a display unit synthesizing and providing the curve to a video image, wherein the curve calculation unit calculates a curve connecting the camera and the target point using a Bezier curve function.

In addition, the display unit of the electronic device displaying the curve pointing to the target point in augmented reality according to the embodiment sets a virtual arrival area having a predetermined area centered on the point where the curve and the plane meet, and sets the virtual arrival area in a video image. It is characterized in that it is synthesized and provided.

In addition, the curve of the electronic device displaying the curve pointing to the target point in augmented reality according to the embodiment is characterized in that it has a dotted line form.

In addition, an electronic device displaying a curve pointing to a target point in augmented reality according to an embodiment includes an object overlay unit that overlays a 3D object having a cursor shape on the curve; and color information of the 3D object and a 3D object in a video image. It is characterized in that it further includes; an outline generator that compares color information of overlapping regions and draws an outline around the 3D object.

In addition, the 3D object of the electronic device displaying the curve pointing to the target point in augmented reality according to the embodiment is characterized in that the width of the cursor is formed thin as the distance from the position of the camera to the target point increases.

In addition, a method of displaying a curve pointing to a target point in augmented reality according to an embodiment includes acquiring a video image of a real environment through a camera, and recognizing a plane existing in the real environment using the video image; emitting a light beam toward a part of the plane, and setting a point where the plane and the light beam meet as a target point; calculating a curve connecting the target points in the camera using camera position information, target point position information, and target point position information having a height value of the camera; and synthesizing and providing a curve to a video image.

By using an electronic device displaying a curve pointing to a target point in augmented reality according to an embodiment, augmented reality can be intuitively and conveniently understood.

In addition, an electronic device displaying a curve pointing to a target point in augmented reality according to an embodiment does not need to have separate equipment for deriving coordinate information in augmented reality.

In addition, an electronic device displaying a curve pointing to a target point in augmented reality according to an embodiment expresses a means for pointing to a target point in a curved form rather than a straight line, so it has a shape suitable for an augmented reality environment, and it is easy to check the target point in augmented reality. It's easy.

1 is a configuration diagram of an electronic device displaying a curve indicating a target point in augmented reality according to an embodiment.
2 is a diagram schematically illustrating an electronic device displaying a curve indicating a target point in augmented reality according to an embodiment.
3 shows a curve indicating a target point in augmented reality according to an embodiment.
4 shows a curve indicating a target point in augmented reality according to another embodiment.
5 is a flowchart illustrating a method of displaying a curve indicating a target point in augmented reality according to an embodiment.
6 is a diagram illustrating an operation of displaying a curve pointing to a target point in augmented reality according to an embodiment.

Advantages and features of the present disclosure, and methods of achieving them, will become clear with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in various different forms, only the present embodiments make the present disclosure complete, and those of ordinary skill in the art to which the present disclosure belongs It is provided to fully inform the person of the scope of the invention, and the present disclosure is only defined by the scope of the claims. Like reference numbers designate like elements throughout the specification.

Referring to FIG. 1 showing a configuration diagram of an electronic device (hereinafter referred to as 'electronic device 100') displaying a curve pointing to a target point in augmented reality according to an embodiment, the electronic device 100 includes a plane recognition unit 110 ), a target point setting unit 120, a curve calculation unit 130, and a display unit 140. The electronic device 100 is a terminal used by a user and may be a smartphone equipped with a camera or a tablet PC equipped with a camera.

The plane recognition unit 110 acquires a video image of a real environment through a camera built in the electronic device 100 and recognizes a plane existing in the real environment using the acquired video image.

As an example, AR Foundation, an augmented reality platform of the Unity 3D Engine, can be used to recognize a plane in a video image. AR Foundation uses a simultaneous localization and mapping process to determine relative positions in the real environment through the camera equipped on the device. Next, each visual feature, called a feature point, is found. Thereafter, clusters of corresponding feature points are found on the vertical and horizontal surfaces, and the clusters of corresponding feature points are collected and recognized as a plane.

Referring to FIG. 2 schematically showing the electronic device 100, the target point setting unit 120 emits a light beam toward a part of the plane recognized by the plane recognition unit 110, and then the point where the emitted light beam and the plane meet. is set as the target point (T).

The curve calculation unit 130 connects the camera C to the target point T by using the location information of the camera C, the location information of the target point T, and the location information of the target point T′ having the height value of the camera. calculate the curve At this time, a curve connecting the target point from the camera can be obtained using a Bezier curve function.

Specifically, the curve calculation unit 130 converts the location information of the camera C, the location information of the target point T' having the height value of the camera, and the location information of the target point T into A (start point) and B (control point), respectively. ), and C (end point), and use the quadratic Bezier curve function to obtain the position of the point according to the change in time.

(mathematical expression)

Then, a straight line is obtained from the position of the point over time. First, the position value at t = 0 and the position value at t = 0.01 are subsequently rendered as a straight line, and the position value at t = 0.01 and the position value at t = 0.02 are subsequently rendered as a straight line. After repeating this process until the position value at t=1, all rendered straight lines are rendered as curves. In the embodiment, a straight line was obtained with an interval of t of 0.01 and rendered as a curve, but as the distance from the camera position to the target point increases, the length of the curve increases, so the curve can be rendered by setting the interval of t smaller there is.

Referring to FIG. 3 showing a curve pointing to a target point in augmented reality according to an embodiment, the display unit 140 synthesizes the curve calculated by the curve calculation unit 130 into a video image of a real environment and provides the result to the user.

In addition, the display unit 140 may set a virtual arrival area S having a predetermined area centered on a point where a curve and a plane meet, and synthesize the set virtual arrival area S with a video image of a real environment. can be provided to users.

In addition, referring to FIG. 4 (a) showing a curve pointing to a target point in augmented reality according to another embodiment, the display unit 140 may shape a curve connecting the camera and the target point into a dotted line DL and provide it to the user. there is. By shaping the curve into a dotted line DL, the user can clearly see the video image provided on the display unit 140 without a portion being covered by the curve.

When the position value of the camera changes as the terminal (camera) moves, the electronic device 100 can express a curve by recognizing a plane again and repeating the above process.

In addition, the electronic device 100 may further include an object overlay unit 150 and an outline generator 160.

4 (b) to (c), the object overlay unit 150 may overlay a 3D object O having a cursor shape on the calculated curve. The width of the cursor of the 3D object O may be formed thinner as the distance from the position of the camera to the target point increases.

The outline generator 160 compares the 3D object O with an area where the 3D object overlaps, and draws an outline OL around the 3D object O. Specifically, the color information of the 3D object (O) is compared with the color information of the area where the 3D object (O) overlaps in the video image of the real environment, and when it is determined that the values of the two color information are similar, the 3D object (O) The visibility of the 3D object O may be increased by drawing an outline OL on the outside of the . At this time, the value of the two color information (the 3D object and the area where the 3D object overlaps) can be obtained using color code values such as RGB, RGBA, and CMYK.

A method of displaying a curve indicating a target point in augmented reality will be described with reference to FIG. 5, which is a flowchart of the electronic device 100, and FIG.

First, in step S110, a video image is acquired through a camera, and a plane existing in the real environment is recognized using the video image. Referring to FIG. 6 (a), it can be confirmed that a video image of a real environment is obtained. Subsequently, referring to FIG. 6 (b), it can be confirmed that the planes P and P' are recognized in the video image of the real environment. The electronic device 100 may recognize a wide area such as a floor and a desk as planes P and P'. After recognizing the plane, it goes to step S120.

In step S120, a beam is fired toward a part of the plane, and a point where the plane and the beam meet is set as a target point. Referring to FIG. 6 (c), it can be confirmed that the target point T is set on a part of the recognized plane P. After setting the target point, the process proceeds to step S130.

In step S130, a curve connecting the target points is calculated by using camera position information, target point position information, and target point position information having a height value of the camera. In this case, the curve may be calculated using a Bezier curve function. After calculating the curve, the process proceeds to step S140.

In step S140, the curve is synthesized into a video image and provided. Referring to FIG. 6 (d), it can be seen that the curve CL pointing to the target point T is synthesized with the video image.

Using the aforementioned electronic device 100, augmented reality can be intuitively and conveniently understood.

In addition, the above-described electronic device 100 does not need to have separate equipment for deriving coordinate information on augmented reality.

In addition, the electronic device 100 described above has a shape suitable for an augmented reality environment by expressing means pointing to a target point in a curved shape rather than a straight line, and it is easy to check the target point in augmented reality.

Expressions (terms, visualized images, etc.) used in describing the embodiments of the present disclosure are merely selected for instrumental purposes to enhance understanding of the technology.

In addition, the present disclosure has been described by a limited number of embodiments, and a person skilled in the art will be able to create new embodiments based on the described embodiments without departing from the technical spirit of the present disclosure.

Therefore, the claims of the present disclosure should not be limited by some expressions shown in the 'description of the invention' and 'drawings', and should be interpreted broadly based on the original technical idea inherent in the entire specification.

100... Electronics
110 ... plane recognition unit
120 ... target point setting unit
130 ... curve calculation unit
140 ... display unit
150 ... object overlay unit
160 ... outline generation unit
P, P'.... Plane C... Camera
T... target point T'... target point having the height value of the camera
CL... curve S... area of virtual arrival
DL... dotted O, O'... 3D objects

Claims (10)

a plane recognition unit that acquires a video image of a real environment through a camera and recognizes a plane existing on the real environment by using the video image;
a target point setting unit for emitting a light beam toward a part of the plane and setting a point where the plane and the light beam meet as a target point;
a curve calculation unit that calculates a curve connecting the target point from the camera using position information of the camera, position information of the target point, and position information of the target point having a height value of the camera; and
A display unit for synthesizing and providing the curve to the video image;
The curve calculation unit,
An electronic device displaying a curve pointing to a target point in augmented reality, characterized in that calculating the curve connecting the camera and the target point using a Bezier curve function. According to claim 1,
the display unit,
A virtual arrival area having a predetermined area is set centered on the point where the curve and the plane meet, and the virtual arrival area is combined with the video image and provided. Displaying a curve pointing to a target point in augmented reality electronic device. According to claim 1,
The curve is
An electronic device displaying a curve pointing to a target point in augmented reality, characterized in that it has a dotted line form. According to claim 1,
An object overlay unit for overlaying a 3D object having a cursor shape on the curve; and
and an outline generator that compares color information of the 3D object with color information of an area where the 3D object overlaps in the video image and draws an outline around the 3D object. An electronic device showing a curve pointing towards. According to claim 4,
The 3D object,
An electronic device displaying a curve pointing to a target point in augmented reality, characterized in that the width of the cursor is formed thin as the distance from the position of the camera to the target point increases. (a) acquiring a video image of a real environment through a camera, and recognizing a plane existing on the real environment using the video image;
(b) emitting a light beam toward a part of the plane, and setting a point where the plane and the light beam meet as a target point;
(c) calculating a curve connecting the target points using the camera location information, the location information of the target point, and the location information of the target point having a height value of the camera; and
(d) synthesizing and providing the curve to the video image;
The curve calculation unit,
A method of displaying a curve pointing to a target point in augmented reality, characterized in that calculating the curve connecting the camera and the target point using a Bezier curve function. According to claim 6,
In step (d),
A virtual arrival area having a predetermined area is set centered on the point where the curve and the plane meet, and the virtual arrival area is combined with the video image and provided. Displaying a curve pointing to a target point in augmented reality method. According to claim 6,
The curve is
A method of displaying a curve pointing to a target point in augmented reality, characterized in that it has a dotted line form. According to claim 6,
(e) overlaying a 3D object having a cursor shape on the curve; and
(f) comparing color information of the 3D object with color information of an area where the 3D object overlaps in the video image, and drawing an outline around the 3D object; How to display a curve pointing to a target point. According to claim 9,
The 3D object,
A method of displaying a curve pointing to a target point in augmented reality, characterized in that the width of the cursor is formed thin as the distance from the position of the camera to the target point increases.

Images