KR102541265B1 - Augmented reality service apparatus and method for providing appropriate distance indication - Google Patents

Abstract

The present invention is used in the ground recognition step for augmenting an object in an augmented reality service device, and guides the user to quickly and easily determine whether an object to be augmented is at an appropriate distance from the user when recognizing the ground. Provides an augmented reality service device and method capable of
An augmented reality service device that provides an appropriate distance display of the present invention includes a camera that recognizes the ground and collects the recognized ground data, a raycasting unit that performs raycasting from the camera to the ground, and the raycasting and an image analysis unit that checks a collision point with the collected ground data according to the method, measures a distance between the collision point and the camera, and performs region classification according to the distance.

Description

Augmented reality service apparatus and method for providing appropriate distance indication

The present invention relates to an augmented reality service apparatus and method, and more particularly, to an augmented reality (AR) service apparatus and method for displaying an appropriate distance according to an augmented object.

Recently, due to the rapid development in the field of imaging, a technology for providing virtual reality (VR) or augmented reality (AR) through a computer or user terminal has emerged.

Virtual Reality (VR) refers to a simulation technology that allows users to experience a reality-like environment through a three-dimensional view, although it is not real reality, and Augmented Reality (AR) refers to a simulation technology It means a simulation technology that superimposes virtual objects on the real world viewed as

In particular, Augmented Reality (AR) is also called Mixed Reality (MR) due to its feature of displaying a single image by combining virtual objects having additional information in real time in the real world.

Unlike Virtual Reality (VR), which consists of only a virtual environment, this Augmented Reality (AR) technology provides an image in which the real environment and virtual objects are mixed. can be seen with the environment.

Therefore, augmented reality (AR) technology has the advantage of providing a better sense of reality and additional information when compared to virtual reality (VR), and applications utilizing this technology have been studied.

For example, in Korean Patent Registration No. 10-1944823, augmented reality technology is used to comprehensively detect the completion status of underground facilities, preventing safety accidents after the construction of underground facilities, while efficiently burying new underground facilities. A construction method was proposed.

However, in the prior art, it was a method of simply augmenting objects around the user according to the location of the user, so it was not possible to know the appropriate distance for the optimal augmentation service that varies depending on the characteristics of the object, such as the size of the object to be augmented. It was difficult to quickly and accurately ascertain whether adequate augmentation had been achieved.

In addition, when an object to be augmented, such as the rotation radius of an excavator, requires a certain degree of separation distance for user safety, a method capable of ensuring user safety while providing an augmented service has been required.

Republic of Korea Patent Registration No. 10-1944823 (2019.01.28.)

An object of the present invention is to provide an augmented reality service apparatus and method in which a user can quickly and accurately determine an appropriate distance for an optimal augmented service that varies depending on the characteristics of an object, such as the size of an augmented target object.

The present invention provides an augmented reality service apparatus and method capable of ensuring the user's safety even during the provision of the augmented service when an object to be augmented, such as the turning radius of an excavator, requires a certain distance for the user's safety. It has another purpose.

According to an aspect of the present invention, an augmented reality service device for providing an appropriate distance display includes a camera for recognizing a ground and collecting recognized ground data; a raycasting unit performing raycasting from the camera to the ground; and an image analysis unit that checks a collision point with ground data collected through ray casting, measures a distance between the collision point and the camera, and performs region classification according to the distance.

Here, recognizing the ground by the camera and collecting the recognized ground data may be performed by recognizing the ground through an image of the camera and collecting ground data of the ground.

Meanwhile, the image analyzer may perform the region classification into at least two or more regions based on an appropriate distance reflecting characteristics of an object to be augmented.

In this case, the characteristics of the object of the augmentation target may include at least one of a size of the object, an annotation set on the object, or a safety distance of the object.

In addition, when the area classification is performed based on the safety distance of the object, the image analyzer may separately display a safety line on an area including the safety distance.

In addition, the video analysis unit may generate an alarm when the user violates the safety line.

Meanwhile, the region classification by the image analyzer may be classified into three regions: too close, good, and too far.

In addition, the area classification by the image analysis unit may be classified into two areas, a danger area and a safe area.

An augmented reality service method for providing an appropriate distance indication according to another aspect of the present invention includes the steps of recognizing the ground in a camera and collecting the recognized ground data; performing ray casting from the camera to the ground in a ray casting unit; Checking a collision point with ground data collected according to the ray casting in an image analysis unit; measuring a distance between the collision point and the camera in the image analyzer; and performing region classification according to the distance in the image analyzer.

Here, recognizing the ground by the camera and collecting the recognized ground data may be performed by recognizing the ground through an image of the camera and collecting ground data of the ground.

In addition, in the step of performing region classification according to the distance in the image analyzer, the region classification may be performed into at least two or more regions based on an appropriate distance reflecting characteristics of an object of an augmentation target.

Also, the characteristics of the object of the augmentation target may include at least one of a size of the object, an annotation set on the object, or a safety distance of the object.

Meanwhile, when the image analyzer performs the area classification based on the safety distance of the object, the image analyzer may separately display a safety line on the area including the safety distance.

In addition, the video analysis unit may generate an alarm when the user violates the safety line.

Also, in the step of classifying regions according to the distance in the image analysis unit, regions may be classified into three regions of too close, good, and too far, or two regions of danger and safety.

The augmented reality service apparatus and method for providing an appropriate distance display according to the present invention has an advantage in that a user can quickly and accurately grasp an appropriate distance capable of providing an optimal augmented service that varies depending on the characteristics of an object such as the size of an object to be augmented.

In addition, the augmented reality service apparatus and method for providing an appropriate distance display according to the present invention, when an object to be augmented, such as the rotation radius of an excavator, requires a certain distance for the user's safety, even during the provision of the augmented service. It has the advantage of ensuring user safety.

1 is a configuration schematic diagram briefly showing the configuration of an augmented reality service device that provides an appropriate distance indication according to an embodiment of the present invention.
2 is a diagram for explaining the principle of a raycasting technique used for measuring a distance to the ground in the present invention.
FIG. 3 is a diagram for explaining an example of region classification according to the distance of the ground analyzed by the image analyzer of FIG. 1 .
4 is an operational flowchart illustrating an augmented reality service method for providing an appropriate distance display according to an embodiment of the present invention.

Hereinafter, specific embodiments for carrying out the present invention will be described with reference to the accompanying drawings.

In describing the present invention, terms such as first and second may be used to describe various components, but the components may not be limited by the terms. Terms are only for the purpose of distinguishing one element from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element, without departing from the scope of the present invention.

When a component is referred to as being connected or connected to another component, it may be directly connected or connected to the other component, but it may be understood that another component may exist in the middle. .

Terms used in this specification are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions may include plural expressions unless the context clearly dictates otherwise.

In this specification, the terms include or include are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features or numbers, It can be understood that the presence or addition of steps, operations, components, parts, or combinations thereof is not precluded.

In addition, shapes and sizes of elements in the drawings may be exaggerated for clearer description.

The present invention is used in the ground recognition step for augmenting an object in an augmented reality service device, and guides the user to quickly and easily determine whether an object to be augmented is at an appropriate distance from the user when recognizing the ground. Provides an augmented reality service device and method capable of

Hereinafter, an augmented reality service apparatus and method for providing an appropriate distance display according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a configuration schematic diagram briefly showing the configuration of an augmented reality service device that provides an appropriate distance indication according to an embodiment of the present invention, and FIGS. 2 and 3 are detailed views for explaining FIG. 1 in detail.

Hereinafter, with reference to FIGS. 1 to 3 , an augmented reality service device providing an appropriate distance indication according to an embodiment of the present invention will be described.

First, referring to FIG. 1, the augmented reality service device 100 for providing an appropriate distance display according to an embodiment of the present invention includes a camera 110 that recognizes the ground 500 and collects the recognized ground data, and a camera The ray casting unit 120 that performs ray casting from 110 to the ground 500 and the collision point with the ground data collected according to the ray casting are checked, and the distance between the collision point and the camera 110 is measured to determine the distance It consists of an image analysis unit 130 that performs region classification according to.

At this time, recognizing the ground 500 in the camera 110 and collecting the recognized ground data may be performed by recognizing the ground through an image of the camera 110 and collecting ground data on the ground.

Also, the image analyzer 130 may perform region classification with an appropriate distance reflecting the characteristics of the object 400 as an augmentation target. For example, when the size of the object 400 is large, an appropriate distance suitable for augmentation of the object 400 is increased in proportion thereto, and when the size of the object 400 is small, an appropriate distance suitable for augmentation of the object 400 is increased. will be reduced accordingly.

Meanwhile, depending on the embodiment, the image analyzer 130 may perform region classification by setting an easily recognizable distance as an appropriate distance after augmenting the annotation set in the object 400 of the augmentation target. At this time, a preset value may be used for a distance at which the annotation set in the object 400 can be easily recognized.

That is, when the annotation to describe the object 400 is too close, it is impossible to see the entire annotation content after augmentation, and conversely, when the annotation is too far away, the annotation after augmentation is small and is not easy to identify. In , area classification can be performed based on an appropriate distance for the user to easily recognize annotations.

As such an embodiment, when it is necessary to augment and check annotations on circuit diagram specifications, etc. provided in the object for maintenance due to a failure occurring in the object of the augmentation target, the augmented reality service device 100 of the present invention guides By utilizing an appropriate distance, the user can easily and quickly check the annotations for the object and can efficiently perform maintenance work.

On the other hand, the image analyzer 130 may perform region classification by setting a distance that can secure an appropriate safety distance as an appropriate distance when it is necessary to maintain the user's safety distance due to the characteristics of the object 400 of the augmentation target. there is.

At this time, the image analyzer 130 may separately display a safety line on the area including the safety distance, and also generate an alarm when the user violates the corresponding safety line. Here, the alarm may generate an audible alarm or may be visually displayed in augmented reality.

For example, if the object 400 of the augmentation target is an excavator, it is necessary to secure a safety distance according to the excavator's rotation radius, and for the user's safety, it is necessary to guide the user not to enter the excavator's rotation radius. .

At this time, the augmented reality service device 100 of the present invention performs area classification by setting a distance that can secure an appropriate safety distance according to the characteristics of the object 400 of the augmentation target as an appropriate distance, while the camera 110 and When the distance to the object 400 is within the safety line, an alarm may be provided to alert the user.

Next, a raycasting technique used in the present invention will be described with reference to FIG. 2. 2 is a diagram for explaining the principle of a raycasting technique used for measuring a distance on the ground in the present invention.

In the present invention, a raycasting technique is used to measure a distance by checking a point colliding with the ground 500 . The raycasting technique is a technology that projects invisible light (Ray) into a virtual space and identifies the surface that the light hits.

In the present invention, the distance is measured by performing raycasting in the forward direction where the camera is looking, checking the collision point, which is the point at which it collides with the ground, and measuring the distance through this this will be done

FIG. 3 is a diagram for explaining an example of region classification according to the distance of the ground analyzed by the image analyzer of FIG. 1 .

As can be seen in FIG. 3, area classification in the present invention can be performed with at least two or more areas including areas close to the recognized ground 510 by a certain distance or less. The results of region classification as close (511), good (512), and too far (513) are shown.

At this time, as described above, the augmented reality service device 100 of the present invention may reflect the characteristics of the object 400 of the augmentation target and perform region classification according to the distance. It can be performed in more areas.

As shown in FIG. 3 , the present invention may first divide the ground into a recognized ground 510 and an unrecognized ground 520 .

Then, the characteristics of the object 400 of the augmentation target are reflected on the recognized ground 510, and the area is classified as too close (511), good (512), or too far (513) according to the distance to the ground. do.

For example, the augmented reality service apparatus 100 of the present invention may use a distance at which an annotation set in the object 400 can be easily recognized for the region classification may be used as a criterion for region classification.

That is, when the annotation for describing the object 400 is too close, it is impossible to see the entire annotation content after augmentation, and conversely, when the annotation is too far away, the annotation after augmentation is small and not easy to identify, so the user recognizes the annotation. Area classification may be performed based on an appropriate distance, which is easy to do. In this case, a preset value may be applied according to the size of the annotation set in the object 400 or the number of annotations.

Through this, the user can easily and quickly check the optimal distance for augmenting the object 400 of the augmentation target based on the area classified by the augmented reality service device 100 of the present invention.

Meanwhile, in the above example, the area classification was classified into three areas, but the present invention is not limited thereto and may be classified into two areas based on the safety distance.

For example, if the object 400 of the augmentation target is an object that requires a safety distance according to a turning radius, such as an excavator, area classification may be performed by setting a distance that can secure an appropriate safety distance as an appropriate distance. .

In this case, the augmented reality service device 100 of the present invention is an area within the safety distance with respect to the recognized ground 510 is divided into two areas: the danger area 511 and the area outside the safety distance are the safety areas 512 and 513. Classification can be performed, and through this, the present invention can ensure the user's safety even while providing the augmented service.

Depending on implementation, a safety line indicating a minimum safety distance may be separately displayed in the danger area 511, and an alarm may be provided to alert the user when the user deviates from the corresponding safety line.

In addition, the augmented reality service device 100 of the present invention can display each area classified in a different color as shown in FIG. 3, through which the user can more easily and quickly augment the object 400 of the augmentation target. You can check the optimal distance for

4 is an operational flowchart illustrating an augmented reality service method for providing an appropriate distance display according to an embodiment of the present invention.

As can be seen in Figure 4, the augmented reality service method for providing an appropriate distance display according to an embodiment of the present invention, recognizing the ground in the camera 110 and collecting the recognized ground data (S100), Performing ray casting from the camera 110 to the ground 500 in the ray casting unit 120 (S200), and checking the collision point with the ground data collected according to the ray casting in the image analysis unit 130 (S300), measuring the distance between the collision point and the camera 110 in the image analyzer 130 (S400), and performing area classification according to the distance in the image analyzer 130 (S500). .

In this case, the camera 110 recognizing the ground and collecting the recognized ground data may be performed by recognizing the ground through an image of the camera 110 and collecting ground data on the ground.

Here, the image analyzer 130 may perform region classification at an appropriate distance reflecting the characteristics of the object 400 to be augmented, such as the size of the object 400 .

For example, the image analyzer 130 may perform region classification by setting an easily recognizable distance as an appropriate distance after augmenting an annotation set in the object 400 of the augmentation target. At this time, a preset value may be used for a distance at which the annotation set in the object 400 can be easily recognized.

On the other hand, the image analyzer 130 may perform region classification by setting a distance that can secure an appropriate safety distance as an appropriate distance when it is necessary to maintain the user's safety distance due to the characteristics of the object 400 of the augmentation target. there is.

At this time, the image analyzer 130 may separately display a safety line on the area including the safety distance, and also generate an alarm when the user violates the corresponding safety line. Here, the alarm may generate an audible alarm or may be visually displayed in augmented reality.

In addition, the image analyzer 130 may display each region classified in a different color as shown in FIG. 3 , and through this, the user can more easily and quickly achieve an optimal distance for augmenting the object 400 to be augmented. can be checked.

In the augmented reality service apparatus and method for providing an appropriate distance display according to the present invention as described above, the user can quickly and accurately grasp the appropriate distance for the optimal augmented service, which varies depending on the characteristics of the object such as the size of the augmented target object, , In addition, when an object to be augmented, such as the rotation radius of an excavator, requires a certain distance for the user's safety, the user's safety can be guaranteed even while the augmented service is provided.

The embodiments described above may be implemented as hardware components, software components, and/or a combination of hardware components and software components. For example, the devices, methods and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate (FPGA). array), programmable logic units (PLUs), microprocessors, or any other device capable of executing and responding to instructions. The processing device may run an operating system (OS) and one or more software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of software. For convenience of understanding, there are cases in which one processing device is used, but those skilled in the art will understand that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it can include. For example, a processing device may include a plurality of processors or a processor and a controller. Also, other processing configurations are possible, such as parallel processors.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. Computer readable media may include program instructions, data files, data structures, etc. alone or in combination. Program commands recorded on the medium may be specially designed and configured for the embodiment or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler. A hardware device may be configured to act as one or more software modules to perform the operations of the embodiments and vice versa.

Software may include a computer program, code, instructions, or a combination of one or more of the foregoing, which configures a processing device to operate as desired or processes independently or collectively. You can command the device. Software and/or data may be any tangible machine, component, physical device, virtual equipment, computer storage medium or device, intended to be interpreted by or provide instructions or data to a processing device. , or may be permanently or temporarily embodied in a transmitted signal wave. Software may be distributed on networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer readable media.

As described above, although the embodiments have been described with limited drawings, those skilled in the art can apply various technical modifications and variations based on this. For example, the described techniques may be performed in an order different from the method described, and/or components of the described system, structure, device, circuit, etc. may be combined or combined in a different form than the method described, or other components may be used. Or even if it is replaced or substituted by equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents of the claims are within the scope of the following claims.

Claims (15)

A camera that recognizes the ground and collects the recognized ground data;
a raycasting unit performing raycasting from the camera to the ground; and
an image analysis unit that checks a collision point with ground data collected through the ray casting, measures a distance between the collision point and the camera, and performs region classification according to the distance;
including,
The video analysis unit,
Performing the region classification into at least two or more regions based on an appropriate distance reflecting the characteristics of an object of an augmentation target;
The characteristics of the object of the augmentation target are,
An augmented reality service device that provides an appropriate distance display, characterized in that it includes at least one of the size of the object, an annotation set on the object, or a safety distance of the object.
According to claim 1,
Recognizing the ground in the camera and collecting the recognized ground data,
Augmented reality service device for providing an appropriate distance display, characterized in that made in a way to recognize the ground through the image of the camera and collect ground data for the ground.
delete delete According to claim 1,
The video analysis unit,
When the area classification is performed based on the safety distance of the object, a safety line is separately displayed on the area including the safety distance.
According to claim 5,
The video analysis unit,
Augmented reality service device that provides an appropriate distance display, characterized in that for generating an alarm when the user violates the safety line.
According to claim 1,
The region classification by the image analysis unit,
Augmented reality service device that provides an appropriate distance display, characterized in that the area is classified into three areas of too close, good, and too far.
According to claim 1,
The region classification by the image analysis unit,
Augmented reality service device that provides an appropriate distance display, characterized in that the area is classified into two areas, a danger area and a safety area.
As an augmented reality service method,
Recognizing the ground by the camera and collecting the recognized ground data;
performing ray casting from the camera to the ground in a ray casting unit;
Checking a collision point with ground data collected according to the ray casting in an image analysis unit;
measuring a distance between the collision point and the camera in the image analyzer; and
performing region classification according to distance in the image analysis unit;
including,
The step of performing region classification according to the distance in the image analysis unit,
Performing the region classification into at least two or more regions based on an appropriate distance reflecting the characteristics of an object of an augmentation target;
The characteristics of the object of the augmentation target are,
Augmented reality service method for providing an appropriate distance indication, characterized in that it includes at least one of the size of the object, an annotation set on the object, or a safety distance of the object.
According to claim 9,
Recognizing the ground in the camera and collecting the recognized ground data,
Augmented reality service method for providing an appropriate distance display, characterized in that made by recognizing the ground through the image of the camera and collecting ground data for the ground.
delete delete According to claim 9,
When the image analysis unit performs the area classification based on the safety distance of the object, the image analysis unit separately displays a safety line on the area including the safety distance. service method.
According to claim 13,
The video analysis unit,
Augmented reality service method for providing an appropriate distance indication, characterized in that when the user invades the safety line, generating an alarm for this.
According to claim 9,
The step of performing region classification according to the distance in the image analysis unit,
An augmented reality service method for providing an appropriate distance indication, characterized in that the areas are classified into three areas of too close, good, and too far, or two areas of danger area and safe area.

Images