WO2024005474A1

WO2024005474A1 - Augmented-reality service device and method for providing proper distance display

Info

Publication number: WO2024005474A1
Application number: PCT/KR2023/008832
Authority: WO
Inventors: 박태욱; 조현준; 박광석
Original assignee: 주식회사 와이엠엑스
Priority date: 2022-06-30
Filing date: 2023-06-26
Publication date: 2024-01-04
Also published as: KR102541265B1

Abstract

The present invention is used in a ground surface recognition step for object augmentation in an augmented-reality service device, and provides an augmented-reality service device and method enabling guidance to be provided so that a user can easily and promptly identify whether an object to be augmented is a proper distance from the user during ground surface recognition. An augmentation-reality service device for providing a proper distance display of the present invention comprises: a camera for recognizing a ground surface and collecting data of the recognized ground surface; a ray casting unit for performing ray casting from the camera to the ground surface; and an image analysis unit for checking a conflict point with the ground surface data collected according to the ray casting, and measuring the distance between the conflict point and the camera to perform region classification according to the distance.

Description

Augmented reality service device and method for providing appropriate distance indication

The present invention relates to an augmented reality service device and method, and more specifically to an augmented reality (AR) service device and method that displays an appropriate distance according to an augmented object.

Recently, due to rapid developments in the imaging field, technologies that provide virtual reality (VR) or augmented reality (AR) through computers or user terminals have emerged.

Virtual Reality (VR) is not actual reality, but refers to a simulation technology that allows users to experience a realistic environment through a three-dimensional perspective, and Augmented Reality (AR) is a simulation technology that allows users to experience a realistic environment through three-dimensional vision. It refers to a simulation technology that overlays virtual objects on the real world.

In particular, Augmented Reality (AR) is also called Mixed Reality (MR) due to its feature of combining virtual objects with additional information in the real world in real time and displaying them as a single image.

Unlike Virtual Reality (VR), which consists of only a virtual environment, this Augmented Reality (AR) technology provides images that are a mixture of the real environment and virtual objects, so users can view virtual objects in real life. It can be seen along with the environment.

Therefore, Augmented Reality (AR) technology has the advantage of providing a better sense of reality and additional information compared to Virtual Reality (VR), and applications utilizing it have been studied.

For example, in Republic of Korea Patent Publication No. 10-1944823, augmented reality technology is used to detect the completion status of underground facilities in a batch, preventing safety accidents after construction of underground facilities and efficiently installing new underground facilities. A construction method was proposed.

However, in the past, the method simply augmented objects around the user according to their location, making it impossible to know the appropriate distance for optimal augmentation service, which varies depending on the characteristics of the object, such as the size of the object to be augmented. It was difficult to quickly and accurately determine whether appropriate reinforcement had been achieved.

In addition, when objects to be augmented, such as the turning radius of an excavator, require a certain separation distance for user safety, there has been a need for a method that can guarantee user safety even while providing augmented services.

The purpose of the present invention is to provide an augmented reality service device and method that allows a user to quickly and accurately determine the appropriate distance for optimal augmentation service, which varies depending on the characteristics of the object, such as the size of the object to be augmented.

The present invention provides an augmented reality service device and method that can ensure user safety even while providing augmented services when a certain distance is required for user safety from objects to be augmented, such as the turning radius of an excavator. There is another purpose.

An augmented reality service device that provides an appropriate distance display according to one aspect of the present invention includes a camera that recognizes the ground and collects recognized ground data; a raycasting unit that performs raycasting from the camera to the ground; and an image analysis unit that checks a collision point with the ground data collected according to the raycasting, measures the distance between the collision point and the camera, and performs area classification according to the distance.

Here, recognizing the ground from the camera and collecting the recognized ground data may be accomplished by recognizing the ground through an image of the camera and collecting ground data about the ground.

Meanwhile, the image analysis unit may perform the region classification into at least two regions based on an appropriate distance that reflects the characteristics of the object to be augmented.

At this time, the characteristics of the object to be augmented may include at least one of the size of the object, an annotation set on the object, or a safety distance of the object.

Additionally, when the image analysis unit performs the area classification based on the safety distance of the object, it may separately display a safety line on the area including the safety distance.

Additionally, the image analysis unit may generate an alarm when the user violates the safety line.

Meanwhile, the region classification by the image analysis unit can be divided into three regions: too close, too good, and too far.

Additionally, the area classification by the image analysis unit may be divided into two areas: a dangerous area and a safe area.

An augmented reality service method for providing an appropriate distance indication according to another aspect of the present invention includes the steps of recognizing the ground from a camera and collecting recognized ground data; performing raycasting from the camera to the ground in a raycasting unit; Checking the collision point with the ground data collected according to the ray casting in the image analysis unit; Measuring the distance between the collision point and the camera in the image analysis unit; and performing region classification according to distance in the image analysis unit.

Here, the step of recognizing the ground from the camera and collecting the recognized ground data may be performed by recognizing the ground through the image of the camera and collecting ground data about the ground.

In addition, in the step of performing region classification according to distance in the image analysis unit, the region classification may be performed into at least two or more regions based on an appropriate distance that reflects the characteristics of the object to be augmented.

Additionally, the characteristics of the object to be augmented may include at least one of the size of the object, an annotation set on the object, or a safety distance of the object.

Meanwhile, when the image analysis unit performs the area classification based on the safety distance of the object, the image analysis unit may separately display a safety line on the area including the safety distance.

In addition, in the step of performing area classification according to distance in the image analysis unit, areas may be classified into three areas, too close, good, and too far, or two areas, a dangerous area and a safe area.

The augmented reality service device and method for providing an appropriate distance indication according to the present invention has the advantage of allowing users to quickly and accurately determine the appropriate distance for optimal augmentation service, which varies depending on the characteristics of the object, such as the size of the object to be augmented.

In addition, the augmented reality service device and method for providing an appropriate distance display according to the present invention can be used even during the provision of augmented services when a certain distance is required for the user's safety to the object to be augmented, such as the turning radius of an excavator. It has the advantage of ensuring user safety.

Figure 1 is a schematic diagram briefly showing the configuration of an augmented reality service device that provides appropriate distance indication according to an embodiment of the present invention.

Figure 2 is a diagram to explain the principles of the raycasting technique used in the present invention to measure the distance to the ground.

FIG. 3 is a diagram illustrating an example of area classification according to distance from the ground analyzed by the image analysis unit of FIG. 1.

Figure 4 is an operation flowchart showing an augmented reality service method that provides appropriate distance indication according to an embodiment of the present invention.

Hereinafter, specific embodiments for carrying out the present invention will be described with reference to the attached drawings.

In describing the present invention, terms such as first and second may be used to describe various components, but the components may not be limited by the terms. Terms are intended only to distinguish one component from another. For example, a first component may be referred to as a second component, and similarly, the second component may be referred to as a first component without departing from the scope of the present invention.

When a component is mentioned as being connected or connected to another component, it can be understood that it may be directly connected to or connected to the other component, but other components may exist in between. .

The terms used herein are only used to describe specific embodiments and are not intended to limit the invention. Singular expressions may include plural expressions, unless the context clearly indicates otherwise.

In this specification, terms such as include or have are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, including one or more other features or numbers, It can be understood that the existence or addition possibility of steps, operations, components, parts, or combinations thereof is not excluded in advance.

Additionally, the shapes and sizes of elements in the drawings may be exaggerated for clearer explanation.

The present invention is used in the ground recognition stage for augmentation of objects in an augmented reality service device, and provides guidance so that the user can easily and quickly determine whether the object to be augmented is at an appropriate distance from the user when recognizing the ground. Provides an augmented reality service device and method that can

Hereinafter, an augmented reality service device and method for providing appropriate distance indication according to the present invention will be described in detail with reference to the attached drawings.

Figure 1 is a schematic diagram briefly showing the configuration of an augmented reality service device that provides appropriate distance indication according to an embodiment of the present invention, and Figures 2 and 3 are detailed drawings for explaining Figure 1 in detail.

Hereinafter, an augmented reality service device that provides an appropriate distance indication according to an embodiment of the present invention will be described with reference to FIGS. 1 to 3.

First, referring to FIG. 1, the augmented reality service device 100 that provides an appropriate distance display according to an embodiment of the present invention includes a camera 110 that recognizes the ground 500 and collects recognized ground data, and a camera The raycasting unit 120 performs raycasting from 110 to the ground 500, and checks the collision point with the ground data collected according to raycasting and measures the distance between the collision point and the camera 110 to determine the distance. It consists of an image analysis unit 130 that performs area classification according to.

At this time, the camera 110 can recognize the ground 500 and collect the recognized ground data by recognizing the ground through the image of the camera 110 and collecting ground data about the ground.

Additionally, the image analysis unit 130 may perform area classification at an appropriate distance that reflects the characteristics of the object 400 to be augmented. For example, if the size of the object 400 is large, the appropriate distance suitable for augmentation of the object 400 is increased proportionally, and if the size of the object 400 is small, the appropriate distance suitable for augmentation of the object 400 is increased. is reduced accordingly.

Meanwhile, depending on the implementation, the image analysis unit 130 may perform area classification by setting an appropriate distance at which the annotation set on the object 400 to be augmented can be easily recognized after augmentation. At this time, a preset value may be used as the distance at which the annotation set on the object 400 can be easily recognized.

That is, when the annotation for describing the object 400 is too close, the entire annotation content cannot be seen after augmentation. Conversely, when the annotation is too far away, the annotation after augmentation is small and difficult to identify, so the present invention In , area classification can be performed based on an appropriate distance that makes it easy for users to recognize annotations.

In one such embodiment, when it is necessary to augment and confirm annotations on circuit diagram specifications, etc. provided within the object for maintenance due to a failure occurring in the object to be augmented, the augmented reality service device 100 of the present invention provides guidance. By utilizing an appropriate distance, users can quickly and easily check annotations for the object and perform maintenance work efficiently.

On the other hand, due to the characteristics of the object 400 to be augmented, the image analysis unit 130 can perform area classification by setting the distance at which an appropriate safety distance can be secured as the appropriate distance when it is necessary to maintain the user's safety distance. there is.

At this time, the image analysis unit 130 can separately display a safety line in the area including the safety distance, and can also generate an alarm when the user violates the safety line. Here, the alarm may generate an audible alarm or may be displayed visually within augmented reality.

For example, if the object 400 to be augmented is an excavator, it is necessary to secure a safety distance according to the turning radius of the excavator, and for the user's safety, it is necessary to guide the user so as not to enter within the turning radius of the excavator. .

At this time, the augmented reality service device 100 of the present invention performs area classification by setting the appropriate distance to ensure an appropriate safety distance according to the characteristics of the object 400 to be augmented, while the camera 110 and If the distance to the object 400 is within the safety line, an alarm may be provided to alert the user.

Next, the raycasting technique used in the present invention will be described with reference to FIG. 2. Figure 2 is a diagram to explain the principles of the raycasting technique used in the present invention to measure distances on the ground.

In the present invention, raycasting technique is used to measure the distance by checking the point of collision with the ground 500. Raycasting is a technique that projects invisible light (ray) into a virtual space and identifies the surface that the light touches.

In the present invention, the distance is measured by performing raycasting forward where the camera is looking, checking the collision point, which is the point of collision with the ground, and measuring the distance through this. This will come true.

As can be seen in FIG. 3, region classification in the present invention can be performed in at least two or more regions including an region that is closer than a certain distance to the recognized ground 510, and as an example, in FIG. 3, the region classification is too small. It shows the results of area classification into close (511), good (512), and too far (513).

At this time, as described above, the augmented reality service device 100 of the present invention can perform area classification according to distance by reflecting the characteristics of the object 400 to be augmented. At this time, area classification is divided into two areas including the minimum distance. It can be carried out in the above areas.

As shown in FIG. 3, the present invention can first divide the ground into recognized ground 510 and unrecognized ground 520.

In addition, the recognized ground 510 reflects the characteristics of the object 400 to be augmented and classifies the area as too close (511), good (512), or too far (513) according to the distance from the ground. do.

For example, the augmented reality service device 100 of the present invention may use the distance at which the annotation set on the object 400 can be easily recognized as a standard for region classification.

In other words, when the annotation to describe the object 400 is too close, the entire annotation content cannot be seen after augmentation. Conversely, when the annotation is too far away, the annotation after augmentation is small and difficult to identify, so the user cannot recognize the annotation. Area classification can be performed based on an appropriate distance that is easy to do, and in this case, a preset value can be applied depending on the size of the annotation set on the object 400 or the number or number of annotations.

Through this, the user can easily and quickly check the optimal distance for augmenting the object 400 to be augmented based on the area classified by the augmented reality service device 100 of the present invention.

Meanwhile, in the above example, the area classification was divided into three areas, but the present invention is not limited to this and may be classified into two areas based on the safety distance.

For example, if the object 400 to be augmented is an object that requires a safety distance according to the turning radius, such as an excavator, area classification can be performed using the distance at which an appropriate safety distance can be secured as the appropriate distance. .

In this case, the augmented reality service device 100 of the present invention divides the recognized ground 510 into two areas: the area within the safe distance is a danger area 511, and the area outside the safe distance is a

safety area

512 and 513. Classification can be performed, and through this, the present invention can ensure user safety even while providing augmented services.

Depending on the implementation, a safety line indicating the minimum safety distance may be separately displayed in the danger area 511, and when the safety line is exceeded, an alarm may be provided to alert the user.

In addition, the augmented reality service device 100 of the present invention can display each classified area in a different color as shown in FIG. 3, which allows the user to more easily and quickly augment the object 400 of the augmentation target. You can check the optimal distance for this.

As can be seen in FIG. 4, the augmented reality service method for providing an appropriate distance indication according to an embodiment of the present invention includes the steps of recognizing the ground by the camera 110 and collecting the recognized ground data (S100); A step of performing ray casting from the camera 110 to the ground 500 in the ray casting unit 120 (S200), and a step of checking collision points with ground data collected according to ray casting in the image analysis unit 130. (S300), measuring the distance between the collision point and the camera 110 in the image analysis unit 130 (S400), and performing area classification according to the distance in the image analysis unit 130 (S500). .

At this time, recognizing the ground from the camera 110 and collecting the recognized ground data can be accomplished by recognizing the ground through the image of the camera 110 and collecting ground data about the ground.

Here, the image analysis unit 130 may perform area classification at an appropriate distance that reflects the characteristics of the object 400 to be augmented, such as the size of the object 400.

For example, the image analysis unit 130 may perform area classification by setting an appropriate distance at which the annotation set on the object 400 to be augmented can be easily recognized after augmentation. At this time, a preset value may be used as the distance at which the annotation set on the object 400 can be easily recognized.

In addition, the image analysis unit 130 can display each classified region in a different color as shown in FIG. 3, which allows the user to more easily and quickly determine the optimal distance to augment the object 400 of the augmentation target. You can check.

The augmented reality service device and method for providing an appropriate distance display according to the present invention as described above allows the user to quickly and accurately determine the appropriate distance for optimal augmented service, which varies depending on the characteristics of the object, such as the size of the object to be augmented. , In addition, if a certain distance is required for the user's safety to the object to be augmented, such as the turning radius of an excavator, the user's safety can be guaranteed even while providing the augmented service.

The embodiments described above may be implemented with hardware components, software components, and/or a combination of hardware components and software components. For example, the devices, methods, and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, and a field programmable gate (FPGA). It may be implemented using one or more general-purpose or special-purpose computers, such as an array, programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. A processing device may perform an operating system (OS) and one or more software applications that run on the operating system. Additionally, a processing device may access, store, manipulate, process, and generate data in response to the execution of software. For ease of understanding, a single processing device may be described as being used; however, those skilled in the art will understand that a processing device includes multiple processing elements and/or multiple types of processing elements. It can be seen that it may include. For example, a processing device may include a plurality of processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are possible.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. Computer-readable media may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on the medium may be specially designed and configured for the embodiment or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Includes optical media (magneto-optical media) and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. A hardware device may be configured to operate as one or more software modules to perform the operations of an embodiment, and vice versa.

Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing unit to operate as desired, or may be processed independently or collectively. You can command the device. Software and/or data may be used on any type of machine, component, physical device, virtual equipment, computer storage medium or device to be interpreted by or to provide instructions or data to a processing device. , or may be permanently or temporarily embodied in a transmitted signal wave. Software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

As described above, although the embodiments have been described with limited drawings, those skilled in the art can apply various technical modifications and variations based on this. For example, the described techniques are performed in a different order than the described method, and/or components of the described system, structure, device, circuit, etc. are combined or combined in a different form than the described method, or other components are used. Alternatively, appropriate results may be achieved even if substituted or substituted by an equivalent.

Therefore, other implementations, other embodiments, and equivalents of the claims also fall within the scope of the following claims.

The present invention relates to an augmented reality (AR) service device and method that displays an appropriate distance according to the augmented object, and can be used in the field of augmented reality.

Claims

A camera that recognizes the ground and collects recognized ground data;

a raycasting unit that performs raycasting from the camera to the ground; and

an image analysis unit that checks a collision point with the ground data collected according to the raycasting, measures the distance between the collision point and the camera, and performs area classification according to the distance;

An augmented reality service device that provides an appropriate distance indication including.
According to clause 1,

Recognizing the ground from the camera and collecting the recognized ground data,

An augmented reality service device that provides an appropriate distance indication, characterized in that it recognizes the ground through the image of the camera and collects ground data about the ground.
According to clause 1,

The video analysis unit,

An augmented reality service device that provides an appropriate distance display, characterized in that the area classification is performed into at least two or more areas based on an appropriate distance reflecting the characteristics of the object to be augmented.
According to clause 3,

The characteristics of the object to be augmented are:

An augmented reality service device that provides an appropriate distance indication, comprising at least one of the size of the object, an annotation set on the object, or a safety distance of the object.
According to clause 4,

The video analysis unit,

An augmented reality service device that provides an appropriate distance indication, characterized in that when the area classification is performed based on the safety distance of the object, a safety line is separately displayed on the area including the safety distance.
According to clause 5,

The video analysis unit,

An augmented reality service device that provides an appropriate distance indication, characterized in that it generates an alarm when the user violates the safety line.
According to clause 3,

The area classification by the image analysis unit is,

An augmented reality service device that provides an appropriate distance indication, characterized in that it is classified into three areas: too close, too good, and too far.
According to clause 3,

The area classification by the image analysis unit is,

An augmented reality service device that provides appropriate distance indication, characterized in that it is classified into two areas: a risk area and a safety area.
As an augmented reality service method,

Recognizing the ground from a camera and collecting recognized ground data;

performing raycasting from the camera to the ground in a raycasting unit;

Checking the collision point with the ground data collected according to the ray casting in the image analysis unit;

Measuring the distance between the collision point and the camera in the image analysis unit; and

performing area classification according to distance in the image analysis unit;

An augmented reality service method that provides an appropriate distance indication including.
According to clause 9,

The step of recognizing the ground from the camera and collecting the recognized ground data is,

An augmented reality service method that provides an appropriate distance indication, characterized in that it recognizes the ground through the image of the camera and collects ground data about the ground.
According to clause 9,

The step of performing area classification according to distance in the image analysis unit is,

An augmented reality service method for providing an appropriate distance indication, characterized in that the area classification is performed into at least two or more areas based on an appropriate distance reflecting the characteristics of the object to be augmented.
According to claim 11,

The characteristics of the object to be augmented are,

An augmented reality service method for providing an appropriate distance indication, comprising at least one of the size of the object, an annotation set on the object, or a safety distance of the object.
According to clause 12,

When the image analysis unit performs the area classification based on the safety distance of the object, the image analysis unit separately displays a safety line on the area including the safety distance. Augmented reality provides an appropriate distance indication. Service method.
According to clause 13,

The video analysis unit,

An augmented reality service method that provides an appropriate distance indication, characterized by generating an alarm when the user violates the safety line.
According to clause 11,

The step of performing area classification according to distance in the image analysis unit is,

An augmented reality service method that provides an appropriate distance indication, characterized in that it is classified into three areas: too close, good, and too far, or two areas: a risk area and a safe area.