KR20230051138A - Method for Providing Augmented Reality by using Distance Sensing - Google Patents

Abstract

According to the present invention, a method for providing augmented reality by using distance sensing of the present invention, executed by an apparatus that is provided with a camera receiving actual reality data, in which an object in a real world is filmed, and is provided with a distance recognition sensor capable of sensing a distance between the object in the real world and the camera comprises: synchronizing a sight direction of the distance recognition sensor and the camera, or maintaining a synchronized state; checking, when the augmented reality is implemented through the actual reality data input through the camera, a distance recognition value for each of n (n>1) distance recognition areas, obtained by sensing a distance between the camera and the object through the distance recognition sensor for each of n distance recognition areas that are one-to-one matched with areas divided from an area including a pixel area capable of doubly marking virtual data of the augmented reality from among pixel areas of the actual real data; recognizing an object area corresponding to the object, on which the virtual data is doubly marked, separately from a background area corresponding to a background based on a distance relation between distance recognition values for each of the sensed n distance recognition areas; and implementing the augmented reality by doubly marking the virtual data in an area, including the separately recognized object area, from among the pixel areas of the actual real data. Accordingly, the present invention can clearly determine an object for outputting virtual reality data.

Description

Method for Providing Augmented Reality by using Distance Sensing

The present invention is a method implemented by a device having a camera for receiving real data obtained by photographing a subject in the real world and a distance recognition sensor capable of sensing a distance between the subject in the real world and the camera, wherein the When synchronizing or maintaining the gaze direction of the camera and the distance sensor, and realizing augmented reality through real reality data input through the camera, virtual data of augmented reality is superimposed among the pixel areas of the real reality data. n distance recognition areas in which the distance between the camera and the subject is sensed through the distance recognition sensor for each n distance recognition areas matched one-to-one with the area including the displayable pixel area divided into n (n>1) pieces After confirming the distance recognition value for each distance, and based on the distance relationship between the sensed distance recognition values for each of the n distance recognition areas, the object area corresponding to the subject to be overlapped and displayed with virtual data is separated from the background area corresponding to the background and recognized. and a method for providing augmented reality using distance sensing, which implements augmented reality by overlapping and displaying virtual data in an area including the segmented and recognized object area among pixel areas of the real reality data.

The reality displayed through the display device is divided into full reality, which captures the real world through a camera, and virtual reality, which expresses a virtual world that does not exist in the real world but is created through a computer. Fiction) There is a chroma-key reality that combines virtual reality by filming real reality in front of a blue screen according to a predetermined scene scenario like a movie, and chroma-key reality belongs to virtual reality in terms of classification.

Recently, with the continuous development of information communication technology, augmented reality (Augmented Reality), which displays virtual reality data in real time on objects in real reality, has been proposed. Augmented reality is different from chroma key reality in that it does not have a pre-determined scene scenario like SF movies and does not shoot real reality in front of a blue screen.

As a method of determining an object to display virtual reality data on real reality in the augmented reality, a marker method of displaying a marker in real reality and then recognizing the marker and displaying the virtual reality data on the marker There is augmented reality and markerless augmented reality that displays virtual reality data by directly recognizing objects in real reality without displaying markers in real reality. Currently, marker-based augmented reality is providing several pilot services using each marker, but there is no standardization of marker configuration at all, and markerless augmented reality uses real reality in which a predetermined object is photographed in a studio. Except for this, it has not been demonstrated due to the instability of object recognition.

In addition, as a method of displaying virtual reality data in real reality in the augmented reality, an object is recognized in a marker/markerless method from real reality photographed through a camera and stored in a storage medium, and the virtual reality data is displayed on the recognized object. There is a method of displaying an object and a method of recognizing an object in a marker/markerless manner from real reality captured in real time through a camera and displaying virtual reality data on the recognized object. A transparent object (eg, glasses, goggles, glass windows, etc.) through which the human eye looks at the real reality beyond it is used as a transparent display without capturing reality, and through sensor technology, the human eye can see the object beyond the transparent display. A method of recognizing and displaying virtual reality data on the transparent display theoretically exists.

On the other hand, when recognizing an object in the markerless method, when reading real reality data photographed in a studio in which conditions for recognizing an object are set, it is relatively easy to recognize an object to output the virtual reality data, but the It is very difficult to recognize an object by reading real reality data photographed in an environment in which such conditions are not set or in an environment in which noise is generated in real reality data.

An object of the present invention is to provide a method that is implemented through a device having a camera for receiving real data obtained by photographing a subject in the real world and a distance recognition sensor capable of sensing the distance between the subject in the real world and the camera. , In the case of implementing augmented reality through the first step of synchronizing or maintaining the gaze direction of the camera and the distance recognition sensor and the real reality data input through the camera, the augmented reality of the pixel area of the real reality data Sensing the distance between the camera and the subject through the distance recognition sensor for each n distance recognition area that is one-to-one matched to an area obtained by dividing an area including a pixel area capable of overlapping display of virtual data of n into n (n>1) pieces Based on the distance relationship between the second step of checking the distance recognition value for each n distance recognition area and the sensed distance recognition value for each n distance recognition area, an object area corresponding to the subject to be overlapped with virtual data is displayed in the background. A distance including a third step of distinguishing and recognizing a corresponding background region and a fourth step of implementing augmented reality by overlapping and displaying virtual data in an area including the identified object region among pixel regions of the real reality data. It is to provide an augmented reality providing method using sensing.

A method for providing augmented reality using distance sensing according to the present invention includes a camera for receiving real data obtained by photographing a subject in the real world and a distance recognition sensor capable of sensing the distance between the subject in the real world and the camera. In the method executed through the device, when the first step of synchronizing or maintaining the gaze direction of the camera and the distance recognition sensor and implementing augmented reality through real reality data input through the camera, the actual The camera through the distance recognition sensor for each n distance recognition area that is one-to-one matched to an area obtained by dividing an area including a pixel area in which virtual data of augmented reality can be overlapped and displayed among pixel areas of real data into n (n>1) pieces. The second step of checking the distance recognition value for each n distance recognition area by sensing the distance between the subject and the subject to be overlapped with virtual data based on the distance relationship between the sensed distance recognition value for each n distance recognition area A third step of distinguishing and recognizing the object area corresponding to the background area corresponding to the background and realizing augmented reality by overlapping and displaying virtual data in the area including the identified object area among the pixel areas of the real reality data. It is characterized in that it comprises a fourth step of doing.

In the method for providing augmented reality using distance sensing according to the present invention, the first step further includes synchronizing a subject recognition range of the camera and a distance recognition range of the distance recognition sensor, and the third step comprises: , In a state in which the subject recognition range of the camera and the distance recognition range of the distance recognition sensor are synchronized, for each n distance recognition area that is one-to-one matched with a pixel area in which virtual data of augmented reality can be overlapped and displayed among pixel areas of the real reality data. It is characterized in that the distance recognition value for each n distance recognition area sensed by the distance between the camera and the subject through the distance recognition sensor is checked.

In the augmented reality providing method using distance sensing according to the present invention, when augmented reality is implemented through real reality data input through the camera, a pixel area of the real reality data is read and the color/saturation of the pixel area is determined. and firstly recognizing a pixel reading-based object region capable of overlapping and displaying virtual data based on a boundary line of the region divided by the above, wherein the second step includes the first recognized pixel reading-based object region. Checking the distance recognition value for each n distance recognition area where the distance between the camera and the subject is sensed through the distance recognition sensor for each n distance recognition area that is one-to-one matched to the area obtained by dividing the area to be divided into n areas; characterized by being made.

In the method for providing augmented reality using distance sensing according to the present invention, in the third step, the distance relationship including the distance difference between the sensed distance recognition values for each of the n distance recognition areas is read and virtual data is overlapped and displayed. It is characterized in that it comprises the step of distinguishing and recognizing an object area corresponding to the subject to be performed and a background area corresponding to the background.

A method for providing augmented reality using a distance difference according to the present invention is provided through a device having a camera for receiving real reality data photographed in the real world and a distance recognition sensor for recognizing a distance to an object existing in the real world. In the method to be executed, the first step of synchronizing or maintaining the gaze direction of the camera and the distance sensor, and reading a pixel area of the real reality data when receiving real reality data through the camera, A second step of firstly recognizing a pixel reading-based object region corresponding to an object capable of overlapping and displaying virtual data based on the boundary line of the region distinguished by the color/saturation of the pixel region, and the first recognized pixel reading-based object The area including the area is divided into n (n>1) distance recognition areas, and the object or background in the real world corresponding to the pixel read-based object area is detected through a distance recognition sensor synchronized with the gaze direction of the camera. A pixel read-based object area that is matched with the n distance-recognition areas by sensing distance-recognition values for each n distance-recognition areas corresponding to the straight-line distance to the camera, and using the sensed distance-recognition values for each n-number of distance-recognition areas. A distance difference between a plurality of pixel areas is calculated, and a pixel area in which the calculated distance difference between a plurality of pixel areas is within a preset range is determined, and a pixel area corresponding to a real object among the firstly recognized pixel read-based object areas is determined. A third step of secondary recognition by dividing the object region and the background region corresponding to the background, and a fourth step of superimposing and displaying designated virtual data on the region corresponding to the secondly recognized real object region among the pixel regions of the real reality data. It is characterized by including steps.

In the method for providing augmented reality using a distance difference according to the present invention, the first step further includes synchronizing a subject recognition range of the camera and a distance recognition range of the distance recognition sensor, and the third step comprises: In a state where the subject recognition range of the camera and the distance recognition range of the distance recognition sensor are synchronized, the area including the pixel reading based object area is divided into n distance recognition areas, and n distance recognition is performed through the distance recognition sensor. It is characterized by sensing the distance recognition value for each area.

In the augmented reality providing method using a distance difference according to the present invention, the distance recognition sensor is characterized in that it comprises a separate camera capable of inputting an image in addition to the camera.

In the method for providing augmented reality using a distance difference according to the present invention, the distance recognition sensor is a sensor that recognizes a distance to an object existing in the real world using at least one of an optical signal, an electromagnetic signal, and a sound wave signal. It is characterized by comprising a.

A method for providing augmented reality using a distance difference according to the present invention includes a camera for receiving real reality data photographed in the real world and a distance recognition sensor for recognizing the distance between an object existing in the real world and the camera. In the method executed by the device, in the first step of synchronizing the object recognition range of the camera and the distance recognition range of the distance recognition sensor, and when receiving real reality data through the camera, a pixel area of the real reality data A second step of firstly recognizing a pixel reading-based object region capable of superimposing and displaying virtual data based on a boundary line of the region that is read and distinguished through the color/saturation of the pixel region, and the first recognized pixel reading-based object region Divides the area including n into n (n>1) distance recognition areas, and the distance recognition value for each n distance recognition area corresponding to the distance between the camera and the object in front of the camera through the synchronized distance recognition sensor. is sensed, and distance differences are calculated for each of a plurality of pixel areas with respect to pixel read-based object areas matched with the n distance-recognition areas using the sensed distance recognition values for each n distance-recognition areas, and the first recognition A third step of secondarily recognizing a real object region including a pixel region in which a distance difference between a plurality of calculated pixel regions is within a preset range among the pixel read-based object regions and the second among the pixel regions of the real reality data and a fourth step of overlapping and displaying the specified virtual data on an area corresponding to the second recognized real object area.

In the method for providing augmented reality using a distance difference according to the present invention, when the subject recognition range of the camera is changed by zooming in or zooming out of the camera, the distance recognition range of the distance sensor is changed by zooming in or zooming out of the camera. It is characterized in that it is made by further comprising the step of synchronizing to match the recognition range.

In the method for providing augmented reality using a distance difference according to the present invention, the distance recognition sensor exists in the real world using at least one of an optical signal, an electromagnetic signal, and a sound wave signal, which are difficult to perceive through the sense organs of the human body. It is characterized in that it comprises a sensor for recognizing the distance to the object to be.

A method for providing augmented reality using a distance difference according to the present invention is provided through a device having a camera for receiving real reality data photographed in the real world and a distance recognition sensor for recognizing a distance to an object existing in the real world. In the method to be executed, the first step of synchronizing the subject recognition range of the camera and the distance recognition range of the distance recognition sensor, and reading a pixel area of the real reality data when receiving real reality data through the camera, The second step of firstly recognizing an object area based on pixel readout that can overlap and display virtual data based on the boundary line of the area distinguished through the color/saturation of the pixel area and the distance recognition sensor synchronized with the subject recognition range of the camera After adjusting the distance recognition range to the primarily recognized pixel read-based object area, a distance difference for each of a plurality of pixel areas corresponding to the pixel read-based object area is calculated through the adjusted distance sensor, and the first recognized A third step of secondarily recognizing a real object region including a pixel region in which a difference in distance between the calculated pixel regions is within a preset range among object regions based on pixel reading; and the recognized real object region among regions on the real reality data. and a fourth step of overlapping and displaying designated virtual data in an area corresponding to .

In the method for providing augmented reality using a distance difference according to the present invention, when the subject recognition range of the camera is changed by zooming in or zooming out of the camera, the distance recognition range of the distance sensor is changed by zooming in or zooming out of the camera. It is characterized in that it is made by further comprising the step of synchronizing to match the recognition range.

In the method for providing augmented reality using a distance difference according to the present invention, the distance recognition sensor exists in the real world using at least one of an optical signal, an electromagnetic signal, and a sound wave signal, which are difficult to perceive through the sense organs of the human body. It is characterized in that it comprises a sensor for recognizing the distance to the object to be.

A method for providing augmented reality using a distance difference according to the present invention is provided through a device having a camera for receiving real reality data photographed in the real world and a distance recognition sensor for recognizing a distance to an object existing in the real world. In the method to be executed, the first step of synchronizing the subject recognition range of the camera and the distance recognition range of the distance recognition sensor, and reading a pixel area on the real reality data when real reality data is input through the camera A second step of firstly recognizing an object area based on pixel reading on the real reality data capable of overlapping data, and the distance recognition range of the distance recognition sensor synchronized with the subject recognition range of the camera as the first recognized pixel on the real reality data After matching with the reading-based object area, the distance difference between the distance recognition sensor and an object in the real world corresponding to the distance recognition range matched with the pixel reading-based object area is selectively sensed, and based on the sensed distance difference, the real reality A third step of recognizing a real object area on the real reality data by separating a real object area and a background area existing in the object area based on pixel reading on the data, and virtual data assigned to the recognized real object area on the real reality data. It is characterized in that it includes a fourth step of overlapping displaying.

In the method for providing augmented reality using a distance difference according to the present invention, when the subject recognition range of the camera is changed by zooming in or zooming out of the camera, the distance recognition range of the distance sensor is changed by zooming in or zooming out of the camera. It is characterized in that it is made by further comprising the step of synchronizing to match the recognition range.

In the method for providing augmented reality using a distance difference according to the present invention, the distance recognition sensor exists in the real world using at least one of an optical signal, an electromagnetic signal, and a sound wave signal, which are difficult to perceive through the sense organs of the human body. It is characterized in that it comprises a sensor for recognizing the distance to the object to be.

In the method for providing augmented reality using a distance difference according to the present invention, in the third step, the distance value for each pixel area of an object existing in the real world corresponding to the firstly recognized object area through the distance recognition sensor and recognizing designated virtual data as a real object area to be overlapped and displayed by selecting a pixel area having a similar distance value within a preset range.

A method for providing augmented reality using a distance difference according to the present invention includes a distance recognition sensor having a camera for receiving real-reality data photographed in the real world and recognizing a distance to an object existing in the real world using a designated signal. A method executed by a device having a first step of receiving real-reality data of a specified subject recognition range through the camera, and reading the color and saturation of the real-reality data to obtain virtual data to implement augmented reality. A second step of firstly recognizing an object area that can be overlapped and displayed, and a distance recognition sensor having a distance recognition range synchronized with a subject recognition range of a camera that receives real reality data recognizing the object area. A third step of selectively sensing a distance between one or more subjects corresponding to objects in an object area on real reality data and the distance recognition sensor. When two or more objects having a difference in distance exist in the object area, the distance recognition Based on a result of selectively sensing a distance difference in the object area through a sensor, distinguishing and recognizing a real object area corresponding to a real object to be overlapped with virtual data among two or more objects existing in the object area on the real reality data A fourth step and a fifth step of overlapping and displaying designated virtual data on the real object area recognized as distinguishable on the real reality data.

In the method for providing augmented reality using a distance difference according to the present invention, by zooming in or zooming out of a camera receiving real reality data recognizing the object area, by zooming in or zooming out of the camera when the subject recognition range of the camera is changed. It is characterized in that it comprises the step of synchronizing the changed object recognition range and the distance recognition range of the distance recognition sensor.

In the method for providing augmented reality using a distance difference according to the present invention, the designated signal is characterized in that it includes at least one of a light signal, an electromagnetic wave signal, and a sound wave signal that are difficult to perceive through the sense organs of the human body. .

In the method for providing augmented reality using a distance difference according to the present invention, in the fourth step, the distance recognition value of the sensing area is calculated based on the subject recognition range of the camera unit synchronized with the distance recognition range of the distance recognition sensor. and calculating a distance value for each pixel region by applying mapping to pixel regions of the real data, and recognizing a real object region on the real reality data based on the distance value for each pixel region.

A method for providing augmented reality using a distance difference according to the present invention includes a camera with synchronized gaze directions and a separate distance recognition sensor, and displaying virtual reality data on an object on real reality data photographed through the camera In the method executed in the augmented reality device, a first step of receiving real reality data of a subject recognition range through the camera, and reading the color and saturation of the real reality data to an object area where an object can exist on the real reality data The second step of firstly recognizing , the third step of synchronizing the subject recognition range changed according to the zoom-in or zoom-out of the camera receiving the real reality data and the distance recognition range for distance sensing of the distance sensor, and the camera Within the distance recognition range synchronized with the distance recognition sensor having a distance recognition range synchronized with the object recognition range, the distance of the subject corresponding to the firstly recognized object area is recognized to determine the real object existing on the real reality data. A fourth step of secondarily recognizing a corresponding object region and a fifth step of displaying designated virtual reality data on real reality data corresponding to the secondarily recognized object region.

In the method for providing augmented reality using a distance difference according to the present invention, the step of dividing the distance recognition range to determine n (n>1) distance recognition areas, and determining the n distance recognition areas through the distance recognition sensor. measuring n distance recognition values for and calculating distance values for each pixel area of the real reality data by matching the n distance recognition areas with pixel areas of real reality data captured in the subject recognition range. In the fourth step, distance values of each pixel area included in the firstly recognized object area are compared to secondarily recognize an object area including a pixel area having a similar distance value within a preset range. It is characterized by comprising a step.

In the method for providing augmented reality using a distance difference according to the present invention, the n distance recognition areas are determined to match pixel areas for the object recognition range, or are determined by dividing the distance recognition range into equal horizontal and vertical regions. or determined by matching with pixel areas having similar color and saturation values within a predetermined range based on the color and saturation values included in the subject recognition range.

An augmented reality device for recognizing an object through a distance difference according to the present invention includes a camera and a distance recognition sensor, and displays virtual reality data on an object on real-reality data obtained by photographing the real world. A real reality input unit that inputs real reality data of a subject recognition range through a camera, and an object recognition unit 1 that primarily recognizes an object area of the real reality data by reading color/saturation of the real reality data; a recognition range determining unit that synchronizes a subject recognition range of a camera with a distance recognition range of a distance detection sensor; a recognition area determination unit that determines n (n>1) distance recognition areas by dividing the distance recognition range; A distance measurement unit that measures n distance recognition values for the n distance recognition areas through a recognition sensor, and matches pixel areas of real reality data captured in the subject recognition range with the n distance recognition areas to determine the actual distance. A distance calculation unit that calculates a distance value for each pixel region for real data, and a distance value for each pixel region included in the first recognized object region is compared to include a pixel region where the distance value is similar within a preset range. An object recognizing unit 2 for secondarily recognizing an object area is provided.

The augmented reality device for recognizing an object through a distance difference according to the present invention may further include a data storage unit that connects and stores the real reality data and the calculated distance value for each pixel area.

In the augmented reality device for recognizing an object through a distance difference according to the present invention, the n distance recognition areas are determined to match pixel areas for the object recognition range, or the distance recognition range is equal horizontally/vertically. , or may be determined by matching with pixel areas having similar hue/saturation values within a preset range based on the hue/saturation values included in the subject recognition range.

According to the present invention, between objects (or It has the advantage of being able to clearly determine the object to output virtual reality data through the distance difference between the object and the background.

1 is a diagram showing the configuration of an augmented reality device according to an implementation method of the present invention.
2 is a diagram illustrating a process of recognizing an object through a distance difference according to an embodiment of the present invention.
3 is a diagram illustrating a configuration for recognizing an object through a distance difference according to an embodiment of the present invention.
4 is an exemplary view of recognizing an object through a distance difference according to an exemplary embodiment of the present invention.

Hereinafter, the operating principle of the preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings and description. However, the drawings shown below and the description below relate to preferred implementation methods among various methods for effectively explaining the features of the present invention, and the present invention is not limited to only the drawings and description below. In addition, in the following description of the present invention, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the subject matter of the present invention, the detailed description will be omitted. In addition, terms to be described later are terms defined in consideration of functions in the present invention, which may vary according to the intention or custom of a user or operator. Therefore, the definition should be made based on the contents throughout the present invention.

In addition, in the embodiments shown below, an identification code in the form of "(number)" or "(English initial)" will be attached to the end of each term in order to more clearly explain the technical features of the present invention. The identification code is only for identifying each term, and it is clearly stated that each term is not distinguished or limited to perform different functions and roles by means of the identification code.

As a result, the technical spirit of the present invention is determined by the claims, and the following embodiments are a means for efficiently explaining the technical spirit of the present invention to those skilled in the art to which the present invention belongs. only

Figure 1 is a diagram showing the configuration of the augmented reality device 100 according to the implementation method of the present invention.

In more detail, this drawing 1 is provided with a camera and a distance recognition sensor, and in the augmented reality device 100 displaying virtual reality data on an object on real reality data photographed in the real world, the subject recognition range of the camera and the distance recognition sensor After synchronizing the distance recognition range of , n (n>1) distance recognition areas are determined by dividing the distance recognition range, n distance recognition values are measured through the distance recognition sensor, and the n distance recognition areas are determined. It shows a configuration for recognizing an object on the real reality data by matching a pixel area of the real reality data to calculate a distance value for each pixel area of the real reality data, which is based on common knowledge in the technical field to which the present invention belongs. If you have it, you can infer various implementation methods for the configuration of the augmented reality device 100 (eg, implementation methods in which some components are omitted, subdivided, or combined) by referring to and/or modifying this Figure 1. However, the present invention is made including all the implementation methods inferred above, and its technical characteristics are not limited only to the implementation method shown in FIG. 1.

The augmented reality device 100 of the present invention has a function of acquiring real reality data, a function of acquiring virtual reality data, a function of determining an object to display virtual reality data from the real reality data, and a function of determining an object on the real reality data. As a generic term for a device having a function of displaying and outputting virtual reality data, a method of acquiring the real reality data, a method of acquiring the virtual reality data, a method of determining an object, and a method of displaying the virtual reality data on the object not limited by For example, the virtual reality data may be stored in a memory of a device, received through a communication network, or generated in real time by combining one or more previously prepared polygon data and texture data. The method of displaying virtual reality data on the object is a method of generating and displaying one output data combining real reality data and virtual reality data, providing a real reality layer and a virtual reality layer in the output data, and A method of displaying data on a real reality layer and displaying virtual reality data on a virtual reality layer, or a method of displaying only virtual reality data at an object position beyond a transparent display may be used.

In addition, the augmented reality device 100 is not limited by the structure constituting the device. For example, the augmented reality device 100 may be in the form of an independent device in which each component is integrated in one case, in the form of an interlocking device in which each component is separately provided in two or more connected devices, and installed in a specific object/building/space. It can be implemented in a form or the like.

Referring to Figure 1, the augmented reality device 100 includes a control unit 105, a memory unit 135 and an output unit 110, and uses a camera unit 130 according to a method of obtaining real reality data. Further, a communication unit 120 is further provided according to a method of acquiring virtual reality data, a sensor unit 125 is further provided according to a method of determining an object on real reality data, and a control unit 115 for user manipulation ) may be further provided.

The control unit 105 physically includes a processor, an execution memory, and a bus, and functionally functions as a recording medium for processing programs included in the augmented reality device 100 to be loaded into the execution memory and operated by the processor. carry out Accordingly, in FIG. 1 , a program component provided in the augmented reality device 100 is shown in the control unit 105 for convenience.

The memory unit 135 includes a non-volatile memory and functions as a storage medium to store/maintain program codes to be loaded into the control unit 105 and data sets used by the program codes. Here, the program code includes a system program code for operating/controlling basic operations of the augmented reality device 100 and an application code for providing various application services provided in the augmented reality device 100 .

According to the implementation method of the present invention, the memory unit 135 may store the virtual reality data, and in this case, the memory unit 135 may perform a function of a virtual reality acquisition means for acquiring the virtual reality data. there is.

The output unit 110 includes an output device for displaying virtual reality data on an object on real reality data, and may include any one of a general output device such as an LCD and a transparent output device such as glasses, goggles, and glass windows. .

The camera unit 130 includes a device for reading raster-based real reality data in a bitmap format from a camera composed of an optical unit and a CCD (Charge Coupled Device). Here, the real reality data may include image data or video data. According to the present invention, the camera unit 130 may further perform the function of the sensor unit 125 that selectively acquires only specific data required for sensor input among the obtained real reality data.

According to the implementation method of the present invention, the camera unit 130 performs a function of a real reality acquisition means for acquiring the real reality data in real time, and may include a zoom-in/zoom-out function for dynamically determining a subject recognition range. .

According to the implementation method of the present invention, the camera unit 130 is integrated with a distance recognition sensor for recognizing the distance to the subject, or the gaze direction is synchronized.

The communication unit 120 includes a communication module that connects to any one of a wired communication network and a wireless communication network according to any one communication standard and protocol, and connects the communication network to a server (or device) and a communication channel, or It performs a function of connecting the augmented reality device 100 and a communication channel.

According to the implementation method of the present invention, the communication unit 120 may perform a function of a virtual reality obtaining means for receiving virtual reality data stored in a server (or device) on a communication network.

The sensor unit 125 includes a sensor module that obtains a sensor signal in the form of an analog signal or a digital signal from a distance sensor. The sensor senses the distance of the object through any one of an optical method, an image input method, an electromagnetic wave method, an object linkage method, and a sound reading method.

Here, the optical distance sensor scans light rays (eg, infrared rays) that are difficult to identify with the naked eye and are harmless to the human body into the real world space, recognizes the light rays reflected through objects in the real world space, and determines the distance of the object. It includes a sensor that senses.

The image input method distance recognition sensor inputs image data in the real world space through a camera including a CCD (Charge Coupled Device) element, reads the image data, and measures the object in the real world space included in the image data. It includes a sensor that senses the distance.

The electromagnetic wave distance detection sensor includes a sensor that receives an electromagnetic wave signal transmitted from an object and senses a distance of the object by a triangulation method when the object in the real world space has an electromagnetic wave transmitting device.

When the object interlocking method distance recognition sensor is provided with a sensor module for sensing its distance to an object in the real world space and a communication module for transmitting sensing data sensed by the sensor module to the augmented reality device 100 , A sensor that connects an object and a communication channel in the sensor unit 125, receives and reads sensing data sensed through a sensor module of the object from the object through the communication channel, and senses the distance of the object. do.

The sound reading method distance recognition sensor scans the distance of the object by injecting sound (eg, ultrasonic wave) inaudible to the human ear into the real world space, and recognizing and reading the sound reflected through the object in the real world space. It includes a sensor that

*

*

*

*

*

However, the distance detection sensor is by no means limited to the above-described type of sensor, and the distance detection sensor may be variously modified and implemented according to those skilled in the art, and the present invention includes all of the above-deformable embodiments.

According to the implementation method of the present invention, the sensor unit 125 is integrated with the camera unit 130 for recognizing a subject and inputting real-world data, or the gaze direction is synchronized.

The manipulation unit 115 includes a device that receives a user manipulation when a user manipulation is required for the augmented reality device 100 . The control unit 115 includes a key input module having key buttons to receive information required for user operation through a key input, and a touch screen module to receive information necessary for user operation through interworking with the output unit 110, A user recognition module that recognizes any one of the user's hand motion, body motion, and eye motion in conjunction with the camera unit 130 or the sensor unit 125 and receives information necessary for user manipulation may be further included.

According to the implementation method of the present invention, a function of manipulating a subject recognition range of the camera unit 130 is provided, and an operation value for manipulating the subject recognition range is transmitted to the control unit 105 .

Referring to Figure 1, the augmented reality device 100, through the camera unit 130, the real reality input unit 140 for inputting the real reality data of the subject recognition range, and the color / saturation of the real reality data An object recognizing unit (1) 145 that firstly recognizes the object area of the real reality data by reading the data, and a recognition range determining unit that synchronizes the subject recognition range of the camera unit 130 with the distance recognition range of the distance detection sensor 150; a recognition area determination unit 155 for determining n (n>1) distance recognition areas by dividing the distance recognition range; and n distance recognition areas for the n distance recognition areas through the distance recognition sensor. The distance measurement unit 160 that measures the distance recognition value and calculates the distance value for each pixel area of the real reality data by matching the n distance recognition areas with the pixel areas of the real reality data captured in the subject recognition range. and a distance calculation unit 165 that compares distance values for each pixel area included in the firstly recognized object area, and secondarily recognizes an object area including a pixel area with a similar distance value within a preset range. It includes a recognizer (2) 170, and may further include a data storage unit 175 for connecting and storing the actual reality data and the calculated distance value for each pixel area. According to the implementation method of the present invention, the actual reality input unit 140, the object recognition unit (1) 145, the recognition range determination unit 150, the distance measurement unit 160, the distance calculation unit 165, the object recognition At least one component of unit (2) 170 is augmented with the control unit 105, memory unit 135, output unit 110, sensor unit 125, camera unit 130 and communication unit 120. It may be provided in a server connected to the reality device 100, and depending on an implementation method, some components may be provided in the augmented reality device 100 and the remaining components may be provided in the server.

The real reality input unit 140 inputs data input corresponding to the subject recognition range determined through the camera unit 130 as real reality data to be output through the output unit 110 .

The object recognizing unit (1) 145 extracts a color/saturation value from the real reality data, reads the extracted color/saturation value, and checks a boundary line of an area divided based on the color/saturation. , The object area included in the real reality data is firstly recognized based on the boundary line.

The recognition range determination unit 150 checks the subject recognition range value manipulated by the manipulation unit 115 in the process of inputting the real reality data through the real reality input unit 140, and determines the subject recognition range value and The distance recognition range of the distance recognition sensor controlled through the sensor unit 125 is synchronized.

When the distance recognition range of the distance recognition sensor is synchronized with the subject recognition range of the camera unit 130 that inputs the real reality data, the recognition area determination unit 155 divides the distance recognition range to n (n>1 ) distance recognition areas are determined.

Here, the n distance recognition areas are determined to match pixel areas for the object recognition range, or determined by dividing the distance recognition range into equal horizontal/vertical areas, or colors included in the object recognition range. It may be determined by matching with a pixel area having a similar hue/saturation value within a preset range based on the /saturation value.

According to the implementation method of the present invention, the number n of the distance detection areas is determined in proportion to the distance recognition speed of the distance detection sensor. For example, if the camera unit 130 acquires real reality data of 320*200 20 times per second and the distance recognition sensor is capable of recognizing distance 1,280,000 times per second, the recognition area determination unit 155 determines the distance recognition area The number n of may be determined to be 64,000 corresponding to the number of pixels of 320*200.

When the n distance recognition areas are determined, the distance measurement unit 160 interlocks with the sensor unit 125 and uses a distance recognition sensor synchronized with the camera unit 130 in the line of sight direction to determine the n distance recognition areas. Measure n distance recognition values for .

According to the implementation method of the present invention, the distance measurement unit 160 determines the distance to the distance recognition area corresponding to the object area primarily recognized through the object recognition unit (1) 145 among the n distance recognition areas. A recognition value may be selectively measured, thereby reducing the load of the distance recognition sensor for measuring the distance.

When the n distance recognition values are measured, the distance calculating unit 165 matches the n distance recognition areas with pixel areas of real reality data photographed in the subject recognition range, so that each pixel area of the real reality data is matched. Calculate the distance value.

*

*

*

*

* When the distance value for each pixel area is calculated, the object recognition unit (2) 170 is primarily recognized based on the color/saturation of real reality data through the object recognition unit (1) 145. By comparing distance values for each pixel area included in , an object area including a pixel area having a similar distance value within a preset range is secondarily recognized.

When the distance value for each pixel area is calculated, the data storage unit 175 converts the distance value for each pixel area calculated through the distance calculation unit 165 to the meta of real reality data input through the camera unit 130. It may be linked in the form of information (or a file), or may be stored in the memory unit 135 by including the distance value for each pixel area in the real reality data.

Referring to Figure 1, the augmented reality device 100 includes a virtual reality determining unit 180 that determines virtual reality data to be displayed on an object corresponding to the finally recognized object area, and one or more virtual reality obtaining means. a virtual reality input unit 185 that inputs the determined lyric reality data through a virtual reality input unit 185 and an augmented reality output unit 190 that displays the virtual reality data on an object on the real reality data and outputs the output through the output unit 110 do.

The virtual reality determining unit 180 determines virtual reality data to be displayed on an object corresponding to a finally recognized object area from the real reality data. If the lyric reality data to be displayed on the object on the real reality data is fixed as one virtual reality data, the virtual reality determining unit 180 determines the virtual reality to be displayed on the object recognized through the fixed object recognizing unit. data can be determined.

The virtual reality input unit 185 converts virtual reality data determined through the virtual reality determination unit 180 among virtual reality data stored in a memory, received through a communication network, or generated in real time to an object of the real reality data. Enter virtual reality data to be displayed.

After the augmented reality output unit 190 displays the virtual reality data input through the virtual reality input unit 185 on an object on the real reality data input through the real reality input unit 140, the output unit 110 ) through the output.

According to one embodiment of the present invention, the augmented reality output unit 190 generates augmented reality data by overlapping virtual reality data including a transparent area with an object on the real reality data, and then converts the augmented reality data to the It can be output through the output unit 110.

According to another implementation method of the present invention, the augmented reality output unit 190 configures an augmented reality data format including a real reality layer and a virtual reality layer, and then adds the real reality layer of the augmented reality data format to the real reality layer. Data may be output and the virtual reality data may be output to a virtual reality layer corresponding to an object position of the real reality data and output through the output unit 110 .

According to another implementation method of the present invention, when the output unit 110 includes a transparent display, the augmented reality output unit 190 places the virtual reality data at an object location on the transparent display determined through the object recognition unit. It can only be displayed and printed.

Figure 2 is a diagram illustrating a process of recognizing an object through a distance difference according to an embodiment of the present invention.

In more detail, FIG. 2 shows that after synchronizing the subject recognition range of the camera and the distance recognition range of the distance recognition sensor in the augmented reality device 100 shown in FIG. 1, the distance recognition range is divided to n (n>1 ) distance recognition areas are determined, n distance recognition values are measured through the distance recognition sensors, and distance values for each pixel area of the real reality data are matched with the n distance recognition areas and pixel areas of real reality data. It shows a process of recognizing an object on the real reality data by calculating Various implementation methods for the recognition process (eg, implementation methods in which some steps are omitted or the order is changed) may be inferred, but the present invention includes all the implementation methods inferred above, and shown in FIG. The technical characteristics are not limited only to the illustrated implementation method.

Referring to Figure 2, the augmented reality device 100 inputs real-reality data of a subject recognition range through a camera (200), reads the input real-reality data, and first determines an object area based on color/saturation. At the same time as recognizing (205), a subject recognition range value for real reality data input through the camera is checked (210), and the distance recognition range of the distance sensor is set to the subject through the checked subject recognition range value. Synchronize with the recognition range (215).

If the recognition range is synchronized (220), the augmented reality device 100 divides the distance recognition range to determine n (n>1) distance recognition areas (225), and through the distance recognition sensor After measuring n distance recognition values for n distance recognition areas (230), pixels for the real reality data are matched with pixel areas of the real reality data captured in the n distance recognition areas and the subject recognition range. A distance value for each area is calculated (235).

If the distance value for each pixel area is calculated (240), the augmented reality device 100 compares the distance value for each pixel area included in the first recognized object area so that the distance value is similar within a preset range. After secondarily recognizing an object area including a pixel area (245) and determining virtual reality data for an object corresponding to the finally recognized object area (250), the determined virtual reality data is assigned to the object of the real reality data. is output (255).

Figure 3 is a diagram illustrating a configuration for recognizing an object through a distance difference according to an embodiment of the present invention.

Referring to Figure 3, when a blue object exists at a certain distance in front of a blue background, the object and the background cannot be distinguished by reading the color/saturation of real data input through the camera, but the object and the background There is a certain distance difference between them.

According to the present invention, if the distance difference measured through the distance recognition sensor is determined by synchronizing the subject recognition range of the camera unit 130 that inputs real reality data for the object and the background with the distance recognition range of the distance recognition sensor. , It is possible to recognize an object existing in the real reality data by distinguishing the object from the background.

4 is an exemplary view of recognizing an object through a distance difference according to an embodiment of the present invention.

Referring to Figure 4, when an object is located at a certain distance in front of a blue background as shown in Figure 4, the pixel of point A among the pixels of the object real reality data is a pixel corresponding to the object, and the point B is Although it is a pixel corresponding to the background, it is impossible to distinguish the object from the background by reading the color/saturation of real reality data input through the camera.

If the point (A) and the point (B) exist in different areas among the distance recognition areas classified through the recognition area determination unit 155, the distance value for each pixel area corresponding to the point (A) and Since the distance value for each pixel area corresponding to the point (B) is different, the object recognizing unit (2) 170 can clearly identify and recognize an object located at a certain distance in front of the blue background.

100: augmented reality device 125: sensor unit
130: camera unit 140: real reality input unit
145: object recognition unit (1) 150: recognition range determination unit
155: Recognition area determining unit 160: Distance measuring unit
165: distance calculation unit 170: object recognition unit (2)
175: data storage unit 180: virtual reality determination unit
185: virtual reality input unit 190: augmented reality output unit

Claims (4)

In the method executed by a device having a camera for receiving real data obtained by photographing a subject in the real world and a distance recognition sensor capable of sensing a distance between the subject in the real world and the camera, the method comprising:
A first step of synchronizing or maintaining the gaze directions of the camera and the distance sensor;
When implementing augmented reality through real-reality data input through the camera, a region including a pixel region in which virtual data of augmented reality can be overlapped and displayed among pixel regions of the real-reality data is divided into n (n>1) pieces. a second step of checking a distance recognition value for each n distance recognition area where the distance between the camera and the subject is sensed through the distance recognition sensor for each n distance recognition area that is one-to-one matched to the area;
a third step of distinguishing and recognizing an object area corresponding to a subject for which virtual data is to be overlapped and displayed from a background area corresponding to a background based on a distance relationship between the sensed distance recognition values for each n distance sensing area; and
A fourth step of implementing augmented reality by overlapping and displaying virtual data on an area including the segmented and recognized object area among pixel areas of the real reality data;
According to claim 1,
The first step further includes synchronizing a subject recognition range of the camera and a distance recognition range of the distance detection sensor,
In the third step, in a state where the subject recognition range of the camera and the distance recognition range of the distance recognition sensor are synchronized, n is matched one-to-one to a pixel area in which virtual data of augmented reality can be overlapped and displayed among pixel areas of the real reality data. A method for providing augmented reality using distance sensing, characterized in that for each distance recognition area, a distance recognition value for each n distance recognition area sensed by the distance sensor through the distance between the camera and the subject is checked.
According to claim 1,
When augmented reality is implemented through real-reality data input through the camera, a pixel area of the real-reality data is read and virtual data is overlapped and displayed based on the boundary line of the area distinguished by the color/saturation of the pixel area. Further comprising the step of first recognizing possible pixel read-based object regions;
The second step,
N distance recognition by sensing the distance between the camera and the subject through the distance recognition sensor for each n distance recognition area that is one-to-one matched to the area obtained by dividing the area including the firstly recognized pixel read-based object area into n areas A method for providing augmented reality using distance sensing, comprising the step of checking a distance recognition value for each area.
The method of claim 1, wherein the third step,
Recognizing an object region corresponding to a subject to be overlapped with virtual data and a background region corresponding to a background by reading a distance relationship including a distance difference between distance recognition values for each of the sensed n distance recognition regions. A method for providing augmented reality using distance sensing, comprising:

Images