KR20140137980A - Mobile termianl, computer-readable recording medium and method for image recognation and implementing augmented reality using smart sensors - Google Patents

Abstract

The present invention can be implemented in a mobile terminal including a camera and a smart sensor, wherein the mobile terminal comprises: a smart sensor for acquiring three-dimensional azimuth angles of the camera and the mobile terminal; a sensor signal processing unit for controlling and processing the acquired signal; and a view angle adjusting unit which compares the sensor signal with an acquired reference image, analyzes a result, and adjusts a correlation between the three-dimensional azimuth angles and a view point of the camera toward a subject. Three-dimensional azimuth angle information of a terminal, which is obtained when taking an image of the subject by using the smart sensor, is analyzed such that images to be compared with the same azimuth angle can be quickly searched for in a database. By using the present invention, by adjusting the correlation between the three-dimensional azimuth angle of the terminal and the view point of the camera and performing a primary filtering process, an unnecessary comparing process related to images, which are irrelevant to the taken image, can be ignored in the data base including a plurality of images to be compared with. Also, by matching the image taken by the camera with a corresponding image to be compared, feature comparison time is reduced and augmented reality is quickly provided.

Description

TECHNICAL FIELD [0001] The present invention relates to a method and a device for providing image recognition and augmented reality using a smart sensor, a terminal device, and a computer readable recording medium. [0002] MOBILE TERMIANL, COMPUTER-READABLE RECORDING MEDIUM AND METHOD FOR IMAGE RECOGNATION AND IMPLEMENTING AUGMENTED REALITY USING SMART SENSORS [

The present invention relates to a method of providing an augmented reality, and more particularly, to a method of providing augmented reality capable of accelerating provision of augmented reality when using a mobile terminal.

Recently, a variety of mobile terminals such as smartphones and tablets have been introduced, and augmented reality technologies that show objects on a real image screen through a camera of a mobile terminal are widely used.

Augmented reality is a technology that superimposes a virtual image on a real image of the real world seen by the user. Such augmented reality technology is gradually expanding to the field of communication in the field of communication, medical device, and mobile device such as smartphone and navigation.

For example, a recent mobile device such as a smart phone includes a camera that captures an image, a control unit that determines image information acquired from the camera, and a display unit that displays image information to a user. When the augmented reality function is implemented in the mobile terminal having the above-described configuration, the control unit analyzes the actual images in front of the camera, and adds the information image related to the actual image to the user, In real time.

Most smart devices have built-in cameras, and various applications using cameras and image processing technologies have been introduced. However, the level of application technology using target object recognition technology is at an early stage or at a stage where many experiments are performed And it is gradually improving to a reliable level.

Augmented reality technology based on recognition technology is a technique using an image acquired by a camera. When there is an image matching an already registered image by analyzing the input image, the augmented reality information corresponding to the previously registered image is provided Technology.

FIG. 1 is a flowchart illustrating a conventional method of providing an augmented reality, and FIG. 2 is a conventional augmented reality providing system.

1 and 2, a conventional augmented reality providing system 10 includes a pattern recognition unit 12 including a real image acquisition device 11, a feature extraction device 13 and a feature comparison device 14, A feature registration device 15, a database 16, a composite image generation device 17, and a composite image output device 18.

Before the augmented reality service is provided, the target images for providing the augmented reality are acquired from the real image acquisition device 11 and the characteristics of the target images are registered in the database 16 through the feature registration device 15. [ It then provides the augmented reality to the user.

Referring to FIG. 1, a conventional method of providing an augmented reality includes a step S01 of acquiring and inputting a real image through a camera, extracting and analyzing a feature of the input image (S02), a server (S03) of comparing the feature points of the input image through the camera and recognizing the image through the comparative analysis (S03). When the recognition is successful, the augmented reality information corresponding to the image is synthesized with the camera image to generate a composite image (S04) and providing the user with the augmented reality information in which the augmented reality information is synthesized in the camera room image (S05).

The pattern recognition unit 12 shown in FIG. 2 forms a core part of the entire augmented reality system, and thus the processing and speed of the system greatly affect the overall processing speed. It goes crazy. The processing time of the augmented reality is increased according to the amount of the registered database to be compared by the feature comparison unit existing in the pattern recognition unit 12. In general, the augmented reality system only acquires the image information obtained through the camera Since it depends on only the single information of the camera input image, in order to provide a reliable comparison of the characteristics and increase the success rate and to provide a reliable augmented reality service, the object image database Securing is essential.

Accordingly, in order to realize a high-level augmented reality providing system, it is desirable to increase the amount of the target object image database, but there is a problem that the number of databases to be compared increases and the speed of feature comparison processing increases. As a result, the amount of the database can not be increased unconditionally because of the processing speed problem.

Accordingly, there is a need for a reliable method for increasing the comparison processing speed while using a large amount of image database for improving the object recognition rate.

In the case of augmented reality (or pattern recognition) technology applied to a conventional smart terminal application, real-time processing is often difficult due to processing and response speed limitations. Thus, a case where a stored still image or a short- There is also a problem that the recognition rate tends to decrease much in the case of a real-time processing device.

It is an object of the present invention to solve all the problems described above.

It is another object of the present invention to provide a new augmented reality providing method capable of simultaneously increasing the image comparison processing speed while using a large amount of image database to improve the recognition rate of a target object.

Another object of the present invention is to provide an augmented reality providing method that can provide a faster processing speed by using image information and sensor information in a general mobile terminal environment including a camera and a smart sensor.

In order to accomplish the above object, a representative structure of the present invention is as follows.

According to an aspect of the present invention, there is provided a method of providing an image enhancement and augmented reality using a smart sensor, comprising the steps of: (a) when a predetermined object desired to receive an augmented reality service is present, (B) when the image of the photographed object is successfully recognized with reference to the reference image, the camera module of the mobile terminal of the user inputs the object Dimensional azimuth information as reference and comparing the size of the object in the reference image and the size of the object photographed by the user to obtain a distance from the camera module of the user's mobile terminal to the photographed object Dimensional azimuth information from the camera module of the mobile terminal of the user, (c) The height of the user holding the mobile terminal and the distance projected on the ground from the camera module of the mobile terminal of the user to the object according to the user characteristic are calculated with reference to the distance value to the object, (D) correcting the 3D azimuth using the correction reference value, and pairing the corrected 3D azimuth with a camera image input through the camera module of the user's mobile terminal (E) obtaining at least one comparison object image corresponding to azimuth information similar to the corrected three-dimensional azimuth information of the user mobile terminal on a database for providing an augmented reality, (f) And if the camera image is determined to be equal to or greater than a predetermined degree of similarity, Suggests using a smart sensor which includes the step of recognizing that the images input through the camera module of the mobile device using the information which is stored corresponding to a particular comparison-object image and the image recognition method for providing augmented reality.

According to another aspect of the present invention, there is provided an image enhancement and realization terminal device using a smart sensor, wherein when a predetermined object desired to receive an augmented reality service is present, When the image of the object is photographed and the image of the photographed object is successfully recognized with reference to the reference image, Dimensional azimuth angle information for viewing the object and comparing the size of the object in the reference image and the size of the object photographed by the user from the camera module of the mobile terminal of the user, Dimensional azimuth angle information and the distance of the user's mobile terminal A distance from the camera module to the object is referred to as a distance from the camera module of the user's mobile terminal to the object according to the height of the user holding the mobile terminal, And a camera module of the mobile terminal of the user. The camera module of the mobile terminal of the present invention may further comprise: a correction reference value calculation unit that calculates a correction reference value by referring to the corrected reference value; and a viewing angle correction unit that corrects the three- Acquiring at least one comparison object image corresponding to azimuth information similar to the corrected three-dimensional azimuth information of the user mobile terminal on a database for providing an augmented reality, And a database An augmented reality providing unit for recognizing an image input through a camera module of the mobile terminal using information stored in correspondence with the specific comparison target image if it is determined that the comparison target image and the camera image are equal to or more than a predetermined degree of similarity, And a terminal device for image recognition and augmented reality using a smart sensor.

 In addition to this, another method for implementing the present invention and a computer-readable recording medium for executing the method are further provided.

According to the present invention, an unnecessary comparison process with images in a database that is not related to a photographed image is performed on a database including a plurality of comparison target images through a primary filtering process by adjusting the correlation between the 3D azimuth angle of the terminal and the camera view point It is possible to obtain the effect of accelerating the provision of the augmented reality by shortening the feature comparison time by matching the image obtained through the camera with the corresponding comparison object image in the corresponding database.

FIG. 1 is a flowchart illustrating a conventional method of providing augmented reality.
2 is a conventional augmented reality providing system.
3 is a flowchart illustrating a method of providing an image recognition and augmented reality using a smart sensor according to the present invention.
Figs. 4 and 5 are reference views for explaining a database acquisition method at various points in time to view objects.
6 is a schematic diagram illustrating a configuration of a terminal device for providing an image enhancement and augmented reality using a smart sensor according to the present invention.

The following detailed description of the invention refers to the accompanying drawings, which illustrate, by way of illustration, specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the present invention are different, but need not be mutually exclusive. For example, certain features, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the invention in connection with an embodiment. It is also to be understood that the position or arrangement of the individual components within each disclosed embodiment may be varied without departing from the spirit and scope of the invention. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present invention is to be limited only by the appended claims, along with the full scope of equivalents to which such claims are entitled, if properly explained. In the drawings, like reference numerals refer to the same or similar functions throughout the several views.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings, so that those skilled in the art can easily carry out the present invention.

The present invention calculates a three-dimensional azimuth angle of a mobile terminal when a mobile terminal captures an image using a smart sensor of the mobile terminal, for example, an acceleration sensor, an angular velocity sensor, a geomagnetic sensor, a tilt sensor, We propose a method to select the images in the database based on the corrected azimuth angle.

3 is a flowchart illustrating a method of providing an image recognition and augmented reality using a smart sensor according to the present invention.

Referring to FIG. 3, first, images of various viewpoints having a certain distance and various three-dimensional azimuth angles from an object recognition object for providing an augmented reality are formed into a database (S301). This step corresponds to a synchronization or initialization process for enabling a user to set a correction value in accordance with the current database and the user's current situation and environment when the user intends to use the augmented reality service. The various points here can be understood as points on the spheres with the same radius.

Figs. 4 and 5 are reference views for explaining a database acquisition method at various points in time to view objects.

When you look at an object in the 3D real world, the shape of the object appears different depending on the angle and the distance from the center of the object. In Fig. 4, a "sphere" which is seen as a circle at any viewing angle and at any viewing angle is shown and described for easy explanation.

The same phenomenon is observed in a two-dimensional image obtained through a camera, depending on the angle (viewing angle) between the viewpoint of the person and the viewpoint of the person from the object (the center or one point of the object). When you look at a specific object, you see the same shape when you look at the same three-dimensional azimuth (a) line toward the center of the object. In this case, the size varies depending on the distance d between the object and the viewpoint. Therefore, as shown in FIG. 4, the spherical surface looks like a circle on a two-dimensional image, so that if the distance between the object and the camera is constant, the spherical surface looks like a circle having the same three-dimensional azimuth angle.

In the method for providing an augmented reality according to the present invention, a database image is photographed at various viewing angles at the time of constructing a database for image recognition in step S301. In the augmented reality providing method, images in which the distance from the object to be recognized is kept constant are grouped into one group, . In other words, referring to FIG. 4, the photographing time point may be a plurality of points of surfaces of spheres of various sizes, and the images photographed at a plurality of points on one spherical surface are composed of one bundle databases.

On the other hand, in the case of the recognition algorithm, although the recognition rate is relatively high in the small change of the object size and the viewing angle through the image processing process, the difference in the viewing angle from the images stored in the database is large, If the object is too small on the image, it may fail to recognize the image at the recognition rate. Therefore, for optimal operation, the user shoots the object within a certain trade range so that the object does not appear too small, and the image is compared with the image in the object database having the closest point to the three- It is necessary to perform the recognition operation efficiently and quickly.

That is, the core of the present invention is to extract a specific object database group (a group of several groups having similar approximate values) using the three-dimensional azimuth angle information that forms the center of the object to be recognized with the camera viewpoint, And to shorten the recognition rate and the operation time by attempting matching for image recognition with a small amount of images in an appropriate object database.

Referring again to FIG. 3, the method for providing an augmented reality according to the present invention includes configuring a database, and when a user desires to receive an augmented reality service, And causes the object to be photographed as a reference image (S302).

The reference image is one image for setting the correction value to be described below and is described with reference to Fig.

When the center of the object is assumed to be the origin, the viewpoint of the camera can be expressed using the 3D coordinate system using the 3D azimuth angle (a) and the distance between the center and the camera (d). Generally, The azimuth angle and the azimuth angle of the mobile terminal can be obtained. However, the distance between the mobile terminal and the object is unknown.

As shown in FIG. 5, in general, when a user uses the augmented reality system as a mobile terminal, even if the user induces him or her to photograph a specific portion (scene on the same surface) of the recognition object, The distance (l) between the height (h) and the point located from the target object will vary depending on the user's personal characteristics and behavioral habits.

Therefore, it is not possible to directly set the comparison object image database (or the image database group) existing on the database with only the three-dimensional azimuth information a of the terminal photographed by the user and to directly compare the correction value reflecting the difference with the azimuth angle of the database You will have to set it up so that you can select the nearest target database. That is, the method according to the present invention includes a step of setting a correction value reflecting a difference between the azimuth angle of the database and the azimuth angle of the database.

One image existing in the database has a three-dimensional azimuth angle a 'and a specific distance value d'. As a process for setting the correction value, one image is regarded as a reference image, Dimensional azimuth information (a) of the mobile terminal is obtained, and when the recognition is successful, the size value of the recognized object (object) is referred to as the three-dimensional azimuth information (a) It is possible to determine the correlation with the size of the object and calculate the distance to the object inversely. As a result, the approximate distance d between the objects of the mobile terminal can be obtained.

The distance l between the three-dimensional azimuth angle a 'of the reference image and the approximate distance value d obtained by the user characteristic and the height h of the user holding the mobile terminal are also roughly calculated These values are used to generate a 3D azimuth correction value that takes into account the correlation with the database, reflecting the user's personal characteristics and behavioral habits.

In order to compare the three-dimensional azimuth angle a obtained by the mobile terminal with the three-dimensional azimuth angle a 'of the image in the database, the correction value extraction process is performed only once, The comparison between the three-dimensional azimuth information a 'of the inner image and the three-dimensional azimuth information a obtained at the terminal is continuously performed.

Referring again to FIG. 3, when the user has succeeded in recognizing the image of the photographed object with reference to the reference image after the step of photographing the object with the camera, as described above, By comparing the size of the object in the reference image and the size of the object photographed by the user with reference to the three-dimensional azimuth angle information for viewing the camera module, the distance from the object photographed from the camera module of the user's mobile terminal (Step S303).

Then, from the camera module of the mobile terminal of the user according to the height of the user holding the mobile terminal and the user characteristic, the three-dimensional azimuth information and the distance value from the camera module of the user's mobile terminal are referred to, And calculates a correction reference value with reference to the distance (S304). This correction reference value is obtained only once.

Then, the 3D azimuth angle is corrected through the correction reference value, and the corrected 3D azimuth is paired with the camera image input through the camera module of the user's mobile terminal (S305).

Then, at least one comparison object image corresponding to the azimuth information similar to the corrected three-dimensional azimuth information of the user mobile terminal is acquired on the database for providing the augmented reality (S306). The comparison between the 3D azimuthal information stored in the database and the 3D azimuthal information obtained from the mobile terminal continues until a similar one of the real input image and the images in the database is found.

Then, if it is determined that the comparison target image and the camera image of the database are equal to or greater than a predetermined degree of similarity, the image input through the camera module of the mobile terminal is recognized using information stored corresponding to the specific image (S307 ).

Specifically, the step S307 may include extracting the feature information from the image input through the camera module of the mobile terminal, extracting the feature information extracted from the image of the user mobile terminal and the feature information of the image stored in the predetermined database, And comparing the specific image among the images stored in the database and the image input through the camera module of the user mobile terminal with the predetermined degree of similarity or more, And recognizing an image input through the camera module of the user mobile terminal using the information.

In operation S308, the information input through the camera module to the display device of the user's mobile terminal is displayed by combining the information stored in association with the recognized specific comparison image.

6 is a schematic diagram illustrating a configuration of a terminal device for providing an image enhancement and augmented reality using a smart sensor according to the present invention.

6, a terminal device 200 according to the present invention includes a camera 201, a smart sensor 202, a correction reference value calculation unit 203, a viewing angle correction unit 204, a sensor signal processing unit 205, An augmented reality providing unit 207, and a synthesized image generating unit 211. The augmented reality providing unit 207 and the augmented reality providing unit 207 are the same.

The smart sensor 202 includes at least one of an acceleration sensor, an angular velocity sensor, a geomagnetic sensor, and a tilt sensor.

When there is a predetermined object desired to receive the augmented reality service, the correction reference value calculation unit 203 induces the user to photograph the object as the reference image with reference to the reference image for the object, When the image of the captured object is recognized with reference to the reference image, the camera module of the user's mobile terminal refers to the three-dimensional azimuth information that the user views the object, And calculates a distance from the camera module of the user's mobile terminal to the photographed object through comparison of the size of the object in the reference image and the size of the object photographed by the user. The correction reference value calculating unit 203 calculates a reference value of the user based on the three-dimensional azimuth information and the distance between the user and the camera module of the mobile terminal, The distance projected on the ground from the camera module of the user's mobile terminal to the object according to the characteristics is calculated and the correction reference value is acquired with reference to the distance.

At this time, the correction reference value calculation unit 203 acquires the correction reference value only once.

The viewing angle correcting section 204 corrects the three-dimensional azimuth angle based on the correction reference value.

The sensor signal processing unit 205 performs a function of pairing the corrected three-dimensional azimuth and the camera image input through the camera module of the user's mobile terminal.

The comparison object image obtaining unit 206 obtains at least one comparison object image corresponding to the azimuth information similar to the corrected three-dimensional azimuth information of the user mobile terminal on the database 300 for providing the augmented reality. The comparison of the three-dimensional azimuth information stored in the database and the three-dimensional azimuth information obtained from the mobile terminal is continuously performed until a similar one of the real input image and the images in the database is found.

The augmented reality providing unit 207 includes a video feature extracting unit 208, a video feature comparing unit 209 and a similarity determining unit 210. The augmented reality providing unit 207 compares the comparison video of the database 300 with the camera video, It is possible to recognize an image input through the camera module of the mobile terminal using the information stored corresponding to the specific image to be compared.

The image feature extracting unit 208 extracts a feature image from the image input through the camera module of the user mobile terminal and converts the feature image into feature information. The image feature comparing unit 209 compares the image of the user mobile terminal The similarity determination unit 210 may compare the characteristic information of the image stored in the database 300 with the characteristic information of the image stored in the database 300, If the image input through the camera module of the mobile terminal is determined to be equal to or greater than a predetermined degree of similarity, the image input through the camera module of the mobile terminal is recognized using information stored corresponding to the specific image.

In addition, the composite image generation unit 211 synthesizes and displays information stored in association with the recognized specific comparison image on the image input through the camera module to the display device of the user mobile terminal.

The terminal device 200 for providing the augmented reality proposed in accordance with the present invention performs a calibration process on sensor values obtained through the actual image obtained through the camera 201 and the smart sensor 202 included in the mobile terminal And the corrected three-dimensional azimuth information is paired through the sensor value signal processing process. In this process, the correction reference value calculating unit 203 calculates the distance between the terminal and the photographed object through the sensor value analysis obtained in the initial reference image photographing process and the object size comparison with the reference image, (Three-dimensional azimuth angle) correction reference value for the three-dimensional viewpoint. Thereafter, the viewing angle correcting section 204 corrects the three-dimensional azimuth using this correction reference value, and the sensor signal processing section 205 pairs the corrected three-dimensional azimuth and the input camera image.

Dimensional azimuth information corrected through the signal processing process makes an access request of the database 300 and the database 300 stores a corresponding specific database image group (i.e., a corrected three-dimensional azimuth angle And a video or image group corresponding to the azimuth information similar to the information). At this time, a specific database image related to a large amount of databases can be quickly indexed through the information received by the comparison object image obtaining unit 206, and through the process, lightening of the data communication and simplification of the comparison process are performed, Acceleration is achieved.

The real image information paired with the 3D azimuth information is converted into feature information by the image feature extraction unit 208. The feature information of the converted real image is compared with the image in the specific database received by the image feature comparison unit 209 And then the feature is matched.

If the characteristic of the real input image coincides with the characteristics of the image in the called database, the composite image generating unit 211 generates an enhanced composite image in which the real image and the augmented data are synthesized. Then, Is provided.

The proposed method and the terminal device do not use only the input image information obtained through the camera, but pair the index value generated by judging and adjusting the value of the smart sensor acquired at the time of photographing with the image information, It is possible to remarkably monitor the amount of data traffic by calling the corresponding database on the network. In the feature comparison process, it is possible to reduce the comparison object and decrease the calculation amount by calling only the specific comparison object having the corresponding item.

As described above, in the method for providing the augmented reality and the augmented reality using the smart sensor according to the present invention, in providing the augmented reality system in a mobile terminal environment including a smart sensor, it is possible to reduce the data communication, It is possible to improve the processing speed through the acceleration process, and to obtain the effect of facilitating real-time processing.

On the other hand, an input image that undergoes a comparison process in order to provide an augmented reality may be input in a form of a short cut image acquired through a camera included in the mobile terminal and a still image and a video stored in advance in the mobile terminal have.

The composite image generation unit 211 and the display display can be modified to provide a variety of multimedia (moving image, voice guidance, audio, hyperlink connection, etc.) environment in addition to the image form through a simple structure change.

The embodiments of the present invention described above can be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, and the like, either singly or in combination. The program instructions recorded on the computer-readable recording medium may be those specially designed and constructed for the present invention or may be those recommended for use by those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROMs and DVDs, magneto- optical media such as floptical disks, optical media), and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules for performing the processing according to the present invention, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, Those skilled in the art will appreciate that various modifications, additions and substitutions are possible, without departing from the scope and spirit of the invention as disclosed in the accompanying claims.

Therefore, the spirit of the present invention should not be construed as being limited to the above-described embodiments, and all of the equivalents or equivalents of the claims, as well as the following claims, I will say.

Claims (13)

(a) when a predetermined object desired to receive an augmented reality service is present, inducing a user to photograph the object as the reference image with reference to a reference image for the object;
(b) when the image of the photographed object is successfully recognized with reference to the reference image, the camera module of the user's mobile terminal refers to the three-dimensional azimuth information that the user views the object, Calculating a distance from the camera module of the mobile terminal of the user to the photographed object through comparison of the size of the object within the user and the size of the object photographed by the user;
(c) a height of the user holding the mobile terminal with reference to the three-dimensional azimuth angle information and a distance value of the user from the camera module of the mobile terminal to the object, Calculating a projection distance on a ground plane from the camera module of the camera module to the object and acquiring a correction reference value with reference to the distance;
(d) correcting the 3D azimuth using the correction reference value, and pairing the corrected 3D azimuth with a camera image input through the camera module of the user's mobile terminal;
(e) obtaining at least one comparison object image corresponding to azimuth information similar to the corrected three-dimensional azimuth information of the user mobile terminal on a database for providing an augmented reality;
(f) if an image to be input through the camera module of the mobile terminal is recognized using the information stored in correspondence with the specific comparison object image, if it is determined that the comparison object image and the camera image of the database are equal to or greater than a predetermined similarity degree ;
The method of claim 1, further comprising: The method according to claim 1,
The smart sensor includes:
An acceleration sensor, an angular velocity sensor, a geomagnetic sensor, and a tilt sensor. The method according to claim 1,
The step (f)
(f1) extracting feature information from an image input through a camera module of the user mobile terminal;
(f2) comparing feature information extracted from an image of the user mobile terminal and feature information of an image stored in a predetermined database through a predetermined image recognition technique; And
(f3) if it is determined that the specific image among the images stored in the database and the image input through the camera module of the user mobile terminal are equal to or more than a predetermined degree of similarity, Recognizing an image input through the camera module of the user mobile terminal
A method for providing an image recognition and augmented reality using a smart sensor. The method according to claim 1,
In the step (c)
Wherein the correction reference value is acquired only once. The method according to claim 1,
And comparing the 3D azimuth information stored in the database with the 3D azimuth information acquired by the mobile terminal. The method according to claim 1,
(g) synthesizing and displaying information, which is stored in association with the recognized specific comparison object image, on the image input through the camera module to the display device of the user mobile terminal, (Image Recognition and Augmented Reality Method Using Sensor). When there is a predetermined object desired to receive the augmented reality service, the user is guided to shoot the object as the reference image with reference to the reference image for the object, and when the image for the object is taken, When the image of the object is successfully recognized with reference to the reference image, the camera module of the user's mobile terminal refers to the three-dimensional azimuth angle information that the user views the object, The method comprising the steps of: calculating a distance from the camera module of the mobile terminal of the user to the photographed object through comparison of the size of the object photographed by the user; And the distance value to the object, A correction reference value calculation unit for calculating a distance projected on the ground between the camera module of the mobile terminal of the user and the object according to the user characteristics and acquiring a correction reference value with reference thereto;
A viewing angle correcting unit for correcting the 3D azimuth through the correction reference value;
A sensor signal processing unit for pairing the corrected three-dimensional azimuth angle with a camera image input through a camera module of the user's mobile terminal;
A comparison object image acquiring unit for acquiring at least one comparison object image corresponding to azimuth information similar to the corrected three-dimensional azimuth information of the user mobile terminal on a database for providing an augmented reality; And
And a controller for recognizing an image input through the camera module of the mobile terminal using the information stored in correspondence with the comparison target image if it is determined that the comparison target image and the camera image of the database are equal to or greater than a predetermined degree of similarity Reality offering;
And a terminal for providing the image recognition and augmented reality using the smart sensor. 8. The method of claim 7,
The smart sensor includes:
An acceleration sensor, an angular velocity sensor, a geomagnetic sensor, and a tilt sensor. 8. The method of claim 7,
Wherein the augmented reality providing unit comprises:
An image feature extraction unit for extracting a feature image from an image input through the camera module of the user mobile terminal and converting the feature image into feature information;
An image feature comparing unit for comparing feature information of an image of the user mobile terminal with feature information of an image stored in a predetermined database through a predetermined image recognition technique; And
If it is determined that the specific image among the images stored in the database and the image input through the camera module of the mobile terminal are equal to or greater than a predetermined degree of similarity, A similarity determination unit for recognizing an image input through the module;
And a terminal device for providing the image recognition and augmented reality using the smart sensor. 8. The method of claim 7,
Wherein the correction reference value calculation unit obtains the correction reference value only once. 8. The method of claim 7,
Wherein the comparison between the 3D azimuth information stored in the database and the 3D azimuth information obtained from the mobile terminal is continuously performed. 8. The method of claim 7,
And a composite image generation unit for synthesizing and displaying information stored in association with the recognized specific comparison image on an image input through the camera module to a display device of the user mobile terminal, Image - recognition and augmented reality - providing terminal using sensors. A computer-readable recording medium recording a computer program for managing and providing image recognition and augmented reality providing method using a smart sensor according to any one of claims 1 to 6.

Images