KR102388230B1 - Service providing system and method for recognizing object, apparatus and computer readable medium having computer program recorded therefor - Google Patents

Abstract

The present invention relates to a system and method for providing an object recognition service, an apparatus for the same, and a computer program recorded thereon. To a system and method for providing object recognition service that provides object-related information through accurate recognition of an object that a user queries based on, and an apparatus and a computer program therefor are recorded. The present invention compares the feature information extracted from the query image generated by the user for querying the object by additionally using the feature information extracted from the interpolated image interpolated based on the representative image as well as the actual photographed representative image. It is possible to broaden the recognition range of an object corresponding to

Description

An object recognition service providing system and method, and an apparatus and a computer program recorded therefor, TECHNICAL FIELD

The present invention relates to a system and method for providing an object recognition service, an apparatus for the same, and a computer program recorded thereon. To a system and method for providing object recognition service that provides object-related information through accurate recognition of an object that a user queries based on, and an apparatus and a computer program therefor are recorded.

With the development of the image process, various services that provide various information related to objects through accurate recognition of objects are emerging. Such a service mainly stores information about a number of objects, and since it is required to distinguish an object that a user queries from among many objects, it is most important to increase the accuracy of object recognition.

The existing object recognition process is limited to two-dimensional images and mostly uses a method of classifying objects through simple image comparison. As it declines, it is difficult to accurately provide object-related information.

Accordingly, there is a demand for an alternative for improving the accuracy of providing object-related information through accurate recognition of the object the user is querying.

Korean Patent Laid-Open Patent No. 10-2011-0136028 [Title of the invention: Augmented reality device for recognizing objects through virtual dots]

The present invention stores characteristic information of an object photographed from a plurality of angles, compares it with characteristic information about an object that a user queries, and identifies an object based on matching characteristic information, The purpose of this is to increase the recognition accuracy of the object inquired by the user by generating and storing the characteristic information of the object inferred from a specific angle based on the characteristic information.

In addition, the present invention induces the user to adjust the shooting angle of the user device when there is no pre-stored feature information that matches the feature information extracted from the image for the object the user is querying to match the angle corresponding to the pre-stored feature information The purpose of this is to increase the object recognition accuracy by allowing matching to occur.

The object recognition service providing apparatus according to an embodiment of the present invention includes an object feature extracting unit for extracting feature information from an image, one or more representative images photographed at a representative angle with respect to the object, and an object feature extracting unit extracted for each representative image. A first storage unit that matches and stores representative characteristic information with object-related information, generates an interpolated image mapped on a virtual space based on each representative image of the first storage unit, and interpolates characteristic information from the interpolated image through an object characteristic extraction unit Receives a query image according to the shooting of an object from the virtual space mapping unit generating After extracting a feature from the first storage unit and comparing it with the first storage unit, object-related information matched with representative characteristic information matching more than a preset reference value is provided. It may include a comparator for providing object-related information matched to the interpolation characteristic information that matches the above.

As an example related to the present invention, the virtual space mapping unit may be characterized in that the representative image is modeled in the virtual space by mapping the position information of the virtual space corresponding to the representative angle of the representative image.

As an example related to the present invention, the virtual space mapping unit is mapped between different representative images in a virtual space based on position information of each modeled representative image in the virtual space to generate an interpolated image that interpolates between different representative images. can be characterized as

As an example related to the present invention, when there is no feature information that matches a preset reference value or more through the first and second storage units, the comparison unit works with the virtual space mapping unit to obtain a first query image whose similarity with object feature information of the query image is greater than or equal to a preset value. Extracting the feature information stored in any one of the first and second storage units, calculating the difference between the location information on the virtual space for the representative or interpolated image corresponding to the extracted feature information and the location information on the virtual space for the query image After generating the movement information according to the difference, it may be characterized in that it is transmitted to the user device.

As an example related to the present invention, the comparator receives angle information about the shooting angle of the query image, maps it with location information in a virtual space, generates location information for the query image, and compares the location information of the query image in the virtual space. It may be characterized in that the movement information for the movement direction and movement distance to match the position information of the representative or interpolated image corresponding to the extracted feature information is generated.

A user device for providing an object recognition service according to an embodiment of the present invention includes a camera unit for photographing an image of an object, an object feature extraction unit for extracting characteristic information about an image photographed from the camera unit, and one or more representatives of the object A first storage unit that matches and stores one or more representative images photographed at an angle and representative characteristic information extracted from each representative image through an object feature extraction unit with object-related information, in a virtual space based on the representative image of the first storage unit Creates one or more interpolated images to be mapped, and stores the interpolated image and interpolation feature information generated by the virtual space mapping unit and the virtual space mapping unit that generate interpolation characteristic information from each interpolated image through the object feature extraction unit by matching object-related information Receives a query image according to the shooting of an object from the second storage unit and the camera unit, extracts features from the query image through the object feature extraction unit, compares it with the first storage unit, and returns to representative characteristic information that matches a preset reference value or more It may include a comparison unit that provides matched object-related information, and provides object-related information matched to interpolation characteristic information that matches the reference value or more by comparing it with the second storage unit when it does not match the reference value or more.

As an example related to the present invention, when there is no matching object through comparison with the first and second storage units, the comparator interworks with the virtual space mapping unit to obtain a first query image whose similarity with object characteristic information of the query image is equal to or greater than a preset value. and extracting the feature information stored in any one of the second storage unit, calculating the difference between the location information on the virtual space for the representative or interpolated image corresponding to the extracted feature information and the location information on the virtual space for the query image It may be characterized in that movement information according to the difference is generated and outputted through an output unit.

A method for providing an object recognition service according to an embodiment of the present invention includes the steps of: an object recognition service providing apparatus receiving a query image for an object from a user device and extracting object characteristic information; A plurality of representative images photographed at an angle and representative characteristic information for each representative image are matched with the object-related information and the first storage unit and the object characteristic information are compared with each other, and the object matched with the representative characteristic information matching more than a preset reference value In the step of providing related information, if it does not match more than the reference value, the object recognition service providing device matches one or more interpolated images mapped in the virtual space based on the representative image and the interpolation characteristic information for each interpolated image is matched with the object related information The method may include comparing the stored second storage unit with the object characteristic information to provide object-related information matched to interpolation characteristic information that matches a preset reference value or more.

As an example related to the present invention, in the object recognition service providing apparatus, when there is no feature information matching the object feature information and the preset reference value or more in the first and second storage units, the similarity to the object feature information of the query image is greater than or equal to a preset value Extracting the characteristic information stored in any one of the first and second storage units, and calculating the difference between the position information in the virtual space for the representative or interpolated image corresponding to the extracted characteristic information and the position information in the virtual space for the query image The method may further include generating movement information according to the difference and then transmitting the generated movement information to the user device.

The object recognition service providing system according to an embodiment of the present invention extracts object characteristic information about a query image received from a user device and a user device that generates and transmits a query image for an object, and represents the object characteristic information for the object One or more representative images photographed at an angle and representative characteristic information extracted from each representative image are matched with object-related information and compared with the stored first storage unit, and object-related information matched with representative characteristic information that matches more than a preset reference value provided, and if it does not match by more than a reference value, one or more interpolated images mapped in the virtual space based on the representative image of the first storage unit and feature information for each interpolated image are compared with the stored second storage unit by matching object-related information and an apparatus for providing an object recognition service providing object-related information matched to interpolation characteristic information matching a preset reference value or more.

The present invention compares the feature information extracted from the query image generated by the user for querying the object by additionally using the feature information extracted from the interpolated image interpolated based on the representative image as well as the actual photographed representative image. It is possible to broaden the recognition range of an object corresponding to

In addition, the present invention provides movement information for adjusting the shooting angle of the representative or interpolated image for the object determined to be similar to the query image when an object similar to the object included in the query image received from the user device cannot be found in the DB. After being provided to the user device, the re-photographed query image is recognized at the shooting angle according to the movement information, so that more accurate object recognition can be achieved for the query image at the same shooting angle as the representative or interpolated image. It can greatly improve the accuracy and reliability of recognition.

1 is a block diagram of an object recognition service providing system according to an embodiment of the present invention;
2 is a detailed configuration diagram of an apparatus for providing an object recognition service according to an embodiment of the present invention;
3 is an exemplary diagram illustrating a virtual space mapping process for a representative image of an apparatus for providing an object recognition service according to an embodiment of the present invention.
4 is an exemplary diagram illustrating generation of an interpolated image mapped on a virtual space of an apparatus for providing an object recognition service according to an embodiment of the present invention;
5 is a block diagram illustrating a process of extracting feature information for an interpolated image of an apparatus for providing an object recognition service according to an embodiment of the present invention.
6 is an exemplary configuration diagram for management of first and second DBs of an apparatus for providing object recognition services according to an embodiment of the present invention;
7 is a configuration diagram for providing object-related information through object recognition from a query image provided by a user of the apparatus for providing object recognition service according to an embodiment of the present invention;
8 is an exemplary configuration diagram for inducing adjustment of a shooting angle of a user device when an object recognition service providing apparatus fails to recognize an object according to an embodiment of the present invention;
9 is a block diagram of a user device for providing an object recognition service according to an embodiment of the present invention;
10 is a flowchart of a method for providing an object recognition service according to an embodiment of the present invention;

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

1 is a block diagram of a system for providing an object recognition service according to an embodiment of the present invention, wherein the system captures an object through a camera, generates a query image for the object, and transmits the image through a communication network. It may include 100 and an object recognition service providing apparatus 200 that receives a query image from the user device 100 through a communication network, recognizes an object from the query image, and provides object-related information.

In this case, various well-known communication methods may be applied to the communication network. In addition, each user device 100 includes a navigation terminal, a tablet PC, a laptop, a smart phone, a personal digital assistant (PDA), and a mobile communication terminal. ) and the like, and the object recognition service providing apparatus 200 may be configured as a server.

The object recognition service providing apparatus 200 extracts feature information about the query image received from the user device 100, and features information extracted from one or more representative images photographed at different representative angles for the object and each representative image. After comparing the characteristic information of the query image with the first DB stored in conjunction with the object, extracts the characteristic information that matches a preset reference value or more, and transmits the object-related information corresponding to the extracted characteristic information to the user device 100 can provide

In addition, the object recognition service providing apparatus 200 provides one or more interpolated images mapped on the virtual space based on the representative image of the first DB when there is no matching characteristic information through comparison of characteristic information between the first DB and the query image. and a second DB in which the characteristic information for each interpolated image is stored in conjunction with the object to extract characteristic information having a similarity greater than or equal to a preset numerical value, and to transmit object-related information corresponding to the extracted characteristic information to the user device 100 can be provided.

In this way, the object recognition service providing apparatus 200 compares the feature information extracted from the query image by additionally using the feature information extracted from the interpolated image interpolated based on the representative image as well as the actual photographed representative image, to obtain the query image. It is possible to widen the recognition range of the corresponding object, thereby greatly increasing the object recognition accuracy.

2 is a detailed configuration diagram of an object recognition service providing apparatus 200 according to an embodiment of the present invention. As shown, an object feature extraction unit 210 , a virtual space mapping unit 220 , and a comparison unit 230 are included. may include

As shown in FIG. 3(a), the object feature extraction unit 210 stores a plurality of representative images photographed for an object from a plurality of representative angles, such as panoramic photographing, and object-related information from the first DB 240 in which matching is stored. By extracting each representative image, it is possible to extract feature information according to a preset algorithm for each representative image.

In this case, the object feature extraction unit 210 may apply various well-known object feature extraction algorithms. Examples of such algorithms include SIFT (Scale Invariant Feature Transform), HOG (Histogram of Oriented Gradient), Haar-like Feature. , Ferns, LBP (Local Binary Pattern), MCT (Modified Census Transform), etc. may be applied.

In addition, the object-related information may include an identification number (ID, etc.) related to the object, product information, manufacturer information, etc., and the representative image contains information about the representative angle taken or is matched with the representative angle in the first DB ( 240) can be stored. Here, the representative angle may mean a photographing angle, and may mean an angle inclined in the X, Y, and Z directions from a preset central point.

Meanwhile, the object feature extraction unit 210 may match and store the representative feature information extracted from the representative image with the object-related information and each representative image in the first DB.

In addition, the virtual space mapping unit 220 may model the representative image in the virtual space by mapping each representative angle with preset position information in the virtual space, as shown in FIG. 3B .

In this case, the virtual space may be a model in which the camera position in the real space according to the photographing angle of the representative image is reflected, and through this, the photographing position of the camera in the three-dimensional space may be mapped and expressed on the virtual space.

As an example, the virtual space mapping unit 220 maps positions in the virtual space corresponding to the representative angles measured when the representative image is captured, and converts the representative angles as X, Y, Z coordinates into position information in the virtual space. Through this, a representative image can be mapped and modeled on a three-dimensional virtual space.

Through this, the virtual space mapping unit 220 determines between the intermediate position information and the representative image corresponding to different adjacent position information with respect to the intermediate position information located between each position information in the virtual space based on the position information in the virtual space. You can create an interpolated image that interpolates.

In this case, the virtual space mapping unit 220 interpolates an interpolated image generated corresponding to the intermediate location information and a representative image corresponding to the location information with respect to other intermediate location information positioned between the intermediate location information and the location information. can create

For example, as shown in FIG. 4 , the virtual space mapping unit 220 maps the representative image photographed at the representative angle of photographing 2 to location information corresponding to the representative angle in the virtual space, and photographing at the representative angle of photographing 3 The representative image can be modeled by mapping it to a position corresponding to the representative angle in the virtual space.

Then, the virtual space mapping unit 220 interpolates the interpolated image between the representative image corresponding to the intermediate position information between the position information corresponding to the representative angle of photographing 2 and the representative angle of photographing 3, respectively, the intermediate position information in the virtual space. It can be created by mapping to . For example, if the representative angle of photographing 2 is 45 degrees as shown, and the representative angle of photographing 3 is 90 degrees, the virtual space mapping unit 220 sets the representative angles of 45 degrees and 90 degrees based on the location information of the virtual space. An interpolated image mapped to position information for an intermediate angle (60 degrees) can be generated.

Through this, the virtual space mapping unit 220 may generate a representative image and an interpolated image each mapped to a plurality of pieces of location information having X, Y, and Z coordinate values in a virtual space.

Next, the virtual space mapping unit 220 may extract and generate interpolation characteristic information through the object characteristic extraction unit 210 for each of the generated interpolated images as shown in FIG. 5 , the extracted interpolation characteristic information, The interpolated image and object-related information may be matched with each other and stored in the second DB 250 .

Through this, as shown in FIG. 6 , object-related information and one or more representative characteristic information PK1 to PK10 may be matched and stored in the first DB 240 , and one or more interpolated images may be stored in the second DB 250 . One or more interpolation feature information (Sub PK1 to Sub PK50) extracted through ? may be matched with object-related information and stored.

Through the configuration as described above, the virtual space mapping unit 220 maps the representative image to the virtual space to generate a plurality of interpolated images interpolating between different adjacent representative images, and then extracts feature information for each interpolated image. By doing so, it is possible to generate characteristic information about an object inferred from various angles and greatly expand the recognition range of the object.

On the other hand, as shown in FIG. 7 , the comparator 230 receives a query image for an object photographed through a camera from the user device 100, information can be provided.

For example, as shown, the comparison unit 230 may extract object characteristic information for an object included in the query image by interworking with the object characteristic extraction unit 220 upon receiving the query image, and set the object characteristic information to the first Compared with the characteristic information stored for each object in the DB 240 , object-related information matched with representative characteristic information matching a preset reference value or more may be extracted from the first DB 240 and transmitted to the user device 100 .

At this time, if the representative characteristic information matching the object characteristic information and the preset reference value or more in the first DB 240 does not exist in the first DB 240 , the comparison unit 230 stores the object characteristic information in the second DB 250 for each object. The interpolation characteristic information and the object characteristic information are compared to each other, and object-related information matched with the interpolation characteristic information matching a preset reference value or more may be extracted and transmitted to the user device 100 .

As such, the comparison unit 230 compares the feature information of the query image with the feature information extracted from the interpolation image for interpolating the representative image even when the object cannot be recognized based on the feature information extracted from the representative image. By doing so, it is possible to significantly increase the object recognition rate even for images taken from angles other than the representative angle.

In addition, as shown in FIG. 8 , the comparison unit 230 compares the object characteristic information for the query image extracted through the object characteristic extraction unit 210 with representative characteristic information and interpolation characteristic information that matches a preset reference value or more. And when none of the second DBs 240 and 250 exist, a representative or interpolated image corresponding to representative or interpolated feature information having a similarity greater than or equal to a preset value is extracted from any one of the first and second DBs 240 and 250 . can do. In this case, the similarity may be the same criterion as the reference value, and accordingly, the comparison unit 230 may extract a representative or interpolated image corresponding to the representative or interpolated characteristic information that matches the object characteristic information of the query image by a preset value or more.

At this time, the comparison unit 230 may receive angle information on the shooting angle of the query image included in the query image or received separately, and calculate the position information corresponding to the query image by mapping the angle information in the virtual space. can do.

In addition, the comparison unit 230 calculates the difference between the location information in the virtual space corresponding to the extracted representative or interpolated image and the location information in the virtual space corresponding to the shooting angle of the query image, and the location information of the query image in the virtual space. In order to match the extracted representative or interpolated image with the position information of the extracted representative or interpolated image, movement information about the movement direction generated according to the calculated difference may be generated and then transmitted to the user device. In this case, the photographing angle may mean an angle inclined in the X, Y, and Z directions from the central point like the representative angle.

In this case, as shown, the movement information may include a distance as well as a direction for moving from the position information of the query image to the position information of the image corresponding to the representative or interpolated feature information whose similarity is greater than or equal to a preset value. In this case, the distance may be an actual distance converted according to a relationship between the virtual space and the real space by measuring the distance between the two location information in the virtual space.

In other words, the comparison unit 230 transmits the movement information to the user device 100, and based on the feature information, the photographing position corresponding to the query image is set to the photographing position corresponding to the representative or interpolated image having a similarity greater than or equal to a preset value. By inducing the user to move the device 100, re-photography may be performed.

Accordingly, the comparison unit 230 receives the re-photographed query image from the user device 100 at the location moved according to the movement information, and then uses the extracted object characteristic information of the first and second DBs 240 and 250 . When representative or interpolation characteristic information equal to or greater than a preset reference value is present in any one of the first and second DBs 240 and 250 compared to the characteristic information, object-related information corresponding to the representative or interpolation characteristic information can be provided. .

As such, when the comparison unit 230 cannot find an object similar to the object included in the query image received from the user device 100 in the first and second DBs 240 and 250, it is determined that the object is similar to the query image. After inducing the user to adjust to the shooting angle of the representative or interpolated image, the re-photographed query image is recognized again, so that more accurate object recognition can be achieved at the same shooting angle as that of the representative or interpolated image. Accuracy and reliability can be greatly improved.

Meanwhile, as shown in FIG. 9 , the above-described object feature extraction unit, virtual space mapping unit, and comparator may be configured in the user device 100 , and the user device includes a camera unit 110 , a sensor unit 150 , and a first It may further include first and second memory units 160 and 170 .

Accordingly, the object feature extraction unit 120 may extract the characteristic information of the captured image from the camera unit 110 that captures the image of the object, and the virtual space mapping unit 130 provides the first DB 240 ) generates an interpolated image mapped on the virtual space based on the representative image from the first memory unit 160 in which the same information is stored, and extracts the feature information from the interpolated image through the object feature extractor 120 and then the interpolated image and feature information may be stored in the second memory unit 170 in association with object-related information.

In addition, the comparison unit 140 receives a query image according to the shooting of the object from the camera unit 110 , extracts a feature from the query image through the object feature extraction unit 120 , and compares it with the first memory unit 160 . After extracting representative characteristic information that matches more than a preset reference value, the corresponding object-related information is provided. can be extracted to provide corresponding object-related information.

In addition, when there is no object matching a preset reference value or more through the first and second memory units 160 and 170 , the comparison unit 140 compares the feature information of the query image with the first and second memory units 160 and 170 . Characteristic information in which the degree of similarity between the characteristic information stored in any one matches a predetermined value or more may be extracted from any one of the first and second memory units 160 and 170 .

Thereafter, the comparator 140 interworks with the virtual space mapping unit 130 based on the sensing information on the shooting angle of the query image obtained through the sensor unit 150 when the query image is captured in the virtual space of the query image. Calculate information and obtain movement information on the movement direction to match the position information in the virtual space for the representative or interpolated image corresponding to the feature information extracted from any one of the first and second memory units 160 and 170 It can be calculated and outputted through the output unit 180 .

Through this, the object recognition rate can be greatly improved by inducing the user to move to a shooting angle corresponding to the representative or interpolated image.

10 is a flowchart of a method for providing an object recognition service according to an embodiment of the present invention. The apparatus 200 for providing an object recognition service receives a query image for an object from the user device 100 (S1) and provides object characteristic information. Extracting (S2), a plurality of representative images photographed from a plurality of representative angles with respect to the object, and the representative characteristic information for each representative image are matched with the object-related information, and the stored first DB 240 and the object characteristic information are compared with each other Thus (S3), it is possible to provide object-related information matched to the representative characteristic information that matches the object characteristic information and the preset reference value or more (S4, S5, S6).

At this time, when there is no representative characteristic information that matches a preset reference value or more according to comparison with the first DB 240 (S4), the object recognition service providing apparatus 200 interpolates one or more interpolations mapped on the virtual space based on the representative image The image and the interpolation characteristic information for each interpolated image are matched with the object-related information, and the stored second DB 250 may be compared with the object characteristic information (S7).

Accordingly, the object recognition service providing apparatus 200 may provide object-related information matched to the object characteristic information and the interpolation characteristic information matching the preset reference value or more (S6, S8, S9).

On the other hand, the object recognition service providing apparatus 200 is similar to the object characteristic information of the query image when there is no characteristic information matching the object characteristic information and the preset reference value or more in the first and second DBs 240 and 250 (S8). Extracting feature information stored in any one of the first and second DBs 240 and 250 equal to or greater than a preset value (S10), and location information and query image in virtual space for a representative or interpolated image corresponding to the extracted feature information After calculating the difference in the location information on the virtual space, movement information according to the difference may be generated and then transmitted to the user device 100 (S11).

The method for providing an object recognition service according to an embodiment of the present invention can be written as a computer program, and codes and code segments constituting the computer program can be easily inferred by a computer programmer in the art. In addition, the corresponding computer program is stored in a computer readable media, and is read and executed by a computer, a user device according to an embodiment of the present invention, an object recognition service providing device, etc., thereby providing an object recognition service method can be implemented.

The information storage medium includes a magnetic recording medium, an optical recording medium, and a carrier wave medium. The computer program for implementing the method for providing an object recognition service according to an embodiment of the present invention may be stored and installed in a user device and an internal memory of the apparatus for providing an object recognition service. Alternatively, an external memory such as a smart card storing and installing a computer program for implementing the method for providing an object recognition service according to an embodiment of the present invention may be mounted on a user device or an apparatus for providing an object recognition service through an interface.

In the above, preferred embodiments according to the present invention have been shown and described. However, the present invention is not limited to the above-described embodiments, and without departing from the gist of the present invention appended in the claims, any person skilled in the art to which the invention pertains will be able to implement various modifications. .

The present invention compares the feature information extracted from the query image generated by the user for querying the object by additionally using the feature information extracted from the interpolated image interpolated based on the representative image as well as the actual photographed representative image. It is possible to broaden the recognition range of objects corresponding to can be applied

100: user device 110: camera unit
120: object feature extraction unit 130: virtual space mapping unit
140: comparison unit 150: sensor unit
160: first memory unit 170: second memory unit
180: output unit 200: object recognition service providing device
210: object feature extraction unit 220: virtual space mapping unit
230: comparison unit 240: first DB
250: second DB

Claims (3)

an object feature extracting unit for extracting feature information from an image;
a first storage unit configured to match one or more representative images photographed at a representative angle with respect to an object and representative characteristic information extracted with respect to each of the one or more representative images through the object characteristic extraction unit to match object-related information;
For each of the one or more representative images, the representative image is modeled in the virtual space by mapping position information in a virtual space corresponding to the representative angle of the representative image, and each of the one or more representative images modeled in the virtual space Generates one or more interpolated images that are mapped between different representative images in the virtual space based on the location information of a virtual space mapping unit that generates information;
a second storage unit configured to match and store the interpolated image and interpolation feature information generated by the virtual space mapping unit with the object-related information; and
After receiving a query image according to the shooting of an object from a user device, extracting a feature from the query image, comparing the first storage unit and the second storage unit, and comparing the first storage unit and the second storage unit, if there is no characteristic information that matches more than a preset reference value, the query image extracts the characteristic information stored in any one of the first and second storage units having a similarity with the object characteristic information of which is equal to or greater than a preset numerical value, and location information in the virtual space for a representative or interpolated image corresponding to the extracted characteristic information and a comparison unit configured to calculate a difference between the location information in the virtual space with respect to the query image, generate movement information according to the difference, and transmit the generated movement information to the user device. ◈Claim 2 was abandoned when paying the registration fee.◈ The method of claim 1,
The comparison unit receives the angle information on the shooting angle of the query image, maps it with the location information in the virtual space, and generates location information for the query image, and extracts the location information of the query image in the virtual space. Object recognition service providing apparatus, characterized in that for generating the movement information on the movement direction and movement distance to match the position information of the representative or interpolated image corresponding to the characteristic information. a camera unit for capturing an image of an object;
an object feature extracting unit for extracting feature information about the captured image from the camera unit;
a first storage unit for matching and storing one or more representative images photographed at one or more representative angles with respect to the object and representative characteristic information extracted from each of the one or more representative images through the object characteristic extraction unit with object-related information;
For each of the one or more representative images, the representative image is modeled in the virtual space by mapping position information in a virtual space corresponding to the representative angle of the representative image, and each of the one or more representative images modeled in the virtual space Generates one or more interpolated images that are mapped between different representative images in the virtual space based on the location information of a virtual space mapping unit that generates information;
a second storage unit configured to match and store the interpolated image generated by the virtual space mapping unit and the interpolated feature information with the object related information; and
After receiving a query image according to the shooting of an object from the camera unit, extracting a feature from the query image, comparing with the first storage unit and the second storage unit, if there is no characteristic information that matches a preset reference value or more, the query Extracting characteristic information stored in any one of the first and second storage units having a similarity with object characteristic information of an image equal to or greater than a preset value, and a position in the virtual space for a representative or interpolated image corresponding to the extracted characteristic information and a comparator configured to calculate a difference between the information and the location information in the virtual space with respect to the query image and generate movement information according to the difference.

Images