JP2011242861A - Object recognition device, object recognition system and object recognition method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an object recognition device, object recognition system and object recognition method for reducing the size of an object model DB to perform collation in accordance with the feature of a place.SOLUTION: A mobile terminal 100 includes a feature point collation unit 107 that collates a feature point of each object written in an object model database and determined based on the position information acquired by a position information acquiring unit 101 with a feature point extracted by a feature point extraction unit 106 and from image data captured by a camera unit 103, and reads information on the object determined to have the highest similarity in feature point from a storage unit 109 (object model DB). A display unit 105 displays the name of the object or the like.

Description

  The present invention relates to an object recognition apparatus, an object recognition system, and an object recognition method for recognizing an object.

  In the field of computer vision, a technique is known in which object recognition is performed by extracting a plurality of feature points from an image acquired by a camera or the like and comparing them with feature points of an object model registered in a database. As algorithm for feature point extraction used in such object recognition technology, SIFT (Scale-Invariant Feature Transform) and SURF (Speeded Up Robust Features) are known. The details of SIFT are described in, for example, US Pat. No. 6,711,293 (Patent Document 1). In this SIFT, the types of objects that can be recognized increase as the number of object models registered in the database increases.

U.S. Pat. Special table 2009-535680

  Thus, in order to realize the object recognition processing, the object model DB needs to be stored in the memory in advance. However, in general, memory has a limited storage capacity, and there is a demand for reducing the size of the object model database (DB) as much as possible in order to improve the object recognition speed.

  On the other hand, an object existing in real space is biased depending on its location. That is, for example, the ratio of stationery is high in stationery stores, and the ratio of natural objects is high in suburban forest parks. It is. However, the conventional object recognition is not aware of the object bias due to such location characteristics.

  Therefore, in order to solve the above-described problem, the present invention reduces the size of the object model DB and can perform collation processing according to the location characteristics, an object recognition system, and an object recognition It aims to provide a method.

  In order to solve the above-described problems, an object recognition apparatus of the present invention is acquired by a photographing unit that acquires image data, a position information acquisition unit that acquires position information of the object recognition device, and the position information acquisition unit. Feature point matching means for matching feature points of each object stored in the object model database determined based on position information with feature points extracted from image data photographed by the photographing means; and the feature points Presenting means for presenting information relating to an object whose feature points are determined to be most similar by the matching means.

  The object recognition method of the present invention is determined based on the photographing step for acquiring image data, the position information acquisition step for acquiring position information of the object recognition device, and the position information acquired from the position information acquisition step. A feature point matching step that matches a feature point of each object stored in the object model database with a feature point extracted from the image data captured in the imaging step, and the feature point matching step is the most characteristic point. A presenting step for presenting information on an object determined to be similar.

  According to the present invention, the feature point in each object stored in the object model database determined based on the acquired position information is collated with the feature point extracted from the captured image data, and the feature point is Information about the object judged to be the most similar is presented. Thus, an appropriate object model database can be used according to the position, and object recognition can be performed with high accuracy using an object model database with a small capacity. Furthermore, object recognition processing can be performed in a relatively short time by using a small-capacity object model database.

  Moreover, the object recognition device of the present invention includes azimuth information acquisition means for acquiring azimuth information indicating the direction in which the object recognition device is facing, and the feature point matching means acquires the azimuth information in addition to position information. Preferably, the feature points of each object stored in the object model database determined based on the orientation information acquired by the means are matched with the feature points extracted from the image data photographed by the photographing means.

  According to this invention, in addition to the position information, by comparing the feature points of each object stored in the object model database determined based on the orientation information with the feature points extracted from the image data, In addition, an appropriate object model database corresponding to the direction can be used, and object recognition can be performed with high accuracy using an object model database with a small capacity.

  Further, the object recognition apparatus of the present invention is replied in response to the position information from the transmission means for transmitting the position information acquired by the position information acquisition means to a server that stores a plurality of object model databases. Receiving means for receiving an object model database, and the feature point matching means preferably performs matching processing based on the object model database received by the receiving means.

  According to the present invention, the acquired position information is transmitted to a server storing a plurality of object model databases, the object model database returned according to the position information from the server is received, and the received object model database is stored in the received object model database. Based on this, a matching process is performed. Thus, the object recognition apparatus does not need to have all of the object model database, and can perform object recognition processing with high accuracy using a low-capacity memory.

  The object recognition system of the present invention is an object recognition system including the object recognition device and a server that communicates with the object recognition device, and the server describes feature point data indicating feature points of each object. Based on the position information acquired by the position information acquisition means from the object model database, a determination means for determining an object model database to be collated in the object recognition device; and an object model database determined by the determination means Transmitting means for transmitting to the object recognition device.

  The object recognition method of the present invention is an object recognition method in an object recognition system comprising an object recognition device and a server that communicates with the object recognition device, wherein the object recognition device obtains image data; and A position information acquisition step of acquiring position information of the object recognition device; a feature point in each object stored in an object model database determined based on the position information acquired from the position information acquisition step; and the imaging step A feature point matching step that matches feature points extracted from the image data captured by the step, and a presentation step that presents information about an object that is determined to be the most similar feature point by the feature point matching step. The server includes an object model database describing feature point data indicating feature points of each object. Based on the position information acquired by the position information acquisition step, a determination step of determining an object model database to be collated in the object recognition device, and the object model database determined by the determination step And a transmission step of transmitting to the object recognition device.

  According to the present invention, in the server, based on the object model database describing the feature point data indicating the feature point of each object, based on the acquired position information, the object model database to be collated in the object recognition device And the determined object model database can be transmitted to the object recognition device. Accordingly, an object model appropriately determined according to the position of the object recognition device can be transmitted to the object recognition device, and the object recognition device can perform object recognition processing with high accuracy.

  In the server of the object recognition system of the present invention, the determination unit may determine an object model database based on the orientation information of the object recognition device in addition to the position information acquired by the position information acquisition unit. preferable.

  According to the present invention, by determining the object model database based on the azimuth information of the object recognition device in addition to the position information, an appropriate object model database corresponding to the position and azimuth can be used. Object recognition processing can be performed with high accuracy on the apparatus side with a small memory capacity.

  In the server of the object recognition system of the present invention, it is preferable that the determining means determines one object model database from a plurality of object model databases.

  According to the present invention, by determining one object model database from a plurality of object model databases, the object recognition process can be performed systematically using a database with a small capacity.

  In the server of the object recognition system of the present invention, the determining unit generates one object model database by combining several object model databases among a plurality of object model databases, and Preferably, an object model database is determined.

  According to the present invention, by combining several object model databases among a plurality of object model databases, it is possible to generate one object model database and determine one object model database to be collated, An appropriate object model database can be constructed, and object recognition processing can be performed with high accuracy with a small memory capacity on the object recognition apparatus side.

  In the server of the object recognition system of the present invention, the determination unit selects a plurality of object models from the entire one object model database and generates one object model database. Preferably, an object model database is determined.

  According to the present invention, by selecting a plurality of object models from one entire object model database and generating one object model database, it is possible to determine one object model database to be collated, which is necessary. It is possible to construct an object model database consisting only of simple object models. Therefore, the object recognition apparatus can accurately perform object recognition processing with a small memory capacity.

  Further, the object recognition system of the present invention is an object recognition system comprising an object recognition device and a server that communicates with the object recognition device, wherein the object recognition device includes a photographing means for acquiring image data, and an object recognition device. Position information acquisition means for acquiring position information, presentation means for presenting information on an object transmitted from the server, and the server from an object model database describing feature point data indicating feature points of each object, Based on the position information acquired by the position information acquisition means, a determination means for determining an object model database to be collated in the object recognition device, and a position information acquired by the position information acquisition means. Feature points of each object stored in the object model database and the image taken by the imaging means A feature point matching unit that matches feature points extracted from image data, a transmission unit that transmits information about an object that is judged to be most similar by the feature point matching unit to the object recognition device, and It has.

  According to the present invention, the position information and feature point data acquired by the object recognition device are transmitted to the server, and the server uses the object model database determined according to the position information to transmit the feature points. A verification process can be performed on the data. Therefore, accurate object recognition processing can be realized using an appropriate object model database corresponding to the position while reducing the processing load of the object recognition apparatus.

  According to the present invention, an appropriate object model database can be used according to the position, and object recognition can be performed with high accuracy using an object model database with a small capacity. Furthermore, object recognition processing can be performed in a relatively short time by using a small-capacity object model database.

It is a system configuration figure showing the composition of the object recognition system of this embodiment. 2 is a block diagram showing a configuration of a mobile terminal 100. FIG. 2 is a block diagram showing a hardware configuration of a mobile terminal 100. FIG. It is a block diagram which shows the structure of an object model database (DB) server. It is explanatory drawing which shows an example of the data format of a zone definition database (DB). It is explanatory drawing which shows an example of the data format of an object model database (DB). It is a flowchart of the object recognition operation | movement of an object recognition system. It is a block diagram which shows the function of the portable terminal 100a in a modification. It is a block diagram which shows the function of the portable terminal 100b in a modification. It is a block diagram which shows the function of the object model database (DB) server 200b in a modification.

  Embodiments of the present invention will be described with reference to the accompanying drawings. Where possible, the same parts are denoted by the same reference numerals, and redundant description is omitted.

  FIG. 1 is a system configuration diagram of an object recognition system of the present embodiment. As shown in FIG. 1, the object recognition system includes a mobile terminal 100 and an object model DB server 200. The portable terminal 100 can access the object model DB server 200 arranged on the network via the base station 120. In FIG. 1, when the mobile terminal 100 captures the object 150, the mobile terminal 100 collates the feature point with the feature point of the object model DB stored in the object model DB server 200, and displays a message regarding the object 150. Can be displayed. Hereinafter, the portable terminal 100 and the object model DB server 200 will be described in more detail.

  FIG. 2 is a block diagram illustrating a functional configuration of the mobile terminal 100. As shown in FIG. 2, the mobile terminal 100 includes a position information acquisition unit 101 (position information acquisition unit), a direction sensor 102 (direction information acquisition unit), a camera unit 103 (imaging unit), an input unit 104, and a display unit 105. (Presentation means), a feature point extraction unit 106 (feature point extraction unit), a feature point collation unit 107 (feature point collation unit), a communication unit 108, and a storage unit 109.

  FIG. 3 is a hardware configuration diagram of the mobile terminal 100. As shown in FIG. 3, the portable terminal 100 shown in FIG. 2 physically includes a CPU 11, a RAM 12 and a ROM 13 that are main storage devices, an input device 14 such as a keyboard and a mouse that are input devices, and an output such as a display. The computer 15 includes a device 15, a communication module 16 that is a data transmission / reception device such as a network card, an auxiliary storage device 17 such as a hard disk, and the like. Each function described in FIG. 2 has the input device 14, the output device 15, and the communication module 16 under the control of the CPU 11 by reading predetermined computer software on the hardware such as the CPU 11 and the RAM 12 shown in FIG. 3. This is realized by reading and writing data in the RAM 12 and the auxiliary storage device 17. Hereinafter, each functional block will be described based on the functional blocks shown in FIG.

  The position information acquisition unit 101 is a part that acquires significance information indicating the current position of the mobile terminal 100 using GPS (Global Positioning System).

  The direction sensor 102 is a part that detects the direction in which the mobile terminal 100 is facing, and detects the shooting direction of the camera unit 103 of the mobile terminal 100 using, for example, a geomagnetic sensor.

  The camera unit 103 is a part that images an actual space, and image data obtained by the imaging is displayed on the display unit 105.

  The input unit 104 is a part that receives an operation by the user, and is configured by a button such as a numeric keypad.

  The display unit 105 is a part that superimposes and displays the image data captured by the camera unit 103 and information (for example, the name of the object) related to the object as a result of collation by the feature point collation unit 107.

  The feature point extraction unit 106 is a part that extracts the feature points of the entire image data photographed by the camera unit 103 or the feature points of the objects included in the image data. For example, the feature point extraction unit 106 uses SIFT or SURF as an algorithm for feature point extraction. When SIFT is used, the feature point extraction unit 106 extracts a predetermined number (for example, 114) of key points, and shows coordinates (X coordinate, Y coordinate), scale, orientation, and features at each key point. A 128-dimensional vector is extracted. In the key point extraction process, the extreme value search in the scale space is performed by performing the DoG process, and the position and scale of the key point can be determined. Then, 128 pieces of luminance gradient information in the peripheral area (within the range of the scale) of the coordinates of the key point are obtained and acquired as a 128-dimensional vector.

  Similarly, in SURF, a key point and a vector representing the feature value at the key point are acquired, and collation processing based on the information can be performed.

  The feature point matching unit 107 is a part that performs a matching process between the feature points extracted by the feature point extraction unit 106 and the feature points described in the object model DB stored in the storage unit 109. When SIFT is used, the feature point matching unit 107 performs matching processing by comparing 128-dimensional vectors at the respective feature points. Then, the feature point matching unit 107 votes a predetermined point to an object model having a feature vector that is the nearest among a plurality of object models described in the object model DB. For example, the feature point matching unit 107 votes for an object model having the closest feature point for each of 100 feature points extracted from the target image, for example. Here, the nearest neighbor means that the Euclidean distance is the shortest. When voting has been completed up to the last feature point, the object model having the largest number of votes, that is, the most similar feature point is determined to be an object recognized from the image data. Then, the feature point matching unit 107 reads the name of the corresponding object from the object model DB and outputs it to the display unit 105.

  The feature point matching unit 107 can also deal with general object recognition (Classification) and specific object recognition (Identification). For example, in the object model DB, feature point data of “car”, which is a generic term for abstract objects, is described, and moreover, when specific feature data of a specific vehicle type is described, feature point matching is performed. The unit 107 may read a general name such as “automobile” depending on the collation result, or may read a more detailed car model name. In this case, the feature point data for the “automobile” is composed of rough feature point data as compared with the feature point data for specifying a specific vehicle type. Further, in the feature point matching unit 107, when a predetermined number of votes is not obtained in each object model and a predetermined number of votes is obtained in the object model for general object recognition, The name “automobile” that is an object model for general object recognition may be read from the object model DB.

  The communication unit 108 is a part that performs communication processing for connecting to a network, and the mobile terminal 100 is a part that executes communication processing by wireless communication. For example, the communication unit 108 transmits the position information of the mobile terminal 100 and the direction information indicating the shooting direction to the object model DB server 200, and the object model DB server 200 determines the position information and the direction information according to the position information and the direction information. The obtained object model DB is received from the object model DB server 200.

  The storage unit 109 is a part that stores the object model DB received by the communication unit 108 from the object model DB server 200.

  The mobile terminal 100 configured as described above acquires an object model DB corresponding to position information from the object model DB server 200, and performs a matching process of feature points using the object model DB, so that an appropriate object is obtained. Recognition processing can be performed.

  Next, the configuration of the object model DB server 200 will be described. FIG. 4 is a block diagram showing a functional configuration of the object model DB server 200. As shown in FIG. 4, the object model DB server 200 includes a communication unit 201 (transmission unit, reception unit), a zone identification unit 202, a zone definition DB 203, an object model DB selection unit 204 (determination unit), and an object model DB. The storage unit 205 (object model DB) is included. This object model DB server 200 is also composed of a computer system, similar to the portable terminal 100, and can execute the functions described above by performing operations according to the program. Hereinafter, the components will be described.

  The communication unit 201 is a part that executes communication processing with the mobile terminal 100. For example, the communication unit 201 receives position information and orientation information transmitted from the mobile terminal 100, and determines an object model determined according to the position information and orientation information. This is the part that transmits the DB to the mobile terminal 100.

  The zone specifying unit 202 is a part that specifies a zone corresponding to the position information transmitted from the mobile terminal 100 with reference to the zone definition DB 203. In addition to the method of specifying the zone based on the actual position of the mobile terminal 100, the zone specifying unit 202 captures the zone based on the range where the mobile terminal 100 is actually located and the shooting direction. May be specified. For example, the zone specifying unit 202 can identify the zone based on the information because the shooting range is known based on the position information and the direction information transmitted from the mobile terminal 100. Note that there may be a plurality of zones according to the position of the mobile terminal 100, such as a plurality of zones overlapping each other. In this case, a plurality of zones may be specified, or one zone may be specified by determining which zone is suitable based on the azimuth information.

  The zone definition DB 203 is a part describing information defining a zone. For example, the zone is defined by the center position and the range radius. It should be noted that the zone is not limited to being defined by a circle, but may be defined by a rectangle, a zone may be defined by a triangle, or other polygon.

  FIG. 5 shows a specific example of the zone definition DB stored in the zone definition DB 203. As shown in FIG. 5, the zone definition DB describes center position information indicating the center position of the zone, its range (radius), zone ID for identifying the zone, and zone name in association with each other. ing.

  The object model DB selection unit 204 selects one corresponding zone from the plurality of object model DBs stored in the object model DB storage unit 205 based on the position information received from the mobile terminal 100, and The object model DB corresponding to the zone is extracted from the object model DB storage unit 205 and distributed to the mobile terminal 100 via the communication unit 201.

  In addition to the method of selecting one object model DB corresponding to one zone, the object model DB selection unit 204 selects several object model DBs with the highest number of votes from a plurality of object model DBs, and these May be combined into one object model DB, or a part of the object model may be extracted from the entire object model DB to generate a new one object model DB.

  The object model DB storage unit 205 is a part that stores an object model DB that describes an object model indicating a feature point of each object. For example, as shown in FIG. 6, the object model DB describes an object model including an object model ID, an object name, and an object feature point. When the feature point of the object is extracted by the above-described SIFT, the feature point is composed of the coordinates of the key point and a 128-dimensional vector of the key point.

  Further, the object model DB storage unit 205 stores an object model DB for each zone, and an object model for each zone such as an object model DB 205a corresponding to the zone 001, an object model DB 205b corresponding to the zone 002, and the like. DB is stored.

  In FIG. 4, the object model DB corresponding to the zone is individually prepared in advance, but a plurality of subunits for configuring a part of the object model DB are prepared, and the object model DB selection unit 204 is prepared. Depending on the zone, the subunits may be combined as appropriate to construct one object model DB.

  Also, in the object model DB storage unit 205, a master DB in which all object models are registered is constructed, and the object model DB selection unit 204 extracts one object model corresponding to one zone, so that one zone is extracted. An object model DB corresponding to the above may be dynamically generated. For example, in the object model DB, attribute information such as an object category is also described in association with the object model. On the other hand, in the zone definition DB 203, attribute information is added for each zone. The corresponding object model may be extracted based on the attribute information, and the object model DB may be constructed. For example, when a user enters a forest, an object model DB for recognizing an object existing in the forest is constructed by extracting an object model to which attribute information such as “insect” and “plant” is added. You may make it do.

  Next, processing of the mobile terminal 100 and the object model DB server 200 configured as described above will be described. FIG. 7 is a flowchart showing processing of the mobile terminal 100 and the object model DB server 200.

  First, when a process activation operation for object recognition is performed, the position information acquisition unit 101 is activated, position information is acquired (S101), the camera unit 103 is activated, shooting processing is performed, and live view is performed. As a display, the image data photographed by the camera unit 103 is displayed on the display unit 105 (S102). When the object recognition menu is entered (S103; YES), the position information acquired by the position information acquisition unit 101 is transmitted to the object model DB server 200 by the communication unit 108 (S104).

  In the object model DB server 200, position information is received by the communication unit 201 (S201), and a zone to which the position of the mobile terminal 100 belongs is specified by the zone specifying unit 202 from the position information (S202). The object model DB corresponding to the identified zone is transmitted to the portable terminal 100 by the communication unit 201 (S203).

  In the portable terminal 100, the object model DB transmitted from the object model DB server 200 is received by the communication unit 108 (S105). When an object recognition button (arranged in the input unit 104) is pressed to perform object recognition processing on image data captured by the camera unit 103 by a user operation (S106: YES), the feature The point extraction unit 106 performs feature point extraction processing on the image data (S107). The feature points extracted here are subjected to feature point matching processing by the feature point matching unit 107 with reference to the received object model DB (S108).

  As a result of the feature point matching process, the object model having the closest feature point is identified by the feature point matching unit 107, the name of the identified object model is extracted, and the name is displayed by the display unit 105. The image data is superimposed and displayed (S110).

  Thus, the object model DB is determined according to the position of the mobile terminal 100, and the mobile terminal 100 can execute the object recognition process using the determined object model DB. Thereby, the object model DB corresponding to the position can be used, and an efficient object recognition process can be performed using a database with a small capacity.

  Next, a modified example will be described. FIG. 8 is a block diagram illustrating functions of the mobile terminal 100a according to the modification. The mobile terminal 100a in this modification can perform an efficient object recognition process by switching the object model DB according to the position without using the object model DB server 200.

  The mobile terminal 100a includes a position information acquisition unit 101, an orientation sensor 102, a camera unit 103, an input unit 104, a display unit 105, a feature point extraction unit 106, a feature point matching unit 107, a communication unit 108, and a storage unit 109. Furthermore, a zone specifying unit 202a, a zone definition DB 203a, an object model DB selection unit 204a, and an object model DB storage unit 205x are provided.

  According to the mobile terminal 100a in this modification, the zone specifying unit 202a specifies a zone based on the position information acquired by the position information acquiring unit 101, and the object model DB selecting unit 204a The object model DB is determined, and the feature point matching unit 107 can perform the object recognition process by performing the matching process using the determined object model DB.

  For each component, the zone identification unit 202a, the zone definition DB 203a, the object model DB selection unit 204a, and the object model DB storage unit 205x are respectively the zone identification unit 202, the zone definition DB 203, and the object model DB selection in the above-described embodiment. This has the same function as the unit 204 and the object model DB storage unit 205, and a description thereof will be omitted.

  Further, as a further modification, a portable terminal 100b and an object model DB server 200b as shown in FIGS. 9 and 10 can be considered. FIG. 9 is a block diagram illustrating functions of the mobile terminal 100b in the modification, and FIG. 10 is a block diagram illustrating functions of the object model DB server 200b in the modification.

  As illustrated in FIG. 9, the mobile terminal 100 b includes a position information acquisition unit 101, an orientation sensor 102, a camera unit 103, an input unit 104, a display unit 105, a feature point extraction unit 106, a communication unit 108, and a storage unit 109. It is configured.

  Also, as shown in FIG. 10, the object model DB server 200b includes a feature point in addition to the communication unit 201, the zone specifying unit 202, the zone definition DB 203, the object model DB selection unit 204, and the object model DB storage unit 205. A collation unit 107a is provided.

  Then, the mobile terminal 100b transmits the feature point data extracted by the feature point extraction unit 106 to the object model DB server 200b together with the position information.

  In the object model DB server 200b, the zone specifying unit 202 specifies a zone based on the position information, and the object model DB selecting unit 204 determines the object model DB based on the zone. Then, using the object model DB determined here, the feature point matching unit 107a performs matching processing of feature point data transmitted from the mobile terminal 100b, and performs object recognition processing. The name of the object, which is information about the object obtained as a result of the collation, is transmitted to the mobile terminal 100 b and displayed on the display unit 105. In this way, the processing load on the mobile terminal 100 can be reduced.

  Next, operational effects of the mobile terminal 100 and the object model DB server 200 of the present embodiment will be described.

  In the mobile terminal 100 of the present embodiment, the feature point matching unit 107 includes a feature point and a feature point in each object described in the object model DB determined based on the position information acquired by the position information acquiring unit 101. The feature point extracted from the image data photographed by the camera unit 103 extracted by the extraction unit 106 is collated. The object model DB may be determined in the object model DB server 200 as a collation target in the mobile terminal 100, or may be determined in the mobile terminal 100 as in a modified example.

  Then, information on the object whose feature points are determined to be most similar by the feature point matching unit 107 is read from the storage unit 109 (object model DB), and for example, the display unit 105 displays the name of the object. . Accordingly, an appropriate object model DB can be used according to the position, and object recognition can be performed with high accuracy using an object model DB with a small capacity. Furthermore, object recognition processing can be performed in a relatively short time by using a small-capacity object model DB.

  Note that the feature point matching unit 107 may be provided in the object model DB 200b and perform the matching process in the object model DB 200b as described in the modification.

  In addition, the mobile terminal 100 of this embodiment includes an orientation sensor 102 for detecting the orientation of the mobile terminal 100, and the feature point matching unit 107 is an object model determined based on the orientation information in addition to the position information. By comparing the feature points of each object stored in the DB with the feature points extracted from the image data, an appropriate object model DB corresponding to the position and orientation can be used, and an object model with a small capacity Object recognition can be performed with high accuracy in the DB.

  In the mobile terminal 100 of the present embodiment, the communication unit 108 transmits the position information acquired by the position information acquisition unit 101 to the object model DB server 200 including a plurality of object model DBs 205a to 205d, and the object model The object model DB determined according to the position information is received from the DB server 200, and the feature point matching unit 107 performs matching processing based on the received object model DB. As a result, the object recognition apparatus does not need to have all of the object model DB, and can perform object recognition processing with high accuracy using a low-capacity memory.

  In the object model DB server 200 in the object recognition system of the present embodiment, the communication unit 201 receives position information from the mobile terminal 100, and the object model DB selection unit 204 is a feature point indicating a feature point of each object. With reference to the object model DB storage unit 205 describing the data, the object model DB to be collated in the object recognition apparatus is determined based on the acquired position information. Then, the object model DB selection unit 204 can transmit the determined object model DB to the mobile terminal 100 via the communication unit 201. Accordingly, an object model appropriately determined according to the position of the object recognition device can be transmitted to the object recognition device, and the object recognition device can perform object recognition processing with high accuracy.

  In the present embodiment, the zone specifying unit 202 determines a zone from the position of the mobile terminal 100, and the object model DB selection unit 204 is stored in the object model DB storage unit 205 with reference to the zone definition DB 203. One object model DB corresponding to the zone is selected from the existing object model DBs 205 a to 205 d and transmitted to the mobile terminal 100 via the communication unit 201.

  In the object recognition system of the present embodiment, the communication unit 201 receives the orientation information of the mobile terminal 100 in addition to the position information, and the zone identification unit 202 and the object model DB selection unit 204 receive the orientation information. By considering one object model DB, an appropriate object model DB corresponding to the position and orientation can be used, and the object recognition process is performed with high accuracy and low memory capacity on the object recognition device side. be able to.

  Note that the object model DB selection unit 204 generates one object model DB by combining several object model DBs among the plurality of object model DBs 205a to 205d, and selects one object model DB to be collated. Therefore, an appropriate object model DB can be constructed, and the object recognition process can be performed with high accuracy with a small memory capacity on the object recognition apparatus side.

  In addition, the object model DB selection unit 204 selects a plurality of object models from the entire one object model DB and generates one object model DB, thereby determining one object model DB to be collated. Thus, it is possible to construct one object model DB consisting only of necessary object models. Therefore, the object recognition apparatus can accurately perform object recognition processing with a small memory capacity.

DESCRIPTION OF SYMBOLS 100, 100a, 100b ... Portable terminal, 101 ... Position information acquisition part, 102 ... Direction sensor, 103 ... Camera part, 104 ... Input part, 105 ... Display part, 106 ... Feature point extraction part, 107 ... Feature point collation part, 107a ... feature point matching unit 108 ... communication unit 109 ... storage unit 120 ... base station 150 ... object 200, 200b ... object model DB server 201 ... communication unit 202, 202a ... zone specifying unit 203 203a ... Zone definition DB, 204, 204a ... Object model DB selection unit, 205, 205x ... Object model DB storage unit.

Claims (11)

Photographing means for acquiring image data;
Position information acquisition means for acquiring position information of the mobile terminal;
The feature points of each object stored in the object model database determined based on the position information acquired by the position information acquisition unit are compared with the feature points extracted from the image data captured by the imaging unit. Feature point matching means;
Presenting means for presenting information relating to an object whose feature points are determined to be most similar by the feature point matching means;
An object recognition apparatus comprising: Azimuth information acquisition means for acquiring azimuth information indicating the direction in which the object recognition device is facing;
The feature point matching means captures the feature points of each object stored in the object model database determined based on the orientation information acquired by the orientation information acquisition means, in addition to the position information, and the imaging means. The object recognition apparatus according to claim 1, wherein a feature point extracted from the obtained image data is collated. Transmitting means for transmitting the position information acquired by the position information acquiring means to a server that stores a plurality of object model databases;
Receiving means for receiving an object model database returned according to the position information from the server,
The object recognition apparatus according to claim 1, wherein the feature point matching unit performs a matching process based on the object model database received by the receiving unit. In an object recognition system provided with the object recognition device according to any one of claims 1 to 3 and a server which communicates with the object recognition device.
The server
Determination to determine an object model database to be collated in the object recognition device based on position information acquired by the position information acquisition unit from an object model database describing feature point data indicating feature points of each object Means,
Transmitting means for transmitting the object model database determined by the determining means to the object recognition device;
An object recognition system comprising: In the server,
5. The object recognition system according to claim 4, wherein the determination unit determines an object model database based on azimuth information of the object recognition device in addition to the position information acquired by the position information acquisition unit. . In the server,
The determining means includes
6. The object recognition system according to claim 4, wherein one object model database is determined from a plurality of object model databases. In the server,
The determining means includes
6. The object model database is generated by combining several object model databases among a plurality of object model databases, and one object model database to be collated is determined. The object recognition system described. In the server,
The determining means includes
6. The object model database to be collated is determined by selecting a plurality of object models from the entire object model database and generating the object model database. The object recognition system described. In an object recognition system comprising an object recognition device and a server communicating with the object recognition device,
Photographing means for acquiring image data;
Position information acquisition means for acquiring position information of the object recognition device;
Determining means for determining an object model database to be a target for matching feature points based on position information acquired by the position information acquiring means from an object model database describing feature point data indicating the feature points of each object; ,
The feature points of each object stored in the object model database determined based on the position information acquired by the position information acquisition unit are compared with the feature points extracted from the image data captured by the imaging unit. Feature point matching means;
Presenting means for presenting information relating to an object whose feature points are determined to be most similar by the feature point matching means;
An object recognition system comprising: A shooting step for acquiring image data;
A location information acquisition step for acquiring location information of the terminal;
The feature points of each object stored in the object model database determined based on the position information acquired in the position information acquisition step are collated with the feature points extracted from the image data captured in the imaging step. A feature point matching step;
A presenting step of presenting information about an object that is determined to have the most similar feature points by the feature point matching step;
An object recognition method comprising: In an object recognition method in an object recognition system comprising an object recognition device and a server that communicates with the object recognition device,
A shooting step for acquiring image data;
A position information acquisition step of acquiring position information of the object recognition device;
Determination based on the object model database describing the feature point data indicating the feature points of each object, and determining the object model database to be the target of feature point matching based on the position information acquired by the position information acquisition step Steps,
The feature points of each object stored in the object model database determined based on the position information acquired in the position information acquisition step are collated with the feature points extracted from the image data captured in the imaging step. A feature point matching step;
A presenting step of presenting information about an object that is determined to have the most similar feature points by the feature point matching step;
An object recognition method comprising:

Images