JP2019021100A - Image search device, merchandise recognition device, and image search program - Google Patents

Abstract

To provide an image search device, a merchandise recognition device, and an image search program for performing highly accurate image search processing.SOLUTION: An image search device comprises: an image acquisition part; and a processor. The image acquisition part acquires an input image. The processor divides an image as a search object included in the input image into a plurality of small regions, and extracts feature points with a predetermined number as an upper limit for each of the divided small regions, and on the basis of a plurality of feature points in the image as the search object and feature points of a plurality of registration images registered in a dictionary, calculates a degree of similarity between the image as the search object and each of the plurality of registration images stored in the dictionary, and specifies the registration image having the maximum degree of similarity.SELECTED DRAWING: Figure 2

Description

  Embodiments described herein relate generally to an image search device, a product recognition device, and an image search program.

  Conventionally, there are category recognition (general object recognition) and specific object recognition as image recognition techniques using image search processing for searching the same image (the same type of image) as an input image from a database (dictionary). Category recognition is an image recognition technique for recognizing objects having variations in image patterns and shapes of individuals such as animals and plants. The specific object recognition is an image recognition technique for recognizing an object having almost no individual difference. For example, specific object recognition is applied as a technique for recognizing a product type from an image of a product package.

  In the specific object recognition, a dictionary storing image information and feature information of all types of objects to be recognized is prepared. In the specific object recognition, the similarity between the image obtained by photographing the object and the image of each object registered in the dictionary is calculated, and the object having the highest similarity is selected. Similarity calculation methods include template matching for directly comparing images, and a feature point comparison method for comparing local feature points on images. Since template matching has problems such as enormous calculation time and a large dictionary capacity for comparing the entire images, a feature point comparison method is often used in specific object recognition.

  In the feature point comparison method, some feature points are extracted from an image, and the feature amount and attribute of each feature point are calculated. The degree of similarity is determined from the number or ratio of corresponding feature points compared to a list of feature points of each image registered in the dictionary. The feature point is a point that can be uniquely specified locally from the gray value distribution of the image. For example, the maximum point of the second derivative of the density value is used. The feature amount is calculated from a density pattern in the vicinity of the feature point. As a feature point extraction method and a feature amount calculation method, there are methods called SIFT, SURF, ORB, and the like.

  As described above, as a method for searching for the same image, a feature point comparison method for comparing local feature amounts is effective. In the feature point comparison method, the similarity of images is determined using information on only a few feature points of the entire image. Compared to a method for comparing the entire image such as a pattern matching method, the amount of calculation is extremely small.

  However, there are some problems in specific object recognition using image search processing by a feature point comparison method. For example, in product recognition processing as specific object recognition applied to a conventional checkout device (POS terminal) or the like, it may be difficult to identify products having a common design. In the same type of products from the same manufacturer, the same mark such as a trademark is often printed in common for a plurality of products. In addition, products of the same series often use similar designs and common designs. In the conventional feature point comparison method, feature points are extracted in descending order of intensity from the entire image to be recognized. For this reason, in the specific object recognition using the conventional feature point comparison method, it is difficult to accurately identify a product group in which feature points having high strength are extracted from similar or common design parts.

JP 2007-140613 A

  In order to solve the above-described problems, an image search device, a product recognition device, and an image search program capable of performing high-precision image search processing are provided.

  According to the embodiment, the image search device includes an image acquisition unit and a processor. The image acquisition unit acquires an input image. The processor divides an image to be searched included in the input image into a plurality of small regions, extracts feature points with a predetermined number as an upper limit for each of the divided small regions, and a plurality of features in the image to be searched The similarity between the image to be searched and each registered image stored in the dictionary is calculated based on the points and the feature points of a plurality of registered images registered in the dictionary, and the registered image having the maximum similarity is calculated. Identify.

FIG. 1 is an external view illustrating a configuration example of a recognition device as an image search device according to an embodiment. FIG. 2 is a block diagram illustrating a configuration example of a control system of the recognition device as the image search device according to the embodiment. FIG. 3 is a flowchart for explaining the flow of a product recognition process including an image search process in the recognition device as the image search device according to the embodiment. FIG. 4 is a flowchart for explaining the flow of a product recognition process including an image search process in the recognition apparatus as the image search apparatus according to the embodiment. FIG. 5 is a diagram illustrating an example of a product image to be processed by the recognition device as the image search device according to the embodiment. FIG. 6 is a diagram illustrating an example in which the recognition device as the image search device according to the embodiment extracts high-intensity feature points from the entire product image illustrated in FIG. 5. FIG. 7 is a diagram illustrating an example in which the recognition device as the image search device according to the embodiment extracts feature points from each divided region in the entire product image illustrated in FIG. 5. FIG. 8 is a diagram illustrating a configuration example of a dictionary used for image recognition processing in the recognition device as the image search device according to the embodiment.

Hereinafter, embodiments will be described with reference to the drawings.
First, the configuration of a product recognition system including a recognition device as an image search device according to the present embodiment will be described.
FIG. 1 is a diagram illustrating a configuration example of a product recognition system 10.
In the configuration example shown in FIG. 1, the product recognition system 10 includes a product stand 101, a lighting device 102, a camera 103, and a recognition device 105. The product stand 101 is a stand on which products to be recognized are arranged. For example, the product stand 101 may place a basket containing products to be recognized. In addition, it is assumed that one or more products are placed on the basket placed on the product stand 101.

  The illumination device 102 emits light toward the product table 101. The illuminating device 102 irradiates light on the product arranged on the product table 101 from above. The camera 103 captures an image including the product stand 101 in the capturing range. The camera 103 captures an image of an area including a product arranged on the product stand 101. The camera 103 converts the captured image into an image signal and transmits the image signal to the recognition device 105. The illumination device 102 and the camera 103 constitute a photographing device 104.

  The recognition device 105 performs product recognition processing (image recognition processing) including image search processing for searching for similar images. The recognition device 105 executes processing such as product recognition processing based on an image captured by the camera 103. The recognition device 105 can be realized by a computer such as a general Neumann computer. For example, the recognition device 105 outputs information indicating a product such as a product code as a recognition result. The recognition apparatus 105 may output information such as the price of the product with reference to a database that stores the data of the product. In the following embodiment, the recognition device 105 is described as an image search device, but a configuration including the photographing device 104 and the recognition device 105 may be used as the image search device.

Next, the configuration of the recognition device 105 as an image search device will be described.
FIG. 2 is a block diagram illustrating a configuration example of the recognition apparatus 105 according to the embodiment.
As shown in FIG. 2, the recognition apparatus 105 includes a processor 201, a memory 202, a nonvolatile memory 203, an image processing accelerator 204, an input / output interface (I / F) 205, a console 206, a NIC 207, and the like via a system bus 208. Connected through.

  The processor 201 is, for example, a CPU. The processor 201 implements various processing functions by executing programs. The memory 202 is a memory that stores work data. The nonvolatile memory 203 is composed of a rewritable nonvolatile storage device. For example, the non-volatile memory 203 is realized by an HDD, an SSD, or the like. Further, the nonvolatile memory 203 may include a ROM or the like. The nonvolatile memory 203 stores programs and data.

  The processor 201 executes a program (program code) stored in the nonvolatile memory 203. The processor 201 expands the program code stored in the nonvolatile memory 203 in the memory 202, and executes the program code expanded in the memory 202. In the present embodiment, it is assumed that a program for executing processing to be described later is stored in the nonvolatile memory 203 and executed by the processor 201.

  The nonvolatile memory 203 has a dictionary (database) 209 that stores registration information (dictionary information) regarding all types of products to be recognized. The registration information related to each product includes an image (registered image) as a dictionary and feature information used for a product recognition process (image search process) described later. A configuration example of the dictionary 209 will be described in detail later. The non-volatile memory 203 also stores management information of data stored in the dictionary 209. The nonvolatile memory 203 also stores information such as an image (input image) input from the camera 103 and the result of the product recognition process.

  The image processing accelerator 204 is a processing unit that executes image processing. An input / output interface (I / F) 205 is an interface for connecting the imaging apparatus 104 including the camera 103 and the illumination 102. An input I / F 205 is an interface (image acquisition unit) that acquires an input image as an image captured by the camera 103. The processor 201 supplies a control instruction to the camera 103 and the illumination 102 via the input / output I / F 205. The console 206 is used by an operator such as an administrator to input an operation instruction. The NIC 207 is an interface for communicating with an external device. The NIC 207 is, for example, a network interface and communicates with an external device via an external network.

Next, a flow of product recognition processing by the product recognition system 10 according to the present embodiment will be schematically described.
A product to be recognized by the product recognition system 10 is placed on the product stand 101. In the present embodiment, when there are a plurality of products, it is assumed that the plurality of products are arranged on the product table 101 so as not to overlap each other. The imaging device 104 irradiates light on the product stand 101 by the illumination device 102, and takes an image including the image on the product stand 101 by the camera 103. The camera 103 supplies the captured image to the recognition device 105.

  The recognition device 105 acquires an image captured by the camera 103 as an input image. The recognition device 105 cuts out an area for each product in the input image. The recognition device 105 performs feature point extraction and feature amount calculation from an image (product image) cut out in an area for each product. The recognition device 105 compares the feature point information extracted from the product image with the feature point information of each product image registered in the dictionary 209 and searches for images having similar feature point information.

  In other words, the recognition device 105 searches the dictionary 209 for an image product having characteristics similar to the product image as an image search process. It is assumed that the recognition apparatus 105 according to the present embodiment determines whether the images are similar using a local feature amount method. The local feature amount method is determined by the amount of corresponding feature points. The recognition device 105 stores feature point information calculated in advance for all target product images in the dictionary 209 together with the product information as one data set. The dictionary 209 only needs to be accessible by the processor 201 of the recognition apparatus 105 in the product recognition process. For example, the dictionary 209 may be provided in an external device that can communicate via the NIC 207.

  A feature point is a point that can be uniquely specified locally in an image. A feature point is a point uniquely determined in a local region such as a corner portion of an image or a maximum point (maximum or minimum point) of a secondary differential function of image density. The feature point information includes information such as a scale indicating the size and an orientation indicating the direction in addition to the coordinates (x, y) indicating the position. The feature amount of the feature point is defined according to a density pattern near the feature point, for example. As a specific example, the feature amount of the feature point is a value obtained by compressing the density pattern in the vicinity of the feature point into a low-dimensional information amount of several tens to several hundreds.

In the present embodiment, these pieces of information are collectively referred to as feature point attributes (attribute information). Further, the attribute is, for example, a scale representing the orientation and size indicating the position coordinates of the feature point and the direction of the feature point. By having the scale and orientation as the feature point information, it can be expressed with the same feature quantity even between images subjected to rotational similarity transformation.
Note that there are methods such as SIFT, SURF, and ORB as feature point extraction methods and feature amount calculation methods.

  In addition, the local feature method can reduce the data size of the dictionary and the amount of calculation in processing, compared to a method using information of the entire image such as template matching. Further, since the feature amount of the feature point is not changed even by the rotation similarity conversion, it is possible to search from an image taken from an arbitrary angle and distance.

  However, the position, orientation, shooting distance, etc. of the product are different between the image obtained by photographing the product during the recognition process (the product image in the input image) and the image obtained by photographing the product at the time of creating the dictionary. For this reason, the product image (input product image) extracted from the input image and the product image at the time of creating the dictionary (product image for registration, registered image) have different image coordinate systems even for the same product. These coordinate systems are related to the combined similarity transformation of translation, rotation, and enlargement / reduction.

  In the product recognition process (image search process), the recognition apparatus 105 according to the present embodiment obtains a candidate for a similarity transformation matrix that is a coordinate system relationship between the input product image and the product image in the dictionary. The recognizing device 105 obtains a valid transformation matrix from the candidates, obtains the number of corresponding feature points after performing coordinate transformation according to the calculated transformation row example. The recognition apparatus 105 sets the ratio of the number of feature points corresponding to the number of feature points in the dictionary as the similarity. The recognition device 105 sets a product in the dictionary having the maximum similarity and a predetermined threshold value or more as a recognition (search) result that matches the input product image.

Next, the product recognition process (image search process) by the recognition apparatus 105 will be described in detail.
3 and 4 are flowcharts for explaining the flow of the product recognition process (image search process).
The camera 103 captures an image of an imaging range including the product stand 101 on which the product to be recognized is placed. The recognition device 105 acquires a captured image (input image) captured by the camera 103 by the input / output I / F 205 (ACT 11). The processor 201 of the recognition apparatus 105 stores an image acquired from the camera 103 by the input / output I / F 205 as an input image in the nonvolatile memory 203. Here, it is assumed that the input image is a monochrome lightness signal, that is, an image signal in which the lightness of the pixels of the image is expressed by a value from 0 to 255 and a brighter value is larger. However, the input image is not limited to a monochrome lightness signal but may be a color image signal such as RGB.

  When the input image is acquired, the processor 201 of the recognition apparatus 105 extracts individual product areas from the input image (ACT 12). The processor 201 cuts out individual product regions from the image input from the camera 103, and extracts an image of each product (product image). In the present embodiment, the product image is a minimum rectangular region surrounding each product region, and the product image region is expressed by coordinate values of four vertices of a rectangle. However, the representation of the product image is not limited to that represented by the coordinate values of the four vertices of the rectangle. For example, the product image may be expressed in such a manner that a two-dimensional array corresponding to the image area is prepared and the type of the product is represented by the value of the array.

  When at least one product image is extracted from the input image, the processor 201 initializes the variable i (i = 0) (ACT 13). After initializing the variable i, the processor 201 increments the variable i (i = i + 1) (ACT 14). When the variable i is incremented, the processor 201 selects the i-th product image among the extracted product images as an image to be processed.

  When the i-th product image is selected, the processor 201 performs processing for obtaining a feature point in the entire selected product image. In the present embodiment, the processor 201 divides the product image into a plurality of small areas (divided areas), and extracts feature points from the divided small areas.

  In the process of extracting feature points for each divided area, the processor 201 divides the product image into a plurality of areas (ACT 15). FIG. 5 is a diagram illustrating an example in which a product image is divided into nine regions. The processor 201 sets a rectangular image area (entire area) including the entire product from the product image. For example, the processor 201 sets a rectangle circumscribing the product (a circumscribed rectangle) as the entire region of the product. In the example illustrated in FIG. 5, the processor 201 sets a virtual boundary line that is parallel to two opposite sides in the entire rectangular area and that divides the other sides into three. In this case, as shown in FIG. 5, two boundary lines are set vertically and horizontally in the entire area, and the entire area of the product is divided into nine areas (divided areas) by these boundary lines.

  In addition, although the example which divides the whole area | region of goods into 9 was shown in FIG. 5, the division | segmentation method of an area | region is not restricted to this. For example, a divided area divided into 16 may be set by dividing the vertical and horizontal directions into four. Also, the vertical and horizontal division numbers may be different values. Furthermore, the number of divisions may be changed according to the size of the product, and the area of each divided region may be divided so as to be substantially the same regardless of the size of the product.

  When the entire region of the product is divided, the processor 201 specifies the upper limit Np of the number of feature points to be extracted for each divided region (ACT 16). For example, the processor 201 sets a value obtained by dividing the number of feature points to be extracted for one product image by the number of divided areas as the upper limit number Np of feature points in each divided area. For example, when the number of feature points for one product image is 500, the upper limit number Np of feature points of each of the nine divided areas is 500 ÷ 9 = 56.

  When the upper limit number Np of feature points in each divided region is specified, the processor 201 extracts feature points having a predetermined intensity or more in the entire region of the product (ACT 17). Here, the predetermined intensity is a threshold value for excluding those that should not be extracted as feature points. For example, a feature point with low intensity is highly likely to be a non-significant feature point that occurs by chance due to dirt on a product image or noise during photographing. Such feature points should not be extracted even in divided regions where the change in shading is small. Therefore, a minimum threshold value of intensity is set for the feature points, and feature points having an intensity lower than the threshold value are not extracted. Thereby, insignificant feature points can be prevented from being extracted in a divided region where there are no feature points such as regions having almost no shading, or there are only feature points with extremely low intensity.

  When feature points having a predetermined intensity or more are extracted from the entire region, the processor 201 determines a divided region where the extracted feature points exist (ACT 18). For example, the processor 201 determines which divided region each feature point enters from the coordinate value of each extracted feature point. That is, the processor 201 determines to which divided area the feature points extracted from the entire area of the product belong.

  When the divided area to which each feature point belongs is determined, the processor 201 arranges the feature points in descending order for each divided area (ACT 19). When the feature points are arranged in descending order for each divided area, the processor 201 extracts the upper limit number Np feature points in the descending order of the intensity in each divided area (ACT 20). For example, when the upper limit number is 56, the processor 201 extracts 56 feature points in descending order of strength in each divided region. However, when the number of feature points in the divided region is smaller than the upper limit number Np, the processor 201 may extract feature points less than the upper limit number Np.

Through the above processing (ACT 15-20), the processor 201 can divide the product image into a plurality of divided areas and extract the number of feature points up to the upper limit number in each divided area.
FIG. 6 is a diagram illustrating an example when feature points are extracted in descending order of intensity from the entire region of the product image illustrated in FIG. 5. FIG. 7 is a diagram illustrating an example when feature points are extracted in descending order of intensity from each divided region of the product image illustrated in FIG. 5. 6 and 7 illustrate the extracted feature points as circles. The circle indicating the feature points indicates the position of the feature point and the size indicates the scale of the feature point.

  As is apparent from FIGS. 6 and 7, by extracting feature points for each divided region, it is possible to prevent the feature points from concentrating on a specific region in the product image. The feature point has a high strength at a portion where the shading change is large. For this reason, generally, feature points having higher strength are concentrated in the character portion than in the photograph portion. In product images, a specific part such as a product name or logo is often conspicuous, and feature points are easily extracted in such areas. Also, feature points can be easily extracted around a character with high contrast such as a black character on a white background.

  In the example illustrated in FIG. 6, when feature points are extracted from the entire product image in descending order, the feature points are concentrated on a logo character portion at the top of the product image and a character portion with high contrast at the lower right. For example, the same series of products often have similar designs on the package, and the product name and logo are often the same design. That is, it is difficult to accurately identify a plurality of products in a package including the same design part or a similar design if feature points are extracted from the entire product image in descending order of strength. In addition, there is a problem that a product having feature points concentrated on a specific area is difficult to recognize when the specific area is hidden by a seal or another product because most of the feature points are hidden.

  On the other hand, in the example shown in FIG. 7, the extracted feature points are dispersed throughout the product image, and the feature points are also scattered in the lower left photographic area. For this reason, in the product recognition (image search) using the feature points as shown in FIG. 7, the discrimination performance is high even between similar products (images). In addition, the risk that most of the feature points are hidden even if part of the product is hidden is reduced. By extracting feature points for each divided region in the product image, product recognition accuracy is improved, and highly accurate product search processing can be provided.

  When feature points in the i-th product image are extracted, the processor 201 calculates a feature amount (local feature amount) for each extracted feature point (ACT21). The local feature amount may be, for example, a representation of a shading pattern in a region near a feature point with a fixed-length code or a numerical value. The feature amount can be calculated by several methods. In the description of the present embodiment, it is assumed that the ORB feature amount is used. The ORB feature amount is represented by 256-dimensional binary information. As the ORB feature value is closer, the Hamming distance between the two 256-dimensional feature values becomes smaller as the pattern shape is closer, and in the case of the same pattern, the Hamming distance has a minimum value of 0. The Hamming distance is expressed as a total in 256 dimensions of exclusive OR of each bit. For this reason, the ORB feature quantity has a feature that it can be calculated at a very high speed compared to the Euclidean distance (L2 norm) or the Manhattan distance (L1 norm).

  The processor 201 specifies attribute information including an X coordinate, a Y coordinate, a scale, an orientation, and a feature amount for the extracted feature point. When these pieces of information are specified, the processor 201 holds the attribute information for the extracted feature points in the memory 202. Further, the processor 201 also calculates a logarithmic scale obtained by converting the logarithm of the scale into an integer, and stores it in the memory 202. Here, the scale and the logarithmic scale have a one-to-one correspondence and are redundant information. Note that the feature point attribute information described above is calculated by the number of feature points by a process described later. As a result, the memory 202 stores the attribute information of each feature point of the i-th product image as an array.

  When the feature point information (attribute information) is stored in the memory 202, the processor 201 initializes the variable k (k = 0) (ACT22). After initializing the variable k, the processor 201 increments the variable k (k = k + 1) (ACT 23). Then, the processor 201 reads the registration information of the kth product stored in the dictionary 209 (ACT 24). The processor 201 sequentially reads registration information of each product stored in the dictionary 209. For example, it is assumed that the dictionary 209 stores registration information regarding each product to be registered (various products to be recognized) in the form of an array table.

FIG. 8 is a diagram illustrating a configuration example of the dictionary 209.
In the example illustrated in FIG. 8, the dictionary 209 is configured by blocks of the number of products to be registered (number of products). In the dictionary 209 shown in FIG. 8, each block is registration information (dictionary information) for each product. In the example shown in FIG. 8, the number of registered products is Ni, and there are Ni blocks. Each block shown in FIG. 8 includes a product name, a product code (identification information uniquely corresponding to the product), the number of feature points in the product image, and attribute information of each feature point (X and Y coordinates of the feature point, scale, Orientation, feature quantity). For example, as described above, the feature amount is 256-dimensional binary information represented by a 32-digit hexadecimal number.

  However, the feature points registered in the dictionary 209 are obtained by performing the same processing as the above-described processing (processing of ACT 15-20) on the product image for registration. In other words, in the dictionary 209, the product image for registration is divided into predetermined small areas, and information about feature points extracted from each divided area is registered.

  When the information of the kth product in the dictionary is read, the processor 201 compares the feature point information calculated from the product image with the feature point information of the kth product and selects a similar feature point pair group. (ACT25). As similar feature point pairs, pairs of feature points having similar feature quantities are selected. For example, the processor 201 calculates the hamming distance Hd based on the brute force between all feature points of the input product image and all feature points of the kth product (block) in the dictionary. The processor 201 regards it as a similar feature point when the calculated Hamming distance Hd is smaller than a predetermined threshold Thd (for example, about 64). The processor 201 obtains all pairs of feature points in which the Hamming distance Hd is smaller than the predetermined threshold Thd.

  When a similar feature point pair is selected, the processor 201 executes a matrix calculation process for calculating a coordinate transformation matrix for coordinate transformation between the coordinate system of the dictionary and the input product image (ACT 26). That is, the processor 201 calculates a coordinate transformation matrix between the input product image and the k-th product image in the dictionary from the set of feature points. If the product image and the dictionary image are the same, the point on the product image and the point on the dictionary image are represented by a similar coordinate transformation matrix. In this case, the feature points also have a positional relationship that is associated with the coordinate transformation matrix. Here, such a coordinate transformation matrix is obtained.

  For example, if all feature point pairs are corresponding points, it is efficient to obtain a coordinate transformation matrix by a technique such as regression analysis using the least square method. However, in many cases, there are a plurality of similar patterns on the image, or a set of points having similar feature amounts exists by chance, so that all feature point pairs are not represented by the same coordinate transformation matrix. The RANSAC method can be used as a robust transformation matrix estimation method in such a case. Here, it is assumed that a coordinate transformation matrix is obtained using a RANSAC modification method using the likelihood of feature point pairs.

  When the coordinate transformation matrix is calculated, the processor 201 performs coordinate transformation on all feature point pairs selected using the coordinate transformation matrix. The processor 201 calculates a conversion error for the result of the coordinate conversion. The processor 201 counts the number of feature point pairs whose conversion error due to the coordinate conversion matrix is equal to or less than a predetermined value (ACT 27).

  The processor 201 calculates the degree of similarity between the input product image and the k-th product in the dictionary based on the number of feature point pairs for which the counted conversion error is a predetermined value or less (ACT 28). For example, when the coordinate transformation matrix is applied to the position coordinates of the feature points of the dictionary image for all the feature point pairs, the estimated value of the coordinates of the corresponding product image is obtained. The distance between the estimated value of the coordinates and the coordinates of the feature point pair of the product image is obtained. If this distance is less than or equal to a predetermined threshold value, the feature point pair is considered as a feature point that corresponds to the correct position. Of all the feature point pairs, a value obtained by dividing a feature point pair whose distance is equal to or smaller than a threshold value by the number of all feature point pairs is obtained, and this is defined as similarity. It is considered that the higher the similarity is, the closer the product image and the dictionary image are, and the higher the probability that they are the same image.

  After calculating the similarity to the kth product, the processor 201 determines whether the variable k has reached the total number of products registered in the dictionary 209 (the number of products in the dictionary) (ACT27). If the variable k has not reached the number of products in the dictionary 209, that is, if there is a product for which the similarity is not calculated (ACT29, NO), the processor 201 returns to ACT23 and the next product (k in the dictionary 209) = K + 1-th product) is calculated.

  When the variable k reaches the number of products registered in the dictionary 209, that is, when it is determined that the similarity for all the products is calculated (ACT29, YES), the processor 201 recognizes the product based on the similarity for each product. The result (image search result) is determined (ACT30). For example, the processor 201 specifies a product having the maximum similarity. If the maximum similarity is equal to or greater than a predetermined value (identification threshold), the processor 201 sets the product as a product recognition result (image search result) for the input product image. If the maximum similarity is smaller than a predetermined value (identification threshold), the processing result is that there is no corresponding product (image) for the i-th product image.

  In addition, the processor 201 performs the process of ACT 14-30 on all product images extracted from the input image. For this determination, the processor 201 determines whether the variable i has reached the total number of product images (number of product regions) extracted from the input image (ACT 29). If the variable i has not reached the total number of product images extracted from the input image, that is, if there is an input product image for which product recognition processing (image search processing) has not been performed (ACT 31, NO), the processor 201 determines that the ACT 14 Returning to the flowchart, the product recognition process for the next product image (i = i + 1th input product image) is executed. When the variable i reaches the number of product areas, that is, when the product recognition process (image search process) for all product images is completed (ACT31, YES), the processor 201 applies the input image input from the camera 103 to the input image. The product recognition process (image search process) is terminated.

  As described above, in the product recognition system according to the present embodiment, the product image extracted from the input image is divided into small regions, and a predetermined number of feature points are extracted for each divided region. Thereby, it can prevent that the feature point concentrates on a specific area | region in a product image. Since the feature points are dispersed throughout the product image, the discrimination performance is improved even between products having similar or identical product names and logos. Further, even if a part of the product is hidden, most of the feature points are hidden and the risk that the discrimination performance is extremely lowered can be reduced. As a result, the product recognition system (image search device, product recognition device) according to the present embodiment improves the accuracy of product (image) discrimination, and can provide highly accurate product recognition (image search) processing.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

  DESCRIPTION OF SYMBOLS 10 ... Product recognition system, 103 ... Camera, 105 ... Recognition apparatus (image search device, product recognition apparatus), 201 ... Processor, 202 ... Memory, 203 ... Non-volatile memory, 205 ... Input / output interface (image acquisition part), 209 …dictionary.

Claims (6)

An image acquisition unit for acquiring an input image;
Divide the image to be searched included in the input image into multiple small areas,
Extract feature points up to a predetermined number for each divided small area,
Based on a plurality of feature points in the search target image and feature points of a plurality of registered images registered in the dictionary, a similarity between the search target image and each registered image stored in the dictionary is calculated. Identify the registered image with the highest similarity,
An image search apparatus comprising: The processor extracts feature points whose intensities are equal to or higher than a predetermined threshold, with a predetermined number as an upper limit for each of the small regions.
The image search device according to claim 1. An image acquisition unit for acquiring an input image;
Divide the product image included in the input image into multiple small areas,
Extract feature points up to a predetermined number for each divided small area,
Based on a plurality of feature points in the product image and feature points for a plurality of products registered in the dictionary, a similarity between the product and each product stored in the dictionary is calculated, and the maximum similarity is obtained. A product identifying processor,
A product recognition apparatus. The processor extracts feature points whose intensities are equal to or higher than a predetermined threshold, with a predetermined number as an upper limit for each of the small regions.
The product recognition apparatus according to claim 3. The image acquisition unit acquires, as an input image, an image taken by a shooting device of an image of an area where at least one product to be recognized is placed;
The processor extracts an image of a product to be recognized from the input image;
The product recognition apparatus according to claim 3 or 4. On the computer,
Processing to acquire the input image;
A process of dividing an image to be searched included in the input image into a plurality of small areas;
A process of extracting feature points up to a predetermined number for each divided small area;
The similarity between the image to be searched and each registered image stored in the dictionary is calculated based on a plurality of feature points in the image to be searched and feature points of a plurality of registered images registered in the dictionary. Processing,
A process of specifying a registered image that is the maximum similarity among the calculated similarities;
An image search program for executing