JP5850812B2 - Article collation apparatus, article collation method, and article collation program - Google Patents

Description

  The present invention relates to an article collation apparatus that collates articles in images obtained by photographing a plurality of articles with previously stored articles, an article collation method using the article collation apparatus, and an article collation program.

  Traditionally, retailers that sell products to end consumers check the actual inventory quantity and understand the difference between the inventory quantity listed in the inventory management table and the actual inventory quantity. For this reason, product verification work is regularly performed. For example, for all products in the store, read the barcode attached to the product, obtain the product number, create the matching inventory table by counting the number for each product based on the acquired product number, and store it in advance Difference information with the actual inventory quantity is obtained by comparing with the managed inventory table.

  However, with the recent diversification of products, there are many cases where a large number of products exist in large stores. For this reason, it is a heavy burden on the retailer to read the bar code of each product and collate the product for inventory management because it requires cost and work time. For this reason, a technique for automatically authenticating and tabulating in-store merchandise is expected.

  In order to solve such a problem, Patent Literature 1 recognizes one product from an image obtained by photographing a plurality of products placed on a storage shelf during a product matching operation for inventory management. The image of the recognized product is collated with a master file in which the product image and character information are associated with the product in advance, and if the recognized product image does not match the image registered in the master file, the recognized product A system is disclosed in which the character information extracted from the image is collated with the character information registered in the master file, and the product with the matching image information or character information is recorded in the inventory management file.

  Patent Document 2 proposes a technique of analyzing the video of a camera mounted on a moving body to identify character information of an article and the position of the article, and detecting the position of the article based on the character information.

Japanese Patent No. 3493007 Japanese Patent No. 4135556

  However, the method of Patent Literature 2 cannot automatically perform a collation process for inventory management based on character information, and the method of Patent Literature 1 needs to collate images for many types of products. Therefore, as the number and types of products increase, the calculation load and processing time for matching increase. Further, according to the method of Patent Document 2, the position of the article corresponding to the character information is only known, and if there are a plurality of different articles with the same character, the article is erroneously recognized. Could be identified.

  The present invention has been made in view of the above circumstances, and based on an image obtained by photographing a plurality of articles, while reducing the calculation load of collation processing between individual articles in the image and a plurality of articles stored in advance. An object of the present invention is to provide an article collation apparatus, an article collation method, and an article collation program that can be appropriately collated.

  An article collation apparatus according to the present invention includes an article information storage unit that stores article information in which attribute information representing each attribute content in a plurality of types of attributes of an article is associated in advance for each of a plurality of different types of articles. An image extraction unit that extracts single-item images of individual articles from images obtained by photographing a plurality of articles, an attribute extraction unit that extracts attribute contents of predetermined types of attributes from the plurality of single-item images, and article information Based on the attribute content of the extracted predetermined type of attribute, the attribute content of the additional type of attribute that is likely to be common to a plurality of single-item images and is different from the predetermined type is estimated, and the estimated additional type of attribute The attribute limiting unit that uses the attribute content in the estimated attribute content, and the article corresponding to the estimated attribute content among the items registered in the product information based on the estimated attribute content And it is characterized in that a matching unit for matching with each article corresponding to separately image.

  An article collation method according to the present invention is an article collation method executed by an article collation apparatus, the step of extracting single item images of individual articles from images obtained by photographing a plurality of articles, and a predetermined type from a plurality of single item images. Extracting attribute contents in the attributes of the item, and extracting a plurality of articles of different types with reference to article information in which attribute information representing the attribute contents of the attributes of the plurality of types of articles is associated with each article in advance. Based on the attribute content of the specified attribute of the specified type, the attribute content of the additional type of attribute that is different from the predetermined type is highly likely to be common to multiple single-item images, and the attribute of the estimated additional type of attribute Based on the estimated attribute contents and the estimated attribute contents, the article corresponding to the estimated attribute contents among the articles registered in the article information. Only, it is characterized in that it has a step for matching with each article corresponding to separately image.

  An article collation program according to the present invention causes a computer to execute the above method.

  In the article collation apparatus of the present invention, the “article” may be any tangible object, and can be suitably applied to, for example, a tangible article that can be displayed and sold. For example, the article can be any one of a recording medium storing predetermined data, a book, and a cosmetic.

  Examples of the recording medium storing the predetermined data include a commodity in which any one of audio, image, and computer software is recorded on a recording medium such as a magnetic medium or an optical medium by an arbitrary recording technique. Typical examples of recording media include DVD, BD, CD, memory card, magnetic tape, and the like.

  The “attribute” means a classification that can classify information belonging to an article, and may be classified from any viewpoint as long as information belonging to the article can be classified. For example, as an “attribute” of a predetermined type attribute or an additional type of attribute, the size, classification, manufacturer, publisher, manufacturing year, release year, author, typeface included in the appearance of the article, appearance of the article Can be an attribute representing one of the numbers of colors included in.

  For example, when the article is a book, the above-mentioned “classification” can be, for example, a paperback book, a book, a comic, a magazine, or the like as the attribute content in the classification. Further, when the article is a music CD, for example, classic, pop music, movie music, and the like can be used as attribute contents in the classification. When the article is a moving picture DVD, for example, action, animation, horror, documentary, and the like can be attribute contents in the classification. Further, the classification of cosmetics can be attributed to the type of cosmetics such as lipstick, mascara, and eye shadow.

  Further, in the article collating apparatus according to the present invention, the attribute limiting unit corresponds to each attribute content in the predetermined type attribute based on the extracted attribute content in the predetermined type attribute. It is preferable that the ratio to be calculated is calculated as the ratio information, and the estimated attribute content is estimated based on the ratio information.

  Further, in the article collating apparatus according to the present invention, the article information storage unit, based on the article information, has a predetermined type when each attribute content in the further further type attribute corresponds to each attribute content in the further type attribute. It is preferable that frequency information in which the frequency corresponding to each attribute content in the attribute is associated is further stored, and the attribute limiting unit estimates the estimated attribute content based on the frequency information and the ratio information.

  Further, in the article collating apparatus according to the present invention, the attribute limiting unit has a predetermined type when each attribute content in the further type attribute is each attribute content in the further type attribute based on the frequency information and the ratio information. Calculates an evaluation value by weighting the frequency corresponding to each attribute content in the attribute according to the ratio of the plurality of articles corresponding to each attribute content in a predetermined type of attribute, and using the evaluation value, the estimated attribute content Is preferably estimated.

  Moreover, in the article matching apparatus according to the present invention, the attribute limiting unit calculates an evaluation value for each attribute type for each attribute content in a plurality of types of attributes different from the further types of attributes as the predetermined types of attributes. The estimated attribute content is preferably estimated using a value obtained by integrating the calculated evaluation values as a new evaluation value.

  Further, in the article matching device according to the present invention, the attribute limiting unit estimates a plurality of estimated attribute contents by assigning priorities based on the evaluation values based on the attribute contents in further types of attributes, It is preferable to collate with each article corresponding to a single-item image only for articles corresponding to the plurality of estimated attribute contents in order of priority.

  In the article collating apparatus according to the present invention, it is preferable that the image represents one area partitioned by a storage shelf or a display stand.

  According to the present invention, for a plurality of different types of articles, single item images of individual articles are extracted from images obtained by photographing the plurality of articles, and attribute contents in predetermined types of attributes are extracted from the plurality of single item images. Common to a plurality of single-item images based on the attribute contents of the predetermined type of attributes extracted with reference to the article information in which the attribute information representing the attribute contents in the plurality of types of attributes of the articles is associated in advance. Estimated attribute content of an additional type of attribute that is highly likely and different from the specified type, and attributed the attribute content of the estimated additional type of attribute to the estimated attribute content, and was registered in the article information based on the estimated attribute content Of the articles, only the articles corresponding to the estimated attribute content are collated with each article corresponding to the single item image.

  In stores where goods are displayed and sold, and libraries where goods are displayed and lent out, articles with similar or common attribute contents regarding types and uses are often placed close to each other. For example, by using an image obtained by photographing within a predetermined range, it is preferable to estimate the common attribute contents from a plurality of matching target articles arranged within the predetermined range included in the image. It is possible to estimate the attribute contents common to the matching target articles. And, since the estimated attribute contents are used, only the articles corresponding to the estimated attribute contents among the previously stored article information are collated with the articles corresponding to the single item images. It is possible to reduce the amount of calculation and the calculation time required for the collation process, compared to the case where collation is performed on the corresponding articles. In addition, since only the article corresponding to the estimated attribute content is collated with the article corresponding to the single item image, it is possible to reduce the possibility that the article candidate corresponding to the attribute content other than the estimated attribute content is erroneously collated. Therefore, based on the image which image | photographed several articles | goods, the some articles | goods in an image can be collated appropriately, reducing the burden of a collation process.

The block diagram which shows schematic structure of the article collation system using one Embodiment of the article collation apparatus of this invention. The figure which shows an example of an article master table The figure which shows an example of the collation result stock table The flowchart for demonstrating the effect | action of the article collation system using one Embodiment of the article collation apparatus of this invention. Flow chart for explaining attribute limiting process The figure which shows an example of a ratio table The figure which shows an example of a frequency table The block diagram which shows schematic structure of one Embodiment of the article master table registration system which produces an article master table Flowchart for explaining the operation of the article master table registration system shown in FIG. Flowchart for explaining a method for acquiring partial feature information

  Hereinafter, an article collation system using an embodiment of an article collation apparatus and method and a program of the present invention will be described in detail with reference to the drawings. The article collation system of the present embodiment acquires image data obtained by photographing articles displayed on a display shelf, and collates the image data with inventory article information registered in advance in the article master table, thereby inventorying articles. Management processing is performed. FIG. 1 is a block diagram showing a schematic configuration of an article matching system 1 of the present embodiment.

  As shown in FIG. 1, the article collation system 1 according to the present embodiment photographs an article collation apparatus 20, a monitor 30 connected to the article collation apparatus, and an article collation apparatus 20 to photograph a plurality of articles to be collated. And the camera 10 for providing the article verification device 20 to the article verification apparatus 20, and the article verification apparatus 20 is connected to the article master table registration system 2 through the communication network so as to be accessible to the aforementioned article master table.

  The article collation apparatus 20 in the present embodiment is a computer used when a user confirms and / or edits an inventory table for managing the inventory status of articles, and includes an input device (not shown) including a keyboard and a mouse. Standard software such as a system is installed. The article verification device 20 is controlled by a program installed from a recording medium such as a CD-ROM, and functions as the article verification device 20 of the present embodiment. The program may be installed after being downloaded from a storage device of a server connected via a network such as the Internet.

  The article collation apparatus 20 acquires an image obtained by photographing an image representing one storage shelf displaying a plurality of articles, and stores an image acquisition unit 21 (image buffer) for storing different types of articles. A master database 27 (article information storage unit) that stores an article master table T1 (article information) in which attribute information representing an attribute of an article is associated in advance for each article, and images obtained by photographing a plurality of articles to be verified. A single product image extraction unit 22 (image extraction unit) that extracts an article as a single product image, an attribute extraction unit 23 that extracts attribute contents of a predetermined type of attribute from the acquired single product image, and an extracted predetermined type of attribute Estimate the attribute content of an attribute of a different type that is different from the predetermined type of attribute that is highly likely to be common to multiple items based on the attribute content. Based on the estimated attribute content, the attribute limiting unit 24 that is the estimated attribute content, and only the articles corresponding to the estimated attribute contents among the articles registered in the article master table T1 are collated with the articles corresponding to the single item image. A first verification unit 25 to perform, a second verification unit 26 to perform further verification processing with an article registered in the article master table T1 for an article that has not been verified by the verification process by the first verification unit 25, and an article verification A communication unit 29 is provided for connecting the device 20 to a communication network.

  The master database 27 is a storage device configured by a known storage medium such as a hard disk, and together with the article master table T1 described above, a matching result inventory table T2 and a management inventory table T3 as shown in FIG. The frequency table T4 and the ratio table T5 described later are stored. Hereinafter, prior to the article matching process of the article matching apparatus 20, the article master table T1, the matching result inventory table T2, and the management inventory table T3 will be described.

  FIG. 2 shows an example of the article master table T1. As shown in FIG. 2, the article master table T1 includes an article code (identification information) for identifying a plurality of articles, overall image information of the article specified by each article code, similar article code, attribute information, and This is associated with partial feature information.

  Here, the article may be any article as long as it is displayed on the display shelf and displayed and sold. Examples of such articles to be displayed and sold include cosmetics and books, DVDs and BDs on which video information is recorded, CDs on which music information is recorded, CD-ROMs on which software applications are recorded, memory cards, magnetics, etc. There is a recording medium product on which predetermined information is recorded such as a tape. In addition, the article is not limited to the commodity sold as described above, but may be a book displayed on a display shelf of a library.

  The article master table T1 of the present embodiment is created for any one of articles such as cosmetics, books, and DVDs as described above.

  The article code in the article master table T1 is identification information for identifying individual articles as described above. For example, when the article is a cosmetic, a JAN (Japanese Article Number) code is used. When the article is a book, an ISBN (International Standard Book Number) code is used. The article code is not limited to JAN or ISBN, and any article code may be used as long as each article can be identified. Since the same article code is assigned to the same article, for example, even if there are a plurality of identical articles in stock, only one article code table corresponding to the article exists.

  The entire image information represents the overall appearance of the article, and specifically is image data obtained by photographing the entire article from the front, rear, left and right with a camera or the like. The entire image information registered in the article master table T1 may be image data itself, or link information of a connection destination such as a URL (Uniform Resource Locator) of a data server in which the image data is stored or a directory of a storage folder. Further, the whole image information includes image data taken from the direction that can be the front direction when the article is displayed on the display shelf. However, for example, in the case of a cylindrical article, the whole image information is not limited to the front / rear / left / right direction. Image data from the direction may be included.

  The similar article code is an article code of an article having overall image information similar to the article specified by the article code.

  The attribute information is information in which a plurality of attributes of an article are associated with attribute contents of each attribute. Here, the “attribute” means a classification that can classify information belonging to the article, and may be classified from any viewpoint as long as the information belonging to the article can be classified. For example, when the article is a cosmetic, as shown in FIG. 2, for example, as shown in FIG. 2, the “classification” of the cosmetic such as “product name” of the cosmetic, “price” of the cosmetic, lotion, lipstick, foundation, etc. ”(Classification)”, “manufacturer” of cosmetics, “manufacturing year” of cosmetics, “size (height)” and “size (width)” of cosmetics.

  For example, when the article is a book, the types of attributes include, for example, the “title” of the book, the “price” of the book, the “classification” of the book such as a paperback book, a book, a comic, a magazine, and the “ “Publisher”, “Series Number” of the book, “Volume” of the book such as Volume 1, Volume 2, Volume 3, “Number of Book” such as January, February, March, ”,“ Size ”of books such as A4, B5, and B6 classified according to a predetermined size standard, and“ fonts ”such as Gothic and Mincho for characters and symbols included in the appearance of articles.

  For example, when the article is a DVD, the types of attributes are, for example, “title” such as a movie recorded on the DVD, “price” of the DVD, action, animation, horror, documentary, etc. “Category” of published movies, “Publisher” issuing DVD, “Standard part number symbol” of DVD, “Volume” of DVD such as 1, 2, 3, DVD, “Issued year” of DVD, DVD There are "size (height)" and "size (width)" of the package.

  The “attribute content” in the attribute means specific information content in the attribute. For example, according to the above-described example, when the article is a copyrighted work such as a book, and the attribute is “classification”, a booklet, a book, a comic, a magazine, and the like correspond to the attribute contents. Further, when the attribute is “maker”, “publisher”, or “author”, an organization name such as a company or an individual name can be used as the attribute content. When the attribute is “size”, for example, A4, B5, B6 classified according to a predetermined size standard, or a predetermined numerical value or a predetermined numerical range classified according to a length or an area in a predetermined direction, etc. It can be. When the attribute is “production year” or “issue year”, a specific year such as 2012 or 2011 can be used as the attribute content. Further, when the attribute is “font” included in the appearance of the article, the type name of a specific typeface such as Gothic or Mincho can be used as the attribute content. When the attribute is “the number of colors” included in the appearance of the article, one color, two colors, three colors, or the like can be used as the attribute content.

  In the above description, the attributes of cosmetics, books, DVDs, and CDs have been described. However, the present invention is not limited to this, and attribute information including various attributes and attribute contents is registered according to the type of article.

  In the article master table T1, the attribute information as described above is registered in association with each article.

  The partial feature information is information of a partial range of the appearance of the article. Specifically, the above-described attribute contents appearing in a partial range of the appearance of the article and position information of a partial range in which the attribute contents appear are registered as partial feature information.

  In FIG. 2, the X coordinate, the Y coordinate, and the size when the horizontal direction is the X direction and the vertical direction is the Y direction are registered as position information of a part of the range. As the X and Y coordinates of a part of the range, when the part of the range is defined by a rectangular area, for example, the position coordinates of the upper left corner of the rectangular area or the center coordinates of the rectangular area are used. be able to. Also, as the size of the partial range, the size in the X direction and the size in the Y direction of the partial range are registered. Note that the position information of the partial range is relative positional information in the overall appearance of the article, and the attribute contents appearing in the partial range of the appearance of the article are directed in the front direction. Position information. The number of partial feature information is not necessarily one, and a plurality of combinations of attribute contents and position information may be registered.

  In the article master table T1, the article code and the above-described whole image information, similar article code, attribute information, and partial feature information are registered in association with each other for many articles sold in the market. . The article master table T1 is created by the article master table creation device 50 in the article master table registration system 2 to be described later, and the creation method will be described in detail later.

  FIG. 3 shows an example of the matching result inventory table T2. As shown in FIG. 3, the verification result inventory table T <b> 2 is a table in which each article code is associated with the inventory number of articles specified by the article code. The verification result inventory table T2 is updated based on the result of the verification process in the first verification unit 25 or the second verification unit 26 described above. Specifically, when one article candidate is extracted from the article master table T1 for the article corresponding to the single item image by the matching process, the number of inventory of the article code of the article in the matching result inventory table T2 is incremented by one. The verification result stock table T2 is updated.

  Similarly to the matching result inventory table T2, the management inventory table T3 is a table in which each article code is associated with the number of articles specified by the article code. It is updated every time an article is carried in and out. Specifically, each time an article is brought into a store, a library, or the like, the management inventory table T3 is incremented by one when the number of inventory corresponding to the article code of the article is incremented and the article is carried out. The stock quantity corresponding to the article code of the article is decremented by one and updated. Note that the delivery of an article at the store means that the article has been received, and the delivery of the article means that the article has been sold and taken out.

  That is, the management inventory table T3 represents the inventory status of articles managed in stores and libraries, and the matching result inventory table T2 represents the inventory status of articles actually present in the stores and libraries. It is. The management stock table T3 and the matching result stock table T2 are essentially identical when the above-described matching processing is completed. However, for example, if an article is stolen or there is a mistake in updating the management inventory table T3 at the time of carrying in or carrying out the article, these tables will not match. The verification result stock table T2 and the management stock table T3 are displayed on the monitor 30.

  Next, the operation of the article matching system 1 of the present embodiment will be described with reference to the flowchart shown in FIG.

  First, prior to the article collation process, the user takes an image of articles arranged on the display shelf using the camera 10. Specifically, here, the shooting range of the camera 10 is set so that the entire display shelf including a plurality of shelves on which a plurality of products are displayed is included, and shooting is performed. And the image data image | photographed with the camera 10 is acquired by the image acquisition part 21 of the goods collation apparatus 20 connected to the camera 10 by wire or radio | wireless, and is once stored.

  Then, when the user activates the article collation processing software, selects an image obtained by photographing the above-described one display shelf stored in the image acquisition unit 21 using the input device, and selects the article collation processing start button. In response to these designation operations and selection operations, the article matching device 20 starts the article matching process.

  First, the single product image extraction unit 22 reads out the image data stored in the image acquisition unit 21, and extracts and classifies single product images (single product image data) of each article from the read image data (S10). . Here, the single product image extraction unit 22 acquires one image obtained by photographing a plurality of articles to be collated from the image acquisition unit 21, and uses the well-known template matching method to obtain an area that matches the external shape of the article stored in advance. By using the article region to be verified, the individual article region is recognized from this image, and single product image data of each article is extracted. It should be noted that various other known methods can be used for the extraction of single item image data.

  Next, the attribute extraction unit 23 acquires the article master table T1 from the master database 27 on the memory (S12).

  And the attribute extraction part 23 extracts the attribute content of an article | item for every acquired single item image of the article | item (S14).

  Here, the attribute extraction unit 23 binarizes the single product image, extracts one or more chunk areas, performs labeling, recognizes each character from each chunk area by a well-known method, and knows the recognized character. It is analyzed by natural language processing technology and recognized as character information. Then, the attribute information or “article code” of each type of attribute such as “manufacturing year” and “manufacturer” is acquired from the character information in the single product image based on a pre-stored association table (not shown). . The association table (not shown) is a table in which known character information representing attribute contents and a known article code are registered, and is stored in the master database 27.

  If the character information extracted from the single item image includes character information registered in the association table, the attribute extraction unit 23 uses the attribute of each type of attribute corresponding to the character information. Recognize that it corresponds to the content. When the article code is extracted from the character information in the single product image, the attribute extraction unit 23 refers to an article attribute information database 59 described later, and is associated with articles represented by various article codes. Extract and obtain attribute contents. In the present embodiment, it is assumed that an article code is extracted from a plurality of single product images and the attribute content in “size” is acquired. In addition, the character information extracted from the single product image includes not only hiragana, katakana, kanji and English characters, but also numbers and symbols. In addition to a plurality of characters, one character or the like is included.

  In addition, the attribute extraction unit 23 uses a well-known method to describe the attribute contents corresponding to the single item image for a plurality of types of attributes such as the “number of colors” of the single item image and the “font” of the character in the character information included in the single item image. Extract and get. For example, the attribute extraction unit 23 compares characters represented by a plurality of different types of fonts with characters in the character information included in the single item image with a uniform character size, and is included in the single item image. The font of the character that most closely matches the character can be obtained as the font of the character included in the single product image, and the number of colors that exceed the predetermined threshold for each predetermined color using the color histogram of the single product image As the number of colors.

  Here, the attribute limiting process (S16) of FIG. 4 will be described in detail with reference to the flowchart shown in FIG.

  The attribute limiting unit 24 calculates the ratio table T5 based on the content of each attribute extracted from the plurality of single product images to be collated extracted from the image (S40).

  The ratio table T5 is based on all the single-item images extracted from the images, and shows the ratio of the articles corresponding to each single-item image corresponding to the content of each attribute in a predetermined type of attribute among the articles corresponding to all the single-item images. It is a correspondence. Here, the attribute limiting unit 24 creates the ratio table T5 for “size” and “font” set in advance.

  FIG. 6 shows an example of a ratio table T5a for “size” and a ratio table T5b for “font” as an example of the ratio table (ratio information). As shown in the ratio table T5a, in this embodiment, for the sizes of the plurality of articles arranged on the display shelf, the number of articles to be collated determined as A4 size is 70% of the total number of articles to be collated. The number of articles to be collated determined to be B5 size is 15% of the total number of articles to be collated, and the number of articles to be collated determined to be A6 size is calculated to be 15% of the total number of articles to be collated. Yes. Further, as shown in the ratio table T5b, in the present embodiment, for a plurality of article fonts arranged on the display shelf, the number of articles to be collated determined to be Gothic is 40% of the total number of articles to be collated. The number of articles to be collated determined to be Mincho is 30% of the total number of articles to be collated, and the number of articles to be collated determined to be other fonts is 30% of the total number of articles to be collated. It is calculated.

  Subsequently, the attribute limiting unit 24 acquires the frequency table T4 from the master database 27 on the memory (S42). The frequency table T4 is created by an article master table creating apparatus 50 described later, acquired by the article collating apparatus 20 via a network, and stored in the master database 27.

  The frequency table T4 is the attribute contents in the “classification” for each attribute content (class A, class B, class C) in the “classification” (further type of attribute) for all articles registered in the article master table T1. In this case, the frequency corresponding to each attribute in a predetermined type of attribute other than “classification” is associated. FIG. 7 shows a frequency table T4a for “size” (a predetermined type of attribute), a frequency table T4b for “manufacturer”, a frequency table T4c for “manufacturing year”, and a frequency table T4d for “font”. And an example of the frequency table T4e for “number of colors”. As shown in the frequency table T4a for the size (predetermined type attribute) in FIG. 7, the frequency table T4a for the size includes the number of articles corresponding to Class A registered in the article master table T1. The ratios of the number of articles whose sizes are A4, B5, and B6 are calculated as 70%, 20%, and 10%, respectively. Similarly, the ratio of the number of articles having sizes A4, B5, and B6 to the number of articles corresponding to Class B registered in the article master table T1 is calculated as 20%, 60%, and 20%, respectively. The ratio of the number of articles having sizes A4, B5, and B6 to the number of articles corresponding to Class C registered in the article master table T1 is calculated as 10%, 10%, and 80%, respectively. Has been.

  Although specific numerical values are not shown in FIG. 7, the frequency table T4b for the manufacturer, the frequency table T4c for the year of manufacture, and the frequency table T4e for the number of colors are also the portions of each% in FIG. The specific numerical value calculated in is inserted.

  Based on the ratio table T5 and the frequency table T4, the attribute limiting unit 24 first sets the attribute contents in the “classification” that has a high probability of being common to a plurality of items to be collated based on the attribute contents in the size and font. The attribute content in the estimated classification is determined as the estimated attribute content (S44). The first attribute estimation method by the attribute limiting unit 24 used in the present embodiment will be described using a specific example.

  In the first attribute estimation method, the attribute limiting unit 24 is A4 which is the attribute content having the highest corresponding ratio of the matching target articles based on the ratio table T5a regarding the size (predetermined type attribute) shown in FIG. Is detected. Then, based on the detected attribute content A4 and the frequency table T4a shown in FIG. 7, the frequency corresponding to A4 in the case of Class A is 70%, the frequency corresponding to A4 in the case of Class B, C% If it is a class, the frequency 10% corresponding to A4 is compared, and class A that is the attribute content with the highest evaluation value is determined as the estimated attribute content. As a modification of the first attribute estimation method, the attribute limiting unit 24 determines a plurality of attribute contents whose frequencies are equal to or higher than a predetermined threshold as estimated attribute contents, and the determined estimated attribute contents have a large frequency. You may attach a priority so that a priority may become low as it becomes small. For example, the attribute limiting unit 24 sets a plurality of attribute contents (class A and class B) having a frequency equal to or higher than a predetermined threshold as estimated attribute contents, and prioritizes the class A and class B in accordance with the frequency. The estimated attribute content may be determined by assigning a priority so that the degree is low.

  As shown in FIG. 4, when the estimated attribute content is determined by the attribute estimation process, the first matching unit 25 selects an article corresponding to each estimated attribute content among the articles registered in the article master table T1. Only the first article matching process is performed on the image data of the entire image information registered in the article master table T1 and the single-item image data (S18). In addition, the 1st collation part 25 is the order of the presumed attribute content with high priority, when the some presumed attribute content to which priority was attached | subjected is decided, among the articles | goods registered into the article master table T1. Article verification is performed between the image data of the article corresponding to the estimated attribute content and the single product image data, and the article whose article verification is not completed is the next highest priority among the articles registered in the article master table T1. The article matching process between the image data of the article corresponding to the estimated attribute content and the single product image data may be repeated.

  Here, the 1st collation part 25 performs template matching with the single-item image of the goods for collation, and the image data of the whole image information of the goods registered into the goods master table T1, and evaluates the correlation between known images. A value representing the similarity is calculated using the evaluation function. Note that the process of calculating the image similarity by the first matching unit 25 may use any known method capable of calculating the image similarity.

  When the calculated value is judged based on the threshold condition, and only one piece of image data of the whole image information having a similarity (similar) greater than (similar to) a predetermined criterion represented by the threshold is extracted. The first verification unit 25 determines an article corresponding to the image data. Similarly, the first collation unit 25 performs the first article collation process on the image data of the entire image information registered in the article master table T1 and all the extracted single item image data, respectively, so that all the single items are obtained. When the image data of the entire image information is extracted one by one for the image data (S20, YES), the collation result is reflected (output) in the collation result inventory table T2 on the master database 25. That is, in the matching result inventory table T2, for each extracted whole image, the number of stocks corresponding to the article code of the article corresponding to the extracted whole image is incremented by the number of times the whole image has been extracted, The verification result stock table T2 is updated (S28). Next, the updated check result inventory table T2 is displayed on the monitor 30 so that the user can check (S30).

  On the other hand, when a plurality of article candidates are extracted by the first matching process (S20, NO, S22, YES), the second matching unit 26 performs the second matching process (S24). Here, the second verification unit 26 acquires attribute contents for the color, size, font, manufacturer, and year of manufacture of the plurality of article candidates from the article master table T1, and compares them with the plurality of attribute contents extracted from the single product image. Then, the article candidate having the largest number of cases with matching attribute contents is extracted. However, the present invention is not limited to this, and the second collation unit 26 applies further collation processing by various known collation methods as long as it determines one article from a plurality of article candidates for a single product image. Good.

  When one article candidate is extracted for each article to be collated by the second collation process described above, the second collation unit 26 determines an article corresponding to the image data. Similarly, the second matching unit 26 performs the second article matching process on all the single item image data from which the plurality of candidates are extracted, and performs the entire image information on all the single item image data. When one piece of image data is extracted (S26, YES), the collation result is reflected in the collation result inventory table T2. That is, the verification result inventory table T2 is updated by incrementing the inventory quantity corresponding to the article code of the article by one (S28). Then, the updated verification result stock table T2 is displayed on the monitor 30 (S30).

  On the other hand, if one item candidate is not extracted for each item to be verified by the first or second matching process (S22, no item candidate and S26, NO), the corresponding item can be extracted. The single image data of the matching target article that has not existed and the fact that the corresponding article could not be extracted are displayed on the monitor 30 (S32). Then, the user who has confirmed the monitor 30 directly confirms the appearance, the article code, etc. of the article to be collated manually as necessary, and the corresponding article in the collation result inventory table T2 on the master database 27 by manual operation. The result of matching is updated in the number of stocks corresponding to the article code. Then, the updated verification result stock table T2 is displayed (S30).

  Further, when the matching result inventory table T2 is displayed, the difference in the inventory number of articles between the matching result inventory table T2 obtained by the article matching process according to the present embodiment and the management inventory table T3 stored in advance is calculated. It is preferable to calculate and display the difference in the inventory quantity of the articles so as to be visible. Further, the process of displaying the updated matching result inventory table T2 on the monitor 30 may be performed after all the matching processes are completed as in the above-described example, and is performed every time each matching process is completed. May be.

  According to the present embodiment, an article that is similar or common in type or use (a matching target article) that is often used in a store that displays and sells merchandise or a library that displays books is placed within a predetermined vicinity range. Attribute content common to a plurality of items to be collated arranged in a predetermined range included in the image using an image obtained by photographing the predetermined range by suitably using the display method of By estimating the attribute content, it is possible to suitably estimate the attribute contents common to the plurality of items to be collated. And, since the estimated attribute content is used, only the articles corresponding to the estimated attribute contents among the previously stored article information are collated with a plurality of single-item images, and therefore the articles corresponding to all the previously stored attributes. The amount of calculation and the calculation time required for the collation process can be reduced as compared with the case where the respective collations are performed. In addition, since only the article corresponding to the estimated attribute content is collated with the article corresponding to the single item image, it is possible to reduce the possibility that the article candidate corresponding to the attribute content other than the estimated attribute content is erroneously collated. Therefore, based on the image which image | photographed several articles | goods, the several collation target articles in an image can each be collated easily and appropriately. In addition, when collation processing is performed using images obtained by displaying a plurality of articles to be collated in the display state, it is not necessary to perform an operation such as checking the article code of each article. Time and effort to make adjustments are reduced.

  Further, the image obtained by photographing the verification target article used in the above embodiment may be an image obtained by photographing a range in which the verification target articles corresponding to the same or common attribute contents are often arranged close to each other. Is preferred. From this point of view, it is possible to appropriately estimate the estimated attribute contents that are commonly used by using an image obtained by photographing a range such as one storage shelf or a display stand as in the above embodiment. In addition, if an object obtained by photographing a predetermined range in which matching target articles corresponding to the same or common attributes are often arranged close to each other is used, the article matching process used in the above embodiment is performed in units of one image. Or may be implemented in units of a plurality of images. Further, the same effect can be obtained by using a photographed image not only for the storage shelf unit but also for one section of the storage shelf divided by a plurality of shelf boards as a predetermined range.

  Further, in the above-described embodiment, when the additional type of attribute is a classification used when determining the display position of the article in the photographed image, the attribute content common to the matching target article can be accurately estimated. When the “classification” in the above embodiment is applied to the article matching process as an additional type of attribute, the display position of the article is determined in a display method in which articles of similar or common use are arranged close to each other. Since “classification” is often used as the classification used in the determination, the attribute contents common to the matching target articles can be estimated appropriately.

  Further, an additional type of attribute may be an “article series name” that is a series of character strings that are uniformly assigned to a plurality of matching target articles. The series of character strings here may include not only characters but also numbers and symbols in any combination. For example, when the article is a cosmetic, it is preferable that the attribute of the additional type is “article series name”, for example, cleansing oil, cleansing gel, facial cleansing foam, lotion, cosmetic liquid, moisturizing cream, sunscreen. An article series name given to a series of skin care cosmetics such as cream, or an article series name given to a series of base makeup cosmetics such as a makeup base, concealer, foundation, face powder, etc. can be used. Further, for example, when the article is a book or a DVD, the “article series name” may be a work name when one work is divided into a plurality of volumes and published.

  In addition, as a predetermined type attribute or a further type of attribute, the size of the article, the manufacturer, the publisher, the production year, the release year, the author, the typeface included in the appearance of the article, the number of colors included in the appearance of the article An attribute representing either one can be preferably applied.

  In addition, it is not limited to this embodiment, The attribute limitation part 24 can determine presumed attribute content using the following 2nd attribute estimation methods, for example.

  In the second attribute estimation method, on the basis of the frequency table T4 and the ratio table T5, the frequency corresponding to the attribute of the other predetermined type attribute when the attribute is a further type attribute, An evaluation value that is weighted and integrated according to the proportion of the matching target article corresponding to the attribute of another predetermined type of attribute is calculated, and the estimated attribute content is estimated using the evaluation value.

  This will be specifically described with reference to FIGS. Based on the attribute contents of the plurality of matching target articles, the attribute limiting unit 24 calculates a ratio table T5a representing the corresponding ratio of the plurality of matching target articles for each attribute content of “size” (a predetermined type of attribute). And get. Then, based on the ratio table T5a and the frequency table T4a, an evaluation value representing the probability that a plurality of matching target articles are commonly applicable to the “class” (further type) class A. When “Classification” is Class A, “Size” indicates the frequency of each attribute content (A4, B5, B6), and “Size” of the object to be verified in the image indicates each attribute content (A4, B5, B6). The weighted sum is obtained by weighting each of them at a ratio corresponding to. Similarly, an evaluation value is obtained for class B and class C. Then, class A that is the attribute content having the highest evaluation value is determined as the estimated attribute content.

The evaluation values calculated based on the ratio table T5a and the frequency table T4a are shown below.
Evaluation value of class A based on size = 70 x 70 + 20 x 15 + 10 x 15 = 5350
Evaluation value of B classification based on size = 20 x 70 + 60 x 15 + 20 x 15 = 2600
Evaluation value of C classification based on size = 10 x 70 + 10 x 15 + 80 x 15 = 2050

  As a modified example of the second attribute estimation method, the attribute limiting unit 24 determines a plurality of attribute contents having a frequency equal to or higher than a predetermined threshold as estimated attribute contents, and the determined estimated attribute contents have a large frequency. You may attach a priority so that a priority may become low as it becomes small. For example, the attribute limiting unit 24 sets a plurality of attribute contents (class A and class B) having a frequency equal to or higher than a predetermined threshold as estimated attribute contents, and prioritizes the class A and class B in accordance with the frequency. The estimated attribute content may be determined by assigning a priority so that the degree is low.

  Furthermore, the attribute limiting unit 24 may determine the estimated attribute content using the third attribute estimation method.

  In the third attribute estimation method, based on the frequency table T4 and the ratio table T5, the attribute contents of a plurality of types of attributes different from the additional types of attributes are evaluated for each type of attribute as predetermined types of attributes. Each value is calculated, and the estimated attribute content is estimated using a value obtained by integrating the calculated evaluation values as a new evaluation value.

  A specific description will be given using a ratio table T5a, a ratio table T5b, a frequency table T4a, and a frequency table T4d shown in FIGS. In the third attribute estimation method, the attribute limiting unit 24 has a corresponding ratio table of matching target articles in an image for each attribute content of a predetermined type (“size” and “font”) based on a plurality of single-item images. T5a and the ratio table T5b are respectively calculated and acquired, and the evaluation value is calculated for the “size” based on the ratio table T5a and the frequency table T4a in the same manner as the second attribute estimation method. With respect to, an evaluation value is calculated in the same manner as the second attribute estimation method based on the ratio table T5b and the frequency table T4d. Then, the sum of the evaluation value based on “size” and the evaluation value based on “font” for “classification” (an additional type of attribute) is used as a comprehensive evaluation value (new evaluation value). Then, class A that is the attribute content having the highest evaluation value is determined as the estimated attribute content.

The evaluation values calculated based on the ratio table T5a, the ratio table T5b, the frequency table T4a, and the frequency table T4c are shown below.
Evaluation value of class A based on size = 70 x 70 + 20 x 15 + 10 x 15 = 5350
Evaluation value of B classification based on size = 20 x 70 + 60 x 15 + 20 x 15 = 2600
Evaluation value of C classification based on size = 10 x 70 + 10 x 15 + 80 x 15 = 2050
Evaluation value of A classification based on font = 80 × 40 + 10 × 30 + 10 × 30 = 3800
B classification evaluation value based on font = 30 x 40 + 30 x 30 + 40 x 30 = 3300
Evaluation value of C classification based on font = 10 × 40 + 30 × 30 + 60 × 30 = 3100
Evaluation value of A classification = (Evaluation value of A classification based on size) + (Evaluation value of A classification based on font) = 5350 + 3800 = 9150
Evaluation value of B classification = (Evaluation value of B classification based on size) + (Evaluation value of B classification based on font) = 2600 + 3300 = 5900
Evaluation value of C classification = (Evaluation value of C classification based on size) + (Evaluation value of C classification based on font) = 2050 + 3100 = 5150

  As a modification of the third attribute estimation method, the attribute limiting unit 24 determines a plurality of attribute contents whose frequencies are equal to or higher than a predetermined threshold as estimated attribute contents, and the determined estimated attribute contents have a large frequency. You may attach a priority so that a priority may become low as it becomes small. For example, the attribute limiting unit 24 sets a plurality of attribute contents (class A and class B) having a frequency equal to or higher than a predetermined threshold as estimated attribute contents, and prioritizes the class A and class B in accordance with the frequency. The estimated attribute content may be determined by assigning a priority so that the degree is low.

  In the third attribute estimation method, the predetermined type of attribute may be one type of attribute as described above, or may be n types of attributes (n> 1). When the predetermined type of attribute is n types of attributes, the ratio table T5 is calculated for each of the n types of attributes in the same manner as the first attribute method, and n for each of the further types of attributes. For each type of attribute, an evaluation value is calculated in the same manner as the second attribute method using the ratio table T5 and the frequency table T4, and the calculated n evaluation values are added to obtain a new evaluation value. Can be used as

  Further, according to the first to third estimated attribute content determination methods, each matching target article corresponds to each attribute content in the predetermined type attribute based on the extracted attribute content in the predetermined type attribute. Since the estimated attribute content is estimated based on the ratio table T5 representing the ratio, it is possible to suitably calculate a predetermined type of attribute common to a plurality of articles in the image. Further, since the frequency table T4 representing the frequency of each attribute content in a predetermined type of attribute in the case of each attribute content in a further type of attribute is used, among the plurality of matching target articles included in the image Even if there is a matching target article whose attribute content in a predetermined type of attribute is unknown, if the attribute content in the predetermined type attribute of another matching target product is extracted, the attribute content of the predetermined type attribute is The matching process can also be performed with respect to the unknown matching target article by limiting the attribute contents, and the calculation process of the matching process with the article master table T1 can be reduced.

  Further, according to the second and third estimated attribute content determination methods, the attribute limiting unit 24 uses the frequency table T4 and the ratio table T5 to determine each attribute content in the additional type attribute in the additional type attribute. An evaluation in which the frequency corresponding to each attribute content in a predetermined type of attribute in the case of each attribute content is weighted and integrated according to the ratio of the plurality of matching target articles corresponding to each attribute content in the predetermined type attribute Since the value is calculated and the estimated attribute content is estimated using the evaluation value, there is a high possibility that the estimated attribute content can be determined more accurately. For this reason, it is possible to match the articles more preferably by limiting the attribute contents.

  Furthermore, according to the third estimated attribute content determination method, the evaluation value is calculated for each attribute type for each attribute content in a plurality of types of attributes different from the additional types of attributes as the predetermined types of attributes. Then, since the estimated attribute content is estimated using a value obtained by integrating the calculated evaluation values as a new evaluation value, there is a high possibility that the estimated attribute content can be determined more accurately. For this reason, it is possible to match the articles more preferably by limiting the attribute contents.

  Further, according to the first to third estimated attribute content determination methods, the attribute limiting unit 24 associates a plurality of priority levels with respect to each attribute content in each of the plurality of further types of attributes, based on the evaluation value. Since the estimated attribute contents are estimated, and the matching unit performs matching with a plurality of matching target articles only for articles corresponding to the plurality of estimated attribute contents in order of priority, each article registered in the article master table T1 The matching processing can be performed in order from the attribute contents having a high possibility of corresponding to a plurality of matching target articles, and the processing time and calculation load for the matching processing can be effectively reduced.

  In the above embodiment, the article can be any commodity that can be displayed and sold. For example, when the article is any one of a recording medium product such as CD, DVD, BD, and magnetic tape in which at least one of voice, image, and computer software is recorded by an arbitrary recording method, a book, and cosmetics, this implementation The article collation processing by the form article collation apparatus 20 can be suitably applied.

  Moreover, in the said embodiment, the goods collation apparatus 20 is good also as a structure which abbreviate | omitted the 2nd collation part 26. FIG.

  Next, an article master table registration system for creating the article master table T1 described above will be described. FIG. 8 is a block diagram showing a schematic configuration of the article master table registration system 2.

  As shown in FIG. 8, the article master table registration system 2 of the present embodiment includes a camera 40, an article master table creation device 50, and a monitor 70.

  The camera 40 captures the overall appearance of the article to be registered in the article master table T1, and registers it in the article master table T1 based on image data obtained by capturing the overall appearance of the article by the camera 40. The entire image information is acquired. Therefore, the imaging | photography with the camera 40 is performed from the front-back, left-right direction of the goods for registration, for example. If the image data includes a barcode image representing an article code, an article code registered in the article master table T1 is acquired based on the barcode image. If the image data of the overall appearance of the registration target article does not include a barcode image representing the article code, the part including the barcode image is separately photographed by the camera 40. Further, the barcode image may be separately read by a barcode reader. In addition, about imaging | photography of the registration object goods, it is performed for every goods.

  In the present embodiment, the entire image information and the article code of the article to be registered are acquired using the image data taken by the camera 40. For example, when the article is a book, You may make it acquire image data using a scanner.

  In addition, image data including the overall appearance of the article and image data including a barcode image are not necessarily acquired by photographing with a camera or photoelectric reading with a scanner. For example, in the process of editing a book, Image data of a front cover, a back cover, and a back cover created in advance may be acquired, and the entire image information and article code may be acquired based on the image data.

  As shown in FIG. 8, the article master table creation device 50 includes an image acquisition unit 51, an image region extraction unit 52, a similarity determination unit 53, a partial region extraction unit 54, a character recognition unit 55, an article code region extraction unit 56, and an article. A code recognition unit 57, an article attribute acquisition unit 58, an article attribute information database 59, a character match determination unit 60, a master database 61, and a communication unit 63 are provided.

  The image acquisition unit 51 acquires image data of an article to be registered photographed by the camera 40 and temporarily stores it in a storage medium such as a semiconductor memory or a hard disk.

  The image area extraction unit 52 reads the image data of the registration target article from the image acquisition unit 51, and acquires the article area image data by deleting the image data of the background of the article from the image data. Various known methods can be used as a method for acquiring the article region image data. The article region image data (for example, front / rear / left / right image data) acquired by the image region extraction unit 52 is registered in the article master table 62 as the entire image information of the article to be registered.

  The similarity determination unit 53 acquires the article region image data acquired by the image region extraction unit 52, and collates the article region image data with the entire image information of a plurality of articles already registered in the article master table 62. Then, an article code having overall image information including image data similar to the article area image data of the article to be registered is acquired as a similar article code. The matching between the article region image data and the entire image information of the registered article may be performed, for example, by calculating the similarity using template matching. The similar article code acquired by the similarity determining unit 53 is registered in the article master table 62 as the similar article code of the registration target article.

  The partial region extraction unit 54 receives the article region image data acquired by the image region extraction unit 52, and based on the article region image data, the above-described attribute content of the registration target article may appear. The region is extracted.

  Specifically, the partial region extraction unit 54 extracts a region where character information appears in the article region image data as a partial region. For example, the partial area extraction unit 54 binarizes the article area image data, performs labeling on the binarized data, and extracts a mass area where character information is supposed to exist based on the labeling information as a partial area To do. Note that the character information here includes not only hiragana, katakana, kanji and English characters, but also numbers and symbols. In addition to a plurality of characters, one character or the like is included.

  The extraction method of the character information area (partial area) is not limited to this, and various known methods can be used. Also, the number of partial areas is not limited to one, and when a plurality of character information areas are extracted, the plurality of character information areas may be extracted as partial areas. The labeling data of the partial area extracted by the partial area extraction unit 54 is output to the character recognition unit 55.

  The character recognition unit 55 performs character recognition processing based on the labeling data in the partial region extracted by the partial region extraction unit 54 and recognizes character information existing in the partial region. Since the character recognition process is already known, a detailed description thereof will be omitted. The character information recognized by the character recognition unit 55 is output to the character match determination unit 60.

  The article code area extraction unit 56 reads image data of a registration target article from the image acquisition unit 51, and extracts an area where a barcode image representing an article code exists from the image data as an article code area. As for the extraction of the area where the barcode image exists, an area where a plurality of line segments of a predetermined length are arranged may be extracted, but various known methods can be used for the extraction method. Information on the article code area extracted by the article code area extraction unit 56 is output to the article code recognition unit 57.

  The article code recognizing unit 57 acquires the article code of the article to be registered by recognizing a barcode image existing in the article code area based on the inputted article code area information. The article code acquired by the article code recognition unit 57 is registered in the article master table 62 as the article code of the article to be registered.

  The article attribute acquisition unit 58 acquires the article code recognized by the article code recognition unit 57, and acquires the attribute of the registration target article and its attribute contents from the article attribute information database 59 based on the contents of the article code. It is. As described above, for example, “series number”, “number of turns”, “number”, “standard part number symbol” or “year of manufacture” are included as attributes, and the attribute contents are “series number”, “number of turns”. , “Number”, “standard part number symbol” or “manufacturing year”. The attributes acquired by the article attribute acquisition unit 58 and their attribute contents are registered in the article master table 62 as attribute information of the registration target article. The attribute information of the registration target article is not limited to that acquired based on the article code, and may be acquired by, for example, setting input using an input unit by the user. The attribute content acquired by the article attribute acquisition unit 58 is also output to the character match determination unit 60.

  Then, the character match determination unit 60 compares the character information recognized by the character recognition unit 55 with the attribute content acquired from the article attribute acquisition unit 58, and if they match, the character information is represented by the character information. The attribute content and the attribute to be registered and the position information of the partial area where the character information exists are registered in the article master table 62 as the partial feature information of the article to be registered. Specifically, for example, when both the character information recognized by the character recognition unit 55 and the content of the attribute information acquired from the article attribute acquisition unit 58 are “222333”, the “222333” The type of attribute information of “222333” (for example, the series number) and the position information of the partial area where the character information of “222333” exists are registered in the article master table 62 as the partial feature information of the article to be registered. . On the other hand, the character information recognized by the character recognition unit 55 is compared with the attribute content acquired from the article attribute acquisition unit 58. If they do not match, the character match determination unit 60 converts the character information into partial information. The feature information is not registered.

  The article attribute information database 59 stores attributes and attribute contents represented by various article codes.

  The master database 61 stores an article master table 62 for registered articles. The details of the article master table 62 are the same as the article master table T1 stored in the master database 27 of the article collating apparatus 20 described above. The master database 61 may store image data 62a of the entire image information of each article in the article master table 62.

  The article master table 62 in the article master table creation device 50 is updated each time a new registration target article table is registered. Then, the article master table 62 in the article master table creation device 50 is output to the article matching device 20 of the article matching system 1 via the communication unit 63 and the communication network in response to a request from the article matching device 20 to create an article master table. The article master table T1 is updated by overwriting the contents of the article master table 62 in the apparatus 50 on the article master table T1 in the article collating apparatus 20. The article master table T1 of the article verification device 20 may be updated every time the article master table 62 of the article master table creation device 50 is updated, or may be periodically updated every predetermined period. Alternatively, it may be performed at an arbitrary timing in accordance with an update instruction input by the user.

  The article master table creation device 50 updates a frequency table (not shown) in the article master table creation device 50 every time a new registration target article table is registered. Then, the frequency table in the article master table creation device 50 is output to the article matching device 20 via the communication unit 63 and the communication network in response to a request from the article matching device 20, and the contents of the frequency table in the article master table creation device 50 are displayed. However, the frequency table T4 is updated by being overwritten on the frequency table T4 in the article matching device 20. The frequency table T4 of the article matching device 20 may be updated every time the frequency table of the article master table creation device 50 is updated, or may be periodically updated every predetermined period. Then, it may be performed at an arbitrary timing in accordance with an update instruction input by the user.

  The monitor 70 displays the article master image 62, the article region image data acquired by the image region extraction unit 52, and the like.

  Next, the operation of the article master table registration system 2 of the present embodiment will be described with reference to the flowcharts shown in FIGS.

  First, an article to be registered is installed at a predetermined position by the user (S50). Then, the entire appearance of the article to be registered is photographed by the user using the camera 40 (S52). At this time, if the image data of the overall appearance of the article does not include the barcode image representing the article code, the camera 40 separately captures the portion including the barcode image.

  Next, the image data photographed by the camera 40 is acquired and temporarily stored by the image acquisition unit 51 of the article master table creation device 50 connected to the camera 40 by wire or wirelessly.

  The image data acquired by the image acquisition unit 51 is output to the article code area extraction unit 56, and the article code area extraction unit 56 defines the area where the barcode image representing the article code is present from the input image data as the article code area. (S54).

  Information on the article code area extracted by the article code area extraction unit 56 is output to the article code recognition unit 57. The article code recognition unit 57 stores the information in the article code area based on the inputted article code area information. Is recognized (S56), and the article code of the article to be registered is acquired (S58). The article code acquired by the article code recognition unit 57 is registered in the article master table 62 as the article code of the article to be registered.

  Next, the image data acquired by the image acquisition unit 51 is output to the image region extraction unit 52, and the image region extraction unit 52 deletes the image data of the background of the article from the input image data, thereby performing the article region image. Get the data. Then, the article region image data (for example, front / rear / left / right image data) acquired by the image region extraction unit 52 is registered in the article master table 62 as the whole image information of the registration target article (S60).

  The article region image data acquired by the image region extraction unit 52 is output to the similarity determination unit 53, and the similarity determination unit 53 includes a plurality of pieces of article region image data that have been input and a plurality of items already registered in the article master table 62. The product code having the overall image information including image data similar to the product region image data of the registration target product is acquired as a similar product code (S62). The similar article code acquired by the similarity determining unit 53 is registered in the article master table 62 as the similar article code of the registration target article.

  Next, the article code recognized by the article code recognition unit 57 is output to the article attribute acquisition unit 58, and the article attribute acquisition unit 58 stores the registration target article from the article attribute information database 59 based on the contents of the article code. An attribute and its attribute content are acquired (S64). The attributes acquired by the article attribute acquisition unit 58 and their attribute contents are registered in the article master table 62 as attribute information of the registration target article.

  Further, partial feature information is acquired for the registration target article. The operation of acquiring the partial feature information will be described with reference to the flowchart shown in FIG.

  First, the article region image data acquired by the image region extraction unit 52 is input to the partial region extraction unit 54, and the partial region extraction unit 54 performs a character in the article region image data based on the input article region image data. The area where the information appears is extracted as a partial area (S70).

  Then, the labeling data of the partial region extracted by the partial region extraction unit 54 is output to the character recognition unit 55, and the character recognition unit 55 performs character recognition processing based on the labeling data in the partial region, and performs the character recognition processing in Is recognized (S72). The character information recognized by the character recognition unit 55 is output to the character match determination unit 60.

  On the other hand, the attribute content acquired based on the item code in the item attribute acquisition unit 58 is also output to the character match determination unit 60.

  Then, the character match determination unit 60 compares the character information recognized by the character recognition unit 55 with the attribute content acquired from the article attribute acquisition unit 58, and if they match, the character information is represented by the character information. The registered attribute contents and the position information of the partial area where the character information exists are registered in the article master table 62 as the partial feature information of the registration target article (S76). On the other hand, the character information recognized by the character recognition unit 55 is compared with the attribute content acquired from the article attribute acquisition unit 58. If they do not match, the character match determination unit 60 converts the character information into partial information. It is not registered as feature information.

  As described above, a new registration target article is registered in the article master table 62.

  According to the article master table registration system 2 of the above-described embodiment, the attribute information and position information of the partial feature information can be extracted with high accuracy and simplicity.

  Note that the present invention is not limited to this embodiment, and some or all of the constituent elements of the clinical information display apparatus may be configured by one workstation, and one or more connected via a network It may be configured by a workstation, a server, and a storage device. Each device is controlled by a program for displaying clinical information of the present specification installed from a recording medium such as a CD-ROM. The program may be installed after being downloaded from a storage device of a server connected via a network such as the Internet.

  Moreover, each embodiment mentioned above shows the one aspect | mode of this invention to the last, and a deformation | transformation and application are arbitrarily possible within the range which does not deviate from the meaning of this invention.

DESCRIPTION OF SYMBOLS 1 Article collation system 2 Article master table registration system 10, 40 Camera 20 Article collation apparatus 21 Image acquisition part 22 Single item image extraction part (image extraction part)
DESCRIPTION OF SYMBOLS 23 Attribute extraction part 24 Attribute limitation part 25 1st collation part 26 2nd collation part 27 Master database 30, 60 Monitor 50 Goods master table preparation apparatus T1 Goods master table T2 Matching result stock table T3 Management stock table T4 Frequency table (frequency information) )
T5 ratio table (ratio information)

Claims (9)

For a plurality of different types of articles, for each article, article information in which attribute information representing each attribute content in the plurality of types of attributes of the article is associated in advance , and each of the predetermined types of attributes for the plurality of articles The attribute content of the predetermined type attribute for the total number of articles corresponding to the attribute content of the additional type attribute for each combination of the attribute content and the attribute content of the additional type of attribute different from the predetermined type; An article information storage unit that stores frequency information associated with a frequency that is a ratio of the number of articles corresponding to both the attribute content of the additional type of attribute ;
An image extraction unit that extracts single-item images of the respective articles from images obtained by photographing a plurality of the articles;
An attribute extraction unit for extracting attribute contents in a predetermined type of attribute from the plurality of single item images;
For each attribute content of the predetermined type attribute, a ratio of the number of single item images corresponding to the article corresponding to the attribute content of the predetermined type attribute to the total number of single item images extracted from the image And calculating an evaluation value for each attribute content of the further type of attribute based on the ratio information and the frequency information, and using the evaluation value, the probability common to the plurality of single-item images is calculated . high, before Symbol attribute limiting unit to estimate attributes contents estimates the attribute contents of additional types of attributes,
Based on the content of the estimated attribute, collation that performs collation with each product corresponding to the single item image only for the product corresponding to the content of the estimated attribute among the products whose product information is stored in the product information storage unit An article collating apparatus comprising: a section. The attribute limiting unit, based on the frequency information and the rate information, the respective attribute content of additional types of attributes, the frequency with which corresponding to each attribute content of the predetermined type of attribute, the predetermined type of corresponding to each attribute contents of the attribute by integrating weighted in accordance with the ratio to calculate the evaluation value, according to claim 1, wherein the estimating the estimated attribute contents using the evaluation value Item verification device. The attribute limiting unit calculates the evaluation value for each type of attribute for each attribute content in a plurality of types of attributes different from the further type of attribute as the predetermined type of attribute, 3. The article collation apparatus according to claim 2, wherein the estimated attribute content is estimated using a value obtained by integrating a plurality of the evaluation values as a new evaluation value. The attribute limiting unit estimates a plurality of the estimated attribute contents by assigning priorities based on the evaluation values based on the attribute contents in the additional types of attributes,
4. The article collation apparatus according to claim 2 , wherein the collation unit performs collation with each article corresponding to the single item image only for articles corresponding to the plurality of estimated attribute contents in order of priority. . The predetermined type attribute or the further type attribute includes the size, manufacturer, publisher, manufacturing year, release year, author, typeface included in the appearance of the article, and color included in the appearance of the article. article collating apparatus according to any one of claims 1, wherein 4 to be a attribute that represents any number. Wherein the image is an article matching device according to any one of claims 1 to 5, characterized in that it represents one region partitioned by the storage rack or display stand. Wherein the article, a recording medium storing predetermined data, books and cosmetic articles collating apparatus according to any one of claims 1, wherein 6 to be any one of the. For a plurality of different types of articles, for each article, article information in which attribute information representing each attribute content in the plurality of types of attributes of the article is associated in advance, and each of the predetermined types of attributes for the plurality of articles The attribute content of the predetermined type attribute for the total number of articles corresponding to the attribute content of the additional type attribute for each combination of the attribute content and the attribute content of the additional type of attribute different from the predetermined type; The article collation method in the article collation apparatus provided with the article information storage unit that stores the frequency information that associates the frequency that is the ratio of the number of articles corresponding to both the attribute contents of the further types of attributes ,
Extracting a single item image of each of the article a plurality of the article from the captured image,
Extracting attribute contents in a predetermined type of attribute from the plurality of single item images;
With reference to the article information, for each attribute content of the predetermined type of attribute, the total number of the single item images extracted from the image corresponds to the article corresponding to the attribute content of the predetermined type of attribute. A ratio of the number of single item images is calculated as ratio information, and an evaluation value is calculated for each attribute content of the further type of attribute based on the ratio information and the frequency information, and the plurality of the plurality of pieces of images are calculated using the evaluation value. a step of high probability common to a single item image, before Symbol estimate the attribute contents of additional types of attributes as the estimated attribute content,
Based on the estimated attribute content, the step of collating with each item corresponding to the single item image only for the item corresponding to the estimated attribute content among the items for which the item information is stored in the item information storage unit An article collation method characterized by comprising: For a plurality of different types of articles, for each article, article information in which attribute information representing each attribute content in the plurality of types of attributes of the article is associated in advance, and each of the predetermined types of attributes for the plurality of articles The attribute content of the predetermined type attribute for the total number of articles corresponding to the attribute content of the additional type attribute for each combination of the attribute content and the attribute content of the additional type of attribute different from the predetermined type; A computer including an article information storage unit that stores frequency information that associates a frequency that is a ratio of the number of articles that correspond to both the attribute contents of the additional type of attribute ,
Extracting a single item image of each of the article a plurality of the article from the captured image,
Extracting attribute contents in a predetermined type of attribute from the plurality of single item images;
With reference to the article information, for each attribute content of the predetermined type of attribute, the total number of the single item images extracted from the image corresponds to the article corresponding to the attribute content of the predetermined type of attribute. A ratio of the number of single item images is calculated as ratio information, and an evaluation value is calculated for each attribute content of the further type of attribute based on the ratio information and the frequency information, and the plurality of the plurality of pieces of images are calculated using the evaluation value. a step of high probability common to a single item image, before Symbol estimate the attribute contents of additional types of attributes as the estimated attribute content,
Based on the estimated attribute content, the step of collating with each item corresponding to the single item image only for the item corresponding to the estimated attribute content among the items for which the item information is stored in the item information storage unit The article collation program characterized by performing these.

Images