JP5891143B2 - Article master table creation apparatus and method, and program - Google Patents

Description

  The present invention relates to an article master table creation apparatus, method, and program for creating an article master table used for specifying an article in stock in so-called inventory management of articles.

  Traditionally, retailers that sell products to end consumers check the actual inventory quantity and understand the difference between the confirmed actual inventory quantity and the inventory quantity listed in the inventory management table. Inventory check work is performed regularly. For example, for all products in the store, read the barcode attached to the product, obtain the product number, create the inventory table by counting the number for each product based on the acquired product number, and store it in advance The difference information with the actual inventory quantity is obtained by comparing with the inventory management table.

  However, with the recent diversification of products, there are many cases where a large number of products exist in large stores. For this reason, taking out each product from the display shelf and reading the barcode to check the inventory is costly and time consuming, which is a heavy burden on the retailer. For this reason, a technique for automatically authenticating and tabulating in-store merchandise is expected.

  Therefore, for example, in Patent Document 1, image data obtained by photographing products displayed on a display shelf is acquired, and the image data and image data included in a product master file stored in advance for each product are obtained. Easily execute inventory management by collating, creating and storing inventory management files for those that match, and outputting and counting the file list of the stored inventory management files A method has been proposed.

  And in patent document 1, when the product master file which corresponds only by the collation of the image data of goods is not extracted, a character is extracted from the image data which image | photographed goods, and the character corresponding to the character is goods. There has been proposed a method for extracting a corresponding product master file by searching whether it is included in the master file.

Japanese Patent No. 3493007 JP 2005-107978 A International Publication No. 98/11509

  However, in the method described in Patent Document 1, when extracting characters from the image data, the extraction area is not particularly limited, so even if the extracted characters match the characters in the product master file, In some cases, the position of the character in the captured image data and the position of the character on the appearance of the product specified by the product master file are completely different. There is a problem.

  Even if some of the characters extracted from the image data match some of the characters in the product master file, the contents represented by these characters may be completely different. In this case, there is a problem that similar products are erroneously extracted.

  Patent Document 2 proposes a system for recognizing image data within a predetermined recognition range in image data obtained by photographing a subject such as a CD, and specifying the type of CD or the like based on the recognition result. However, nothing has been proposed about how to set the recognition range.

  Note that Patent Document 3 also proposes a method for determining the type of a book based on image data obtained by photographing the book. However, what is the method for determining the type of a book using characters in the image data? Has also not been proposed.

  In view of the above circumstances, the present invention provides an article master table that can appropriately specify the range of character information when collating with the article master table by focusing on part of character information of the appearance of the article. It is an object of the present invention to provide an article master table creation apparatus, method, and program that can be created and character information and a range of the character information can be easily and accurately specified and included in the article master table.

  An article master table creation device of the present invention includes an image acquisition unit that acquires an image representing the appearance of an article to be registered, a character extraction unit that extracts characters in the image acquired by the image acquisition unit, and an article to be registered The attribute information acquisition unit for acquiring the attribute information, the character extracted by the character extraction unit, and the attribute information acquired by the attribute information acquisition unit are compared, and whether or not the character represents the attribute information Create an article master table for an article to be registered that includes a character judgment unit to be judged and a character and a position information in the image of the character when the character judgment unit judges that the character represents attribute information And a master table creation unit.

  In the article master table creation device of the present invention, an identification information acquisition unit for acquiring identification information for identifying an article to be registered is provided, and the attribute information acquisition unit acquires attribute information based on the identification information. You can do it.

  In addition, the image acquisition unit can acquire an image of an article to be registered including identification information, and the identification information acquisition unit can acquire identification information based on an image including identification information.

  Further, the master table creation unit can create the article master table by associating the identification information with the character representing the attribute information and the position information within the image of the character.

  Further, the master table creation unit can create the article master table including the size of the portion where the characters appear.

  In addition, as attribute information, at least one of a series number, volume number, issue number, standard product number symbol, manufacturer, publisher, year of manufacture, issue year, and author can be used.

  Further, as the position information, position information of a rectangular area surrounding a character can be used.

  Further, the master table creation unit can create an article master table including an image representing the appearance of an article to be registered.

  In addition, the article can be any one of cosmetics, books, and recording medium products on which predetermined information is recorded.

  The article master table creation method of the present invention acquires an image representing the appearance of an article to be registered, extracts characters in the acquired image, acquires attribute information of the article to be registered, acquires the extracted characters and If the character represents attribute information, the position information in the image of the character and the character is determined. An article master table of registration-target articles including the above is created.

  The article master table creation program of the present invention includes a computer, an image acquisition unit that acquires an image representing the appearance of an article to be registered, a character extraction unit that extracts characters in the image acquired by the image acquisition unit, and a registration Whether the character represents the attribute information by comparing the attribute information acquisition unit that acquires the attribute information of the target article, the character extracted by the character extraction unit, and the attribute information acquired by the attribute information acquisition unit An article master of an article to be registered, which includes a character judging section for judging whether or not, and when the character judging section judges that the character represents attribute information, the character and position information in the image of the character It is characterized by functioning as a master table creation unit for creating a table.

  According to the article master table creation device, method, and program of the present invention, an image representing the appearance of an article to be registered is acquired, characters in the acquired image are extracted, and attribute information of the article to be registered is acquired. The extracted character and the acquired attribute information are compared to determine whether the character represents attribute information. When it is determined that the character represents attribute information, the character and the character Since the article master table including the position information in the image is created, for example, in the case where collation with the article master table is performed by paying attention to partial character information of the appearance of the article, the article master table described above is used. By using the position information included in the character, it is possible to appropriately specify the position of the character in the verification target article.

  In addition, since the character and the position information of the character in the image of the item are specified based on the attribute information of the item and are included in the item master table, the character and the position information of the character are specified easily and with high accuracy. And can be included in the article master table.

The block diagram which shows schematic structure of the goods master table registration system using one Embodiment of the goods master table preparation apparatus of this invention. The figure which shows an example of an article master table The flowchart for demonstrating the effect | action of the goods master table registration system using one Embodiment of the goods master table preparation apparatus of this invention. Flowchart for explaining a method for acquiring partial feature information The block diagram which shows schematic structure of the goods collation system which performs the collation process of the predetermined | prescribed collation object goods using the goods master table shown in FIG. The figure which shows an example of the collation result stock table The flowchart for demonstrating the effect | action of the article collation system shown in FIG. The figure for demonstrating the method to image | photograph the image of the goods for collation Flow chart for explaining the second matching process The figure for demonstrating a 2nd collation process

  Hereinafter, an article master table registration system using an embodiment of an article master table creation apparatus and method and a program of the present invention will be described in detail with reference to the drawings. The article master table registration system according to the present embodiment creates and registers an article master table used for specifying an article in stock when so-called inventory confirmation of an article is performed. More specifically, an article master table for performing inventory check processing of articles is created by comparing image data obtained by photographing articles displayed on the display shelf with article information registered in the article master table in advance. Registered.

  FIG. 1 is a block diagram showing a schematic configuration of an article master table registration system 1 of the present embodiment. FIG. 2 shows an example of an article master table created by the article master table registration system 1 of the present embodiment.

  Before describing the configuration of the article master table registration system 1 of the present embodiment, first, the contents of the article master table will be described with reference to FIG.

  As shown in FIG. 2, the article master table 63 includes an article code (identification information) for identifying a plurality of articles, overall image information of the article specified by each article code, similar article code, attribute information, and This is associated with partial feature information.

  Here, the article may be any product as long as it is displayed on the display shelf and displayed and sold. Examples of such products to be displayed and sold include cosmetics and books, DVDs and BDs on which video information is recorded, CDs on which music information is recorded, CD-ROMs on which software applications are recorded, memory cards, magnetics, etc. There is a recording medium product on which predetermined information is recorded such as a tape. In addition, the article is not limited to the commodity sold as described above, but may be a book displayed on a display shelf of a library.

  The article master table 63 of the present embodiment is created for any one of the articles such as cosmetics, books, and DVDs as described above.

  The article code in the article master table 63 is identification information for identifying individual articles as described above. For example, when the article is a cosmetic, a JAN (Japanese Article Number) code is used. When the article is a book, an ISBN (International Standard Book Number) code is used. The article code is not limited to JAN or ISBN, and any article code may be used as long as each article can be identified. Since the same article code is assigned to the same article, for example, even if there are a plurality of identical articles in stock, only one article code table corresponding to the article exists.

  The entire image information represents the overall appearance of the article, and specifically is image data obtained by photographing the entire article from the front, rear, left and right with a camera or the like. The entire image information registered in the article master table 63 may be image data itself or link information of a connection destination such as a URL (Uniform Resource Locator) of a data server in which the image data is stored or a directory of a storage folder. Further, the whole image information includes image data taken from the direction that can be the front direction when the article is displayed on the display shelf. However, for example, in the case of a cylindrical article, the whole image information is not limited to the front / rear / left / right direction. Image data from the direction may be included.

  The similar article code is an article code of an article having overall image information similar to the article specified by the article code.

  The attribute information is information belonging to an article specified by the article code, and may be any information as long as it is information related to the article. For example, when the article is a cosmetic, as the type of attribute information, for example, as shown in FIG. 2, the “product name” of the cosmetic, the “price” of the cosmetic, the lotion, the lipstick, the foundation, etc. There are “classification”, “manufacturer” of cosmetics, “manufacturing year” of cosmetics, “size (height)” and “size (width)” of cosmetics.

  For example, when the article is a book, the types of attribute information include, for example, the “title” of the book, the “price” of the book, the “classification” of the book such as a paperback book, a book, a comic, and a magazine, “Publisher”, “Series Number” of the book, “Volume” of the book such as Volume 1, Volume 2 and Volume 3, “Number of Book” such as January, February and March, “Issuance of Book” "Year", "Author" of the book, "Size (height)" and "Size (width)" of the book.

  For example, when the article is a DVD, the types of attribute information include, for example, “title” such as a movie recorded on the DVD, “price” of the DVD, action, animation, horror, documentary, etc. “Classification” of recorded movies, “Publisher” issuing DVD, “Standard part number symbol” of DVD, “Volume” of DVD such as 1, 2, 3, DVD, “Issued year” of DVD, There are “size (height)” and “size (width)” of a DVD package.

  In the above description, attribute information of cosmetics, books, DVDs, and CDs has been described. However, the present invention is not limited to this, and various attribute information is registered according to the type of article.

  In the article master table 63, the types of attribute information as described above and their contents are registered in association with each other. The content of the attribute information is, for example, when the type of attribute information is “manufacturer” of cosmetics, and is the specific name of the manufacturer “XX company”, and the type of attribute information is cosmetics. Is “2012”, which is a specific year of manufacture of the cosmetic product.

  The partial feature information is information of a partial range of the appearance of the article. Specifically, the type and content of the above-described attribute information appearing in a partial range of the appearance of the article and the position information of the partial range in which the content of the attribute information appears are registered as partial feature information Is done.

  In this embodiment, the X coordinate, the Y coordinate, and the size when the horizontal direction is the X direction and the vertical direction is the Y direction are registered as position information of a part of the range. As the X and Y coordinates of a part of the range, when the part of the range is defined by a rectangular area, for example, the position coordinates of the upper left corner of the rectangular area or the center coordinates of the rectangular area are used. be able to. Also, as the size of the partial range, the size in the X direction and the size in the Y direction of the partial range are registered. Note that the position information of the partial range is relative positional information in the overall appearance of the article, and the content of the attribute information appearing in the partial range of the appearance of the article is directed to the front direction. Position information.

  Further, the number of partial feature information is not necessarily one, and a plurality of combinations of types and contents of attribute information and their position information may be registered.

  In the article master table 63, the article codes of all the in-stock articles are registered in association with the above-described entire image information, similar article codes, attribute information, and partial feature information.

  Next, the article master table registration system 1 for creating and registering the article master table 63 as described above will be described.

  As shown in FIG. 1, the article master table registration system 1 according to the present embodiment includes a camera 40, an article master table creation device 50, and a monitor 70.

  The camera 40 captures the overall appearance of the article to be registered in the article master table 63, and is registered in the article master table 63 based on image data obtained by capturing the overall appearance of the article by the camera 40. The entire image information is acquired. Therefore, the imaging | photography with the camera 40 is performed from the front-back, left-right direction of the goods for registration, for example. If the image data includes a barcode image representing an article code, an article code registered in the article master table 62 is acquired based on the barcode image. If the image data of the overall appearance of the registration target article does not include a barcode image representing the article code, the part including the barcode image is separately photographed by the camera 40. Further, the barcode image may be separately read by a barcode reader. Note that photographing of the registration target article is performed for each article.

  In the present embodiment, the entire image information and the article code of the article to be registered are acquired using the image data taken by the camera 40. For example, when the article is a book, You may make it acquire image data using a scanner.

  In addition, image data including the overall appearance of the article and image data including a barcode image are not necessarily acquired by photographing with a camera or photoelectric reading with a scanner. For example, in the process of editing a book, Image data of a front cover, a back cover, and a back cover created in advance may be acquired, and the entire image information and article code may be acquired based on the image data.

  As shown in FIG. 1, the article master table creation device 50 includes an image acquisition unit 51, an image region extraction unit 52, a similarity determination unit 53, a partial region extraction unit 54, a character recognition unit 55, an article code region extraction unit 56, an article. A code recognition unit 57, an attribute information acquisition unit 58, an article attribute information database 59, a character match determination unit 60, a master table creation unit 61, a master database 62, and a communication unit 64 are provided. In the present embodiment, the partial region extraction unit 54 and the character recognition unit 55 correspond to the character extraction unit in the claims, and the character matching determination unit 60 corresponds to the character determination unit in the claims.

  The article master table creation device 50 is obtained by installing an embodiment of the article master table creation program of the present invention in a computer. When the article master table creation program is executed by the central processing unit (CPU), the image acquisition unit 51, the image region extraction unit 52, the similarity determination unit 53, the partial region extraction unit 54, the character recognition unit 55, the article code The area extraction unit 56, the article code recognition unit 57, the attribute information acquisition unit 58, the article attribute information database 59, the character match determination unit 60, and the master table creation unit 61 function. The article master table creation program may be recorded on a recording medium such as a CD-ROM, or may be downloaded from a server or the like via the Internet.

  The image acquisition unit 51 acquires image data of an article to be registered photographed by the camera 40 and temporarily stores it in a storage medium such as a semiconductor memory or a hard disk.

  The image area extraction unit 52 reads the image data of the registration target article from the image acquisition unit 51, and acquires the article area image data by deleting the image data of the background of the article from the image data. Various known methods can be used as a method for acquiring the article region image data. The article region image data (for example, front / rear / left / right image data) acquired by the image region extraction unit 52 is output to the master table creation unit 61 as the entire image information of the registration target article.

  The similarity determination unit 53 acquires the article region image data acquired by the image region extraction unit 52, and collates the article region image data with the entire image information of a plurality of articles already registered in the article master table 63. Then, an article code having overall image information including image data similar to the article area image data of the article to be registered is acquired as a similar article code. The matching between the article region image data and the entire image information of the registered article may be performed, for example, by calculating the similarity using template matching. The similar product code acquired by the similarity determination unit 53 is output to the master table creation unit 61 as a similar product code of the registration target product.

  There is a possibility that the partial region extraction unit 54 receives the article region image data acquired by the image region extraction unit 52, and the contents of the above-described attribute information of the registration target article appear based on the article region image data. A partial area is extracted.

  Specifically, the partial region extraction unit 54 extracts a region where character information appears in the article region image data as a partial region. For example, the partial area extraction unit 54 binarizes the article area image data, performs labeling on the binarized data, and extracts a mass area where character information is supposed to exist based on the labeling information as a partial area To do. Note that the character information here includes not only hiragana, katakana, kanji and English characters, but also numbers and symbols. In addition to a plurality of characters, one character or the like is included.

  The extraction method of the character information area (partial area) is not limited to this, and various known methods can be used. Also, the number of partial areas is not limited to one, and when a plurality of character information areas are extracted, the plurality of character information areas may be extracted as partial areas. The labeling data of the partial area extracted by the partial area extraction unit 54 is output to the character recognition unit 55.

  The character recognition unit 55 performs character recognition processing based on the labeling data in the partial region extracted by the partial region extraction unit 54 and recognizes character information existing in the partial region. Since the character recognition process is already known, a detailed description thereof will be omitted. The character information recognized by the character recognition unit 55 is output to the character match determination unit 60.

  The article code area extraction unit 56 reads image data of a registration target article from the image acquisition unit 51, and extracts an area where a barcode image representing an article code exists from the image data as an article code area. As for the extraction of the area where the barcode image exists, an area where a plurality of line segments of a predetermined length are arranged may be extracted, but various known methods can be used for the extraction method. Information on the article code area extracted by the article code area extraction unit 56 is output to the article code recognition unit 57.

  The article code recognizing unit 57 acquires the article code of the article to be registered by recognizing a barcode image existing in the article code area based on the inputted article code area information. The article code acquired by the article code recognition unit 57 is output to the master table creation unit 61 as the article code of the registration target article.

  The attribute information acquisition unit 58 acquires the product code recognized by the product code recognition unit 57, and acquires the type and content of the attribute information of the registration target product from the product attribute information database 59 based on the content of the product code. To do. As described above, the type of attribute information includes, for example, “series number”, “number of volumes”, “number”, “standard product number symbol” or “year of manufacture”. ”,“ Number of turns ”,“ number ”,“ standard part number symbol ”or“ production year ”.

  Then, the type and content of the attribute information acquired by the attribute information acquisition unit 58 are output to the master table creation unit 61 as the attribute information of the registration target article. The attribute information of the registration target article is not limited to that acquired based on the article code, and may be acquired by, for example, setting input using an input unit by the user. The content of the attribute information acquired by the attribute information acquisition unit 58 is also output to the character match determination unit 60.

  The article attribute information database 59 stores the types and contents of attribute information represented by various article codes.

  The character match determination unit 60 compares the character information recognized by the character recognition unit 55 with the contents of the attribute information acquired from the attribute information acquisition unit 58, and if they match, the character information is represented by the character information. The contents and type of the attribute information to be performed and the position information of the partial area where the character information exists are output to the master table creation unit 61 as partial feature information of the article to be registered. Specifically, when both the character information recognized by the character recognition unit 55 and the content of the attribute information acquired from the article attribute acquisition unit 58 are “222333”, the “222333” and the “ The type of the attribute information “222333” (for example, the series number) and the position information of the partial area where the character information “222333” exists are registered in the article master table 62 as the partial feature information of the article to be registered. On the other hand, the character information recognized by the character recognition unit 55 is compared with the content of the attribute information acquired from the attribute information acquisition unit 58. If they do not match, the character match determination unit 60 determines that the character information Is not recognized as partial feature information.

  The master table creation unit 61 outputs the article code of the registration target article output from the article code recognition unit 57, the entire image information of the registration target article output from the image area extraction unit 52, and the similarity determination unit 53. The similar article code of the registration target article, the attribute information of the registration target article output from the attribute information acquisition unit 58, and the partial feature information of the registration target article output from the character match determination unit 60 are associated with each other. An article master table 63 is created and output to the master database 62 for storage.

  The master database 62 stores an article master table 63 for registered articles. The master database 62 may store the image data 63a of the entire image information of each article in the article master table 63.

  The article master table 63 in the article master table creation device 50 is updated each time a new registration target article table is registered. Then, the article master table 63 in the article master table creation device 50 is output to the article matching device 20 of the article matching system 2 described later via the communication unit 64 and the communication network, and the article master table in the article master table creation device 50. The article master table 26 in the article collating apparatus 20 is updated by overwriting the contents 63 in the article master table 26 in the article collating apparatus 20. The article master table 26 of the article collation apparatus 20 may be updated every time the article master table 63 of the article master table creation apparatus 50 is updated, or may be periodically updated every predetermined period. Alternatively, it may be performed at an arbitrary timing in accordance with an update instruction input by the user. The article verification device 20 will be described in detail later.

  The monitor 70 displays the article master table 63, the article area image data acquired by the image area extraction unit 52, and the like.

  Next, the operation of the article master table registration system 1 of the present embodiment will be described with reference to the flowcharts shown in FIGS.

  First, an article to be registered is installed at a predetermined position by the user (S50). Then, the entire appearance of the article to be registered is photographed by the user using the camera 40 (S52). At this time, if the image data of the overall appearance of the article does not include the barcode image representing the article code, the camera 40 separately captures the portion including the barcode image.

  Next, the image data photographed by the camera 40 is acquired and temporarily stored by the image acquisition unit 51 of the article master table creation device 50 connected to the camera 40 by wire or wirelessly.

  The image data acquired by the image acquisition unit 51 is output to the article code area extraction unit 56, and the article code area extraction unit 56 defines the area where the barcode image representing the article code is present from the input image data as the article code area. (S54).

  Information on the article code area extracted by the article code area extraction unit 56 is output to the article code recognition unit 57. The article code recognition unit 57 stores the information in the article code area based on the inputted article code area information. Is recognized (S56), and the article code of the article to be registered is acquired (S58). The article code acquired by the article code recognition unit 57 is output to the master table creation unit 61 as the article code of the registration target article.

  Next, the image data acquired by the image acquisition unit 51 is output to the image region extraction unit 52, and the image region extraction unit 52 deletes the image data of the background of the article from the input image data, thereby performing the article region image. Get the data. Then, the article region image data (for example, front / rear / left / right image data) acquired by the image region extraction unit 52 is output to the master table creation unit 61 as entire image information of the registration target article (S60).

  Further, the article region image data acquired by the image region extraction unit 52 is output to the similarity determination unit 53, and the similarity determination unit 53 includes a plurality of pieces of article region image data that has been input and a plurality of items already registered in the article master table 63. The product code having the overall image information including image data similar to the product region image data of the registration target product is acquired as a similar product code (S62). The similar product code acquired by the similarity determination unit 53 is output to the master table creation unit 61 as a similar product code of the registration target product.

  Next, the article code recognized by the article code recognition unit 57 is output to the attribute information acquisition unit 58, and the attribute information acquisition unit 58 stores the registration target article from the article attribute information database 59 based on the contents of the article code. The type of attribute information and its contents are acquired (S64). Then, the type and content of the attribute information acquired by the attribute information acquisition unit 58 are output to the master table creation unit 61 as the attribute information of the registration target article.

  Further, partial feature information is acquired for the registration target article. The operation of acquiring the partial feature information will be described with reference to the flowchart shown in FIG.

  First, the article region image data acquired by the image region extraction unit 52 is input to the partial region extraction unit 54, and the partial region extraction unit 54 performs a character in the article region image data based on the input article region image data. The area where the information appears is extracted as a partial area (S70).

  Then, the labeling data of the partial region extracted by the partial region extraction unit 54 is output to the character recognition unit 55, and the character recognition unit 55 performs character recognition processing based on the labeling data in the partial region, and performs the character recognition processing within the partial region. Is recognized (S72). The character information recognized by the character recognition unit 55 is output to the character match determination unit 60.

  On the other hand, the content of the attribute information acquired based on the article code in the attribute information acquisition unit 58 is also output to the character match determination unit 60.

  Then, the character match determination unit 60 compares the character information recognized by the character recognition unit 55 with the contents of the attribute information acquired from the attribute information acquisition unit 58, and if they match, the character information Is output to the master table creation unit 61 as the partial feature information of the article to be registered (S76). On the other hand, the character information recognized by the character recognition unit 55 is compared with the content of the attribute information acquired from the attribute information acquisition unit 58. If they do not match, the character match determination unit 60 determines that the character information Is not recognized as partial feature information.

  Then, the master table creation unit 61 includes the article code of the registration target article output from the input article code recognition unit 57, the entire image information of the registration target article output from the image region extraction unit 52, and the similarity determination unit. The similar article code of the registration target article output from 53, the attribute information of the registration target article output from the attribute information acquisition unit 58, and the partial feature information of the registration target article output from the character match determination unit 60 are associated with each other. Then, an article master table 63 of the registration target articles is created, and this is output and stored in the master database 62 (S68).

  As described above, registration of a new registration target article to the article master table 63 is performed.

  According to the article master table registration system 2 of the above embodiment, an image representing the appearance of an article to be registered is acquired, characters in the acquired image are extracted, attribute information of the article to be registered is acquired, and extraction is performed. Compare the acquired character with the acquired attribute information, determine whether the character represents attribute information, and if it is determined that the character represents attribute information, the character and the image of the character Since the article master table 63 including the position information within the article is created, for example, in the case of collating with the article master table 63 by paying attention to partial character information of the appearance of the article, the article master table described above is used. By using the position information included in 63, the position of the character in the verification target article can be appropriately specified.

  In addition, since the character and the position information of the character in the image of the item are specified based on the attribute information of the item and are included in the item master table 63, the character and the position information of the character are easily and accurately included. It can be specified and included in the article master table 63.

  Next, an article collation system 2 that performs an inventory check process of an article by collating an article master table including partial feature information as described above with image data obtained by photographing the article displayed on the display shelf will be described. FIG. 5 is a block diagram illustrating a schematic configuration of the article matching system 2.

  As shown in FIG. 5, the article collation system 2 includes a camera 10, an article collation apparatus 20, and a monitor 30.

  The camera 10 is for photographing the object to be collated displayed on the display shelf. Image data photographed by the camera 10 is output to the article collation apparatus 20. Note that it is more efficient for the camera 10 to shoot a plurality of items to be collated together. However, the present invention is not limited to this, and each shoot may be taken separately.

  As shown in FIG. 5, the article verification device 20 includes an image acquisition unit 21, a single product image extraction unit 22, a first verification unit 23, a second verification unit 24, a master database 25, and a communication unit 29. I have.

  The article collation apparatus 20 is obtained by installing an embodiment of the article collation program of the present invention on a computer. Then, when the article collation program is executed by the central processing unit (CPU), the image acquisition unit 21, the single product image extraction unit 22, the first collation unit 23, the second collation unit 24, and the communication unit 29 function. The master database 25 is referred to by the first verification unit 23 and the second verification unit 24. The article collation program may be recorded on a recording medium such as a CD-ROM, or may be downloaded from a server or the like via the Internet.

  The image acquisition unit 21 acquires the image data of the object to be collated photographed by the camera 10 and temporarily stores it in a storage medium such as a semiconductor memory or a hard disk.

  The single product image extraction unit 22 reads the image data of the verification target article from the image acquisition unit 21, and when the image data includes image data of a plurality of verification target articles, the single product image of each verification target article. Data is divided and extracted. As for the extraction of single product image data, for example, a rectangular area included in captured image data may be automatically extracted, and the image data of each rectangular area may be used as single product image data, or the external shape of an article may be determined. Template data to be represented may be stored in advance, and single product image data may be extracted by performing template matching between captured image data and template data. It should be noted that various other known methods can be used for the extraction of single item image data.

  In addition, when the captured image data includes only image data of one item to be collated, the captured image data may be used as single image data as it is, or the background of the article may be determined from the captured image data. The image data removed may be extracted as single product image data.

  The first collation unit 23 and the second collation unit 24 perform collation processing on the collation target article based on the single item image data output from the single item image extraction unit 22 and the article master table 26 registered in the master database 25 in advance. Is to do. The article master table 26 registered in the master database 25 is registered and updated by acquiring the article master table 63 created by the article master table registration system 1 described above via the communication network and the communication unit 29. Is.

  The first verification unit 23 performs the first verification process using the single item image data of the verification target article output from the single item image extraction unit 22 and the entire image information of each item registered in the item master table 26. Is what you do.

  The first matching process is performed, for example, by template matching between single item image data of a matching target article and image data of each article in the article master table 26. Specifically, for example, using a predetermined correlation function, the degree of similarity between the single item image data of the item to be verified and the image data of each item in the item master table 26 is calculated, and the degree of similarity is greater than a predetermined reference value. Articles with overall image information including high image data are extracted from the article master table 26. The first matching process is not limited to the template matching as described above, and other known matching processes may be used.

  When a plurality of article candidates are extracted from the article master table 26 by the first matching process described above, the second matching unit 24 performs the second matching process to further narrow down the plurality of article candidates. Is to do. In the second matching process, the single item image data of the matching target article corresponding to the attribute information included in the partial feature information of the plurality of article candidates extracted by the first matching process and the position information included in the partial feature information. Is done by comparing with a part of. The second verification process will be described later in detail.

  The master database 25 stores a matching result stock table 27 and a management stock table 28 as shown in FIG. 5 together with the article master table 26 described above.

  As shown in FIG. 6, the verification result inventory table 27 associates each article code with the number of articles specified by the article code. The matching result inventory table 27 is updated based on the result of the matching process in the first matching unit 23 or the second matching unit 24 described above. Specifically, when one article corresponding to a predetermined object to be collated is extracted from the article master table 26 by the collation process, the inventory number of the article code of the article in the collation result inventory table 27 is incremented by one. The verification result stock table 27 is updated.

  Similarly to the matching result inventory table 27, the management inventory table 28 is also a table in which each article code is associated with the number of articles specified by the article code. It is updated every time an article is carried in and out. Specifically, each time an article is brought into a store or a library, the management inventory table 28 increments the number of inventory corresponding to the article code of the article by one, and whenever the article is carried out, The stock quantity corresponding to the article code of the article is decremented by one and updated. Note that the delivery of an article at the store means that the article has been received, and the delivery of the article means that the article has been sold and taken out.

  That is, the management inventory table 28 represents the inventory status of articles managed in a store or library, and the matching result inventory table 27 represents the inventory status of articles actually present in the store or library. It is. The management stock table 28 and the matching result stock table 27 are originally identical when the above-described matching processing is completed. However, for example, if an article is stolen or there is a mistake in updating the management stock table 28 at the time of carrying in or carrying out the article, these tables will not match. The verification result stock table 27 and the management stock table are displayed on the monitor 30.

  The communication unit 29 communicates with an article master table registration system 1 to be described later via a communication network such as the Internet or a LAN, and the article master table 26 created in the article master table registration system 1 as described above is used. It is acquired via a communication network and output to the master database 25.

  As described above, the monitor 30 displays the matching result inventory table 27 and the management inventory table 28. In addition, the monitor 30 also displays image data acquired by the article master table 26 and the image acquisition unit 21, and single item image extraction. The single item image data extracted by the unit 22 is also displayed.

  Next, the operation of the article matching system 2 of the present embodiment will be described with reference to the flowchart shown in FIG.

  First, a plurality of items to be collated displayed on the display shelf are arranged by the user (S10). This is an operation performed so that the collation processing based on the photographed image data of the article as described above is appropriately performed. At this time, each article to be collated is arranged so that the attribute information included in the partial feature information of each article described above faces the front direction, but the user does not grasp the content of the attribute information of the partial feature information Are arranged so that a predetermined character string appearing in a part of the appearance of the article faces the front.

  Next, after the arrangement of the articles as described above is completed, the user photographs the articles arranged on the display shelf using the camera 10 (S12). Specifically, as shown in FIG. 8, for example, the shooting range of the camera 10 is set so that a plurality of items to be collated 101 to 105 displayed on a predetermined shelf of the display shelf 200 are included. Is called. Note that the shooting range is not limited to the shooting range as shown in FIG. 8, and for example, the shooting range may be set so as to include all of a plurality of articles displayed on one shelf. The shooting range may be set so that the entire shelf 200 is included.

  And the image data image | photographed with the camera 10 is acquired by the image acquisition part 21 of the goods collation apparatus 20 connected to the camera 10 by wire or radio | wireless, and is once stored (S14).

  Next, the image data stored in the image acquisition unit 21 is read out by the single product image extraction unit 22, and the single product image extraction unit 22 classifies the single product image data of each target article for matching from the read image data. To extract (S16).

  Then, the single product image data extracted by the single product image extraction unit 22 is output to the first verification unit 23, and the first verification unit 23 performs a first verification process using the input single product image data (S18). .

  Specifically, the first matching unit 23 performs template matching between the single product image data and the image data of the entire image information of each article registered in the article master table 26, and the similarity is based on a predetermined reference value. The article of the entire image information including the higher image data is extracted as a corresponding article candidate.

  If only one item candidate is extracted by this first collation process (S20, YES), information on the item master table 26 of the item candidate is displayed on the monitor 30 (S28). Next, when the information displayed on the monitor 30 is confirmed by the user and confirmed to be the corresponding article, the user inputs an instruction to update the matching result inventory table 27 using a predetermined input unit. The first verification unit 23 updates the verification result inventory table 27 by incrementing the inventory number corresponding to the article code of the corresponding article in the verification result inventory table 27 by one according to the instruction input (S30).

  On the other hand, when a plurality of article candidates are extracted by the first matching process (S20, NO, S22, YES), the second matching unit 24 extracts the partial feature information of the plurality of article candidates and a single item. A second collation process is performed based on a part of the image data (S24).

  Here, the second matching process will be described in detail with reference to the flowchart shown in FIG.

  In the second matching process, first, partial feature information is obtained for each of a plurality of article candidates with reference to the article master table 26 (S40).

  Next, position information and attribute information included in the partial feature information of a predetermined article candidate among the plurality of article candidates are acquired (S42). Then, when the acquired position information is, for example, the X coordinate is X1, the Y coordinate is Y1, and the size is W1 × H1, the rectangular area indicated by the position information is displayed on the single product image data as shown in FIG. A part of the single item image data in the rectangular area is collated with the content of the attribute information included in the partial feature information (S44). Specifically, for example, the type of attribute information included in the partial feature information is “series number”, “volume”, “number”, “standard product number symbol”, or “production year”, and the attribute information Is a number representing the attribute information, collation is performed by recognizing whether or not the number is included in a part of the single item image data in the rectangular area. Then, if the number is included in a part of the single item image data in the rectangular area, it is assumed that the item candidate is the corresponding item. The second matching process as described above is performed for all of the plurality of article candidates. Further, when a plurality of partial feature information is registered for one article candidate, the above-described collation is performed for all the partial feature information.

  Then, returning to the flowchart shown in FIG. 7, when one article candidate is extracted by the above-described second collation process (S26, YES), information on the article master table 26 of the article candidate is displayed on the monitor 30. (S28). Next, when the information displayed on the monitor 30 is confirmed by the user and confirmed to be the corresponding article, the user inputs an instruction to update the verification result inventory table 27 using a predetermined input unit. In response to the instruction input, the second verification unit 24 updates the verification result inventory table by incrementing the inventory number corresponding to the article code of the corresponding article in the verification result inventory table 27 by 1 (S30).

  On the other hand, when one article candidate is not extracted by the second matching process (S26, NO), the monitor 30 displays that the corresponding article could not be extracted (S32). Then, the user who has confirmed the monitor 30 directly confirms the appearance, product code, and the like of the verification target article directly (S34), and updates the verification result inventory table 27 based on the confirmed result (S30).

  In addition, not only when one article candidate is not extracted by the second matching process, but also when a corresponding article candidate is not extracted in the first matching process described above, the corresponding article candidate is extracted. The fact that it was not possible is displayed on the monitor 30 (S32), and the user confirming the monitor 30 directly confirms the appearance, product code, etc. of the verification target article manually (S34), and based on the confirmation result The verification result stock table 27 is updated (S30).

  The first matching process and the second matching process as described above are performed in the same manner on the single item image data of all matching target articles, and the matching result inventory table 27 is updated. In the present embodiment, as described above, the matching result inventory table 27 is updated each time the first and second matching processes for one matching target article are performed. After the first and second matching processes for all the matching target articles are completed, an instruction may be input so as to collectively update the matching result inventory table 27 of all the matching target articles based on the result. .

  Note that when all the verification target articles have been collated, the number of stocks in the collation result stock table 27 and the number of stocks in the management stock table 28 are compared. Information to be displayed may be displayed on the monitor 30.

  The collation processing method in the article collation system 2 is an example of the collation processing method using the article master table 26, and the article master table 26 can be used for other collation processing methods. is there.

DESCRIPTION OF SYMBOLS 1 Article master table registration system 2 Article collation system 10 Camera 20 Article collation apparatus 21 Image acquisition part 22 Single item image extraction part 23 1st collation part 24 2nd collation part 25 Master database 26 Goods master table 27 Collation result stock table 28 Management stock Table 40 Camera 50 Article master table creation device 51 Image acquisition section 52 Image area extraction section 53 Similarity determination section 54 Partial area extraction section 55 Character recognition section 56 Article code area extraction section 57 Article code recognition section 58 Attribute information acquisition section 59 Article attribute Information database 60 Character match determination unit 61 Master table creation unit 62 Master database 63 Article master table

Claims (11)

An image acquisition unit that acquires an image representing the appearance of an article to be registered;
A character extraction unit for extracting characters in the image acquired by the image acquisition unit;
An attribute information acquisition unit for acquiring attribute information of the article to be registered;
A character determination unit that compares the character extracted by the character extraction unit with the attribute information acquired by the attribute information acquisition unit, and determines whether the character represents the attribute information;
When the character determining unit determines that the character represents the attribute information, the character position information in the image is acquired, and the article, the character, and the character position information are associated with each other. attached to the article master table creating apparatus characterized by comprising a master table creation unit for creating an article master table of said article. An identification information acquisition unit for acquiring identification information for identifying the article to be registered;
The article master table creation device according to claim 1, wherein the attribute information acquisition unit acquires the attribute information based on the identification information. The image acquisition unit acquires an image of the registration target article including the identification information;
3. The article master table creation device according to claim 2, wherein the identification information acquisition unit acquires the identification information based on an image including the identification information.   The master table creation unit creates the article master table by associating the identification information with characters representing the attribute information and position information of the characters in the image. 2. The article master table creation device according to 2 or 3.   5. The article master table creation device according to claim 1, wherein the master table creation unit creates the article master table including a size of a portion where the characters appear. .   6. The attribute information according to claim 1, wherein the attribute information is at least one of a series number, a volume number, a number number, a standard product number symbol, a maker, a publisher, a production year, an issue year, and an author. Item master table creation device according to item.   The article master table creation device according to claim 1, wherein the position information is position information of a rectangular area surrounding the character.   The article master table creation according to any one of claims 1 to 7, wherein the master table creation section creates the article master table including an image representing an appearance of the article to be registered. apparatus.   The article master table creation device according to any one of claims 1 to 8, wherein the article is any one of cosmetics, books, and recording medium products on which predetermined information is recorded. Obtain an image that represents the appearance of the item to be registered
Extracting characters in the acquired image;
Obtaining attribute information of the article to be registered;
Compare the extracted character and the acquired attribute information, determine whether the character represents the attribute information,
When it is determined that the character represents the attribute information, the character position information in the image is acquired, and the article, the character, and the character position information are associated with each other. An article master table creation method comprising creating an article master table. An image acquisition unit for acquiring an image representing an appearance of an article to be registered;
A character extraction unit for extracting characters in the image acquired by the image acquisition unit;
An attribute information acquisition unit for acquiring attribute information of the article to be registered;
A character determination unit that compares the character extracted by the character extraction unit with the attribute information acquired by the attribute information acquisition unit, and determines whether the character represents the attribute information;
When the character determining unit determines that the character represents the attribute information, the character position information in the image is acquired, and the article, the character, and the character position information are associated with each other. attached to the article master table creation program for causing to function as a master table creating unit for creating an article master table of said article.

Images