JP2012053711A - Information processor, store system and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce burden on a salesclerk when checking an inventory tag.SOLUTION: An information processor 1 includes an image data capturing part 601, an inventory tag recognition part 602, a bar code reading part 603, a character recognition part 604, a collation part 605, and a notification part 606. The image data capturing part 601 captures an image photographed by a camera 50. The inventory tag recognition part 602 recognizes an inventory tag included in the captured image. The bar code reading part 603 reads commodity identification information to identify a commodity from the recognized inventory tag. The character recognition part 604 recognizes characters written on the recognized inventory tag. The collation part 605 refers to commodity registration information in which information associated with the commodity is registered for each of the commodity identification information and performs collation between the information registered corresponding to the read commodity identification information and the recognized characters. The notification part 606 performs notification according to the collation result.

Description

  Embodiments described herein relate generally to an information processing apparatus, a store system, and a program.

  2. Description of the Related Art Conventionally, an electronic shelf label system in which a shelf label that displays a price of a product is digitized is known. In this electronic shelf label system, a code identifying a product and a code identifying a shelf label are read by a handy terminal or the like, and the product and the shelf label are associated with each other by the electronic shelf label system. Then, the product data is transmitted to the shelf label associated with the product via a wireless or wired network, and information such as the product name and price is displayed on the shelf label.

  By the way, since the shelf labels of the electronic shelf label system are unified with the same size and are not conspicuous, the shelf labels for sale etc. may be printed large on paper. Further, in a store that does not introduce an electronic shelf label system, a shelf label on which information such as a product name and a price is printed is used for all products to be displayed. With regard to shelf labels other than these electronic shelf label systems, the store clerk looks around the store because there is a tendency for the product name and price on the system to be different from the product name and price printed on the shelf label. It is necessary to check, but this check work is a heavy burden on the store clerk.

  In order to solve the above-described problem, an embodiment of the present invention includes an image capturing unit, a shelf label recognition unit, a reading unit, a character recognition unit, a collation unit, and a notification unit. Is an information processing apparatus. The image capturing unit captures an image captured by the imaging unit. The shelf label recognition means recognizes a shelf label included in the captured image. The reading unit reads product identification information for identifying a product from the recognized shelf label. The character recognizing means recognizes the characters described on the recognized shelf label. The collating means refers to the product registration information in which information related to the product is registered in advance for each product identification information, the information registered corresponding to the read product identification information, and the recognized character Perform verification. The notification means performs notification according to the result of collation.

  Moreover, embodiment of this invention is a store system characterized by having an imaging device and information processing apparatus. The imaging device images the inside of the store. The information processing apparatus includes an information processing apparatus including an image capturing unit, a shelf label recognition unit, a reading unit, a character recognition unit, a collation unit, and a notification unit. The image capturing means captures an image captured by the imaging device. The recognizing unit recognizes the shelf label included in the captured image. The shelf label reading means reads product identification information for identifying a product from the recognized shelf label. The character recognizing means recognizes the characters described on the recognized shelf label. The collating means refers to the product registration information in which information related to the product is registered in advance for each product identification information, the information registered corresponding to the read product identification information, and the recognized character Perform verification. The notification means performs notification according to the result of collation.

  Further, the embodiment of the present invention causes the computer of the information processing apparatus to function as an image capture unit, a shelf label recognition unit, a reading unit, a character recognition unit, a collation unit, and a notification unit. Is a program characterized by The image capturing unit captures an image captured by the imaging unit. The shelf label recognition means recognizes a shelf label included in the captured image. The reading unit reads product identification information for identifying a product from the recognized shelf label. The character recognizing means recognizes the characters described on the recognized shelf label. The collating means refers to the product registration information in which information related to the product is registered in advance for each product identification information, the information registered corresponding to the read product identification information, and the recognized character Perform verification. The notification means performs notification according to the result of collation.

FIG. 1 is a conceptual diagram illustrating an example of use of an information processing apparatus in a store. FIG. 2 is an external view showing the front external appearance of the information processing apparatus. FIG. 3 is an external view showing the rear external appearance of the information processing apparatus. FIG. 4 is a block diagram illustrating a configuration of the information processing apparatus. FIG. 5 is a flowchart illustrating an example of the operation of the information processing apparatus. FIG. 6 is an external view of the information processing apparatus showing a display example on the display. FIG. 7 is a system configuration diagram schematically showing the overall configuration of the store system. FIG. 8 is a diagram illustrating an example of a display screen of the information processing apparatus in the store system.

  Hereinafter, an information processing apparatus, a store system, and a program according to the present embodiment will be described with reference to the drawings.

[First embodiment]
The first embodiment exemplifies an information processing apparatus used when a store clerk looks around a store to check the product name and price on the system and the product name and price printed on the shelf label. FIG. 1 is a conceptual diagram illustrating an example of use of an information processing apparatus in a store. As shown in FIG. 1, in the first embodiment, the store clerk H checks the information displayed on the information processing device 1 while holding the information processing device 1 and looks around the store 100, and prints it on the shelf label. Check the product name and price.

  FIG. 2 is an external view showing the front external appearance of the information processing apparatus 1. FIG. 3 is an external view showing the back external appearance of the information processing apparatus 1. As shown in FIG. 2, a display 20 such as an LCD (Liquid Crystal Display), a touch panel 30 stacked on the display 20, and operations for performing various operation inputs are provided on the front surface of the housing 10 of the information processing apparatus 1. Key 40. As shown in FIG. 3, a camera 50 such as a CMOS image sensor or a CCD image sensor is provided on the back surface of the housing 10 of the information processing apparatus 1.

  The store clerk H holds the information processing apparatus 1 with his / her hand on the side or back of the housing 10, and confirms and uses the operation input by the touch panel 30 or the operation key 40 or the screen display of the display 20. When the image captured by the camera 50 is displayed on the display 20 during this use, the camera 50 captures the direction in which the store clerk H looks at the display 20 and the camera 50 because the camera 50 is provided on the rear surface of the housing 10. The direction is substantially the same.

  FIG. 4 is a block diagram illustrating a configuration of the information processing apparatus 1. The information processing apparatus 1 includes a CPU 60 (Central Processing Unit), a ROM 62 (Read Only Memory), a RAM 63 (Random Access Memory), a communication I / F 64, an input / output port 65, an operation controller 66, and a display controller 67. And a camera controller 68 and an HDD 69, which are connected by a bus line 61 such as an address bus or a data bus.

  The CPU 60 functions as a control unit that controls the overall operation of the information processing apparatus 1. The ROM 62 stores programs according to the present embodiment, other programs that control the operation of the CPU 60, and the like. The RAM 63 functions as a work area for the CPU 60 when developing various programs stored in the ROM 62. The RAM 63 stores various data such as image data captured by the camera 50.

  The communication I / F 64 controls data communication with an information device such as a store computer SC that manages and manages store data connected via a wired or wireless connection. The input / output port 65 is a communication port for connecting to an external connection I / F 651 such as a USB (Universal Serial Bus) or a buzzer 652. The input / output port 65 performs input / output of data in the external connection I / F 651 and output of a drive signal to the buzzer 652 under the control of the CPU 60. The operation controller 66 takes in operation signals from the touch panel 30 and the operation keys 40. The display controller 67 controls data display on the display 20. The camera controller 68 controls the imaging operation of the camera 50.

  An HDD 69 (Hard Disk Drive) stores programs and various files. All or a part of the programs and various files stored in the HDD 69 are copied to the RAM 63 and sequentially executed by the CPU 60 when the information processing apparatus 1 is activated. An example of a program stored in the HDD 69 is a program PR that is activated when a store clerk checks a shelf label. An example of a file stored in the communication I / F 64 is a PLU file F distributed and stored from the store computer SC.

  The PLU file F is a file for storing information relating to a product such as a product name, a price, and a product classification for each product code (product identification information) uniquely assigned to each product in order to identify the product. That is, the PLU file F is product registration information in which information related to products is registered in advance for each product identification information. The information processing apparatus 1 refers to the information stored in the PLU file F with the product code read by the barcode reading unit 603 (details will be described later), and information related to the product corresponding to the read product code. To get.

  Here, a functional configuration of the information processing apparatus 1 realized by the CPU 60 executing the program will be described. The CPU 60 operates in accordance with a program stored in the ROM 62 expanded in the RAM 63 and a program PR, so that, as shown in FIG. 4, the image data capturing unit 601, the shelf label recognition unit 602, the barcode reading unit 603, , Character recognition unit 604, collation unit 605, and notification unit 606.

  The image data capturing unit 601 outputs an imaging on signal to the camera controller 68 and causes the camera 50 to start an imaging operation. The image data capturing unit 601 captures image data (frame image) captured by the camera 50 after the start of the image capturing operation and stores it in the image work area of the RAM 63.

  The shelf label recognition unit 602 recognizes the shelf label of the product included in the frame image captured by the image data capture unit 601. A shelf label is an over-the-counter advertisement that contains information for identifying a product on a medium such as paper, a product name, and a price. It may be displayed. In this embodiment, it is assumed that at least one of a barcode including a product code for identifying a product, a product name, and a price is written on the shelf label.

  As one method for the shelf label recognition unit 602 to recognize a shelf label included in a frame image, a contour line is extracted from an image obtained by binarizing the frame image, and a striped line (barcode) There is a method of recognizing a frame including a line (product name, price) as a shelf label. In addition, there is a method in which information related to the shape of the shelf label used in the store is registered in advance, and an area similar to the shape of the registered shelf label is recognized as a shelf label from the frame image by the pattern matching technique. . The shelf label recognition unit 602 recognizes the shelf label by using one or a combination of the above-described methods.

  The barcode reading unit 603 reads a code symbol (bar code) written on the shelf label recognized by the shelf label recognition unit 602. Specifically, barcode detection by a pattern matching technique, detection of a two-dimensional code from a predetermined finder pattern, and the like are performed from the shelf label area recognized in the captured frame image. The barcode reading unit 603 reads the product code corresponding to the detected barcode by converting the detected barcode into code information (for example, JAN code) according to a predetermined rule. Thereby, the barcode reading unit 603 can read the product code corresponding to the barcode attached to the shelf label when the shelf label is imaged by the camera 50.

  The character recognition unit 604 reads characters (product name, price) written on the shelf label recognized by the shelf label recognition unit 602. Specifically, character recognition is performed by performing OCR processing on the area of the shelf label recognized in the captured frame image. In this character recognition, the price may be read for a character string in which “yen” is described, and the product name may be read for a character string in which “yen” is not described. Since at least one of the product name and the price is written on the shelf label, the character recognition unit 604 recognizes at least one character of the product name and the price.

  The collation unit 605 refers to the PLU file F, and registers information corresponding to the product code read by the barcode reading unit 603, such as the product name and price, and the character recognition unit 604. Check against recognized characters (product name, price). Specifically, information relating to the product registered in the PLU file F corresponding to the product code read by the barcode reading unit 603 is acquired. Next, it is determined whether or not there is a part that matches the character recognized by the character recognition unit 604 in the information related to the acquired product, and the result is a collation result. For example, when the product name written on the shelf label is recognized by the character recognizing unit 604, if the description on the shelf label matches the registration in the PLU file F, the product name in the acquired product information Are detected as matching parts. In addition, when the price described on the shelf label is recognized by the character recognition unit 604, if the description on the shelf label and the registration in the PLU file F match, the price matches in the acquired product information. It will be detected as a part to do. On the contrary, if the description of the shelf label and the registration of the PLU file F do not coincide with each other, the coincident portion in the information relating to the acquired product is not detected.

  The notification unit 606 performs notification according to the result of collation by the collation unit 605 by sound output by the buzzer 652 or screen display by the display 20. Specifically, when the description of the shelf label and the registration of the PLU file F do not match, an alarm sound is output, a warning screen is displayed, and the like.

  Next, the operation of the information processing apparatus 1 will be described in detail. FIG. 5 is a flowchart illustrating an example of the operation of the information processing apparatus 1.

  As shown in FIG. 5, when the process is started, the image data capturing unit 601 outputs an imaging on signal to the camera controller 68 and starts imaging by the camera 50 (S1). Next, the CPU 60 starts displaying an image captured by the camera 50 on the display 20 (S2).

  Next, the image data capturing unit 601 captures a frame image (captured image) captured by the camera 50 and stored in the RAM 63 (S3). Next, the shelf label recognition unit 602 recognizes the shelf label included in the frame image captured by the image data capture unit 601 (S4). The position of the shelf label recognized here is stacked in the work area of the RAM 63 and used in later processing. When a plurality of shelf labels are recognized, the positions of the respective shelf labels are stacked.

  Next, the barcode reading unit 603 reads the barcode described on the shelf label recognized in S4, and obtains a product code (S5). Next, the character recognition unit 604 recognizes the characters (product name, price) described on the shelf label recognized in S4 (S6). Next, the collation unit 605 acquires information on the product such as the product name and price registered in correspondence with the product code acquired in S5 from the PLU file F, and the characters (product name, product name, Price) is collated (S7).

  Next, as a result of the collation in S7, the CPU 60 determines whether the price and product name acquired from the PLU file F by reading the barcode are different from the price of the shelf label and the product name by character recognition (match / mismatch). (S8).

  If they match (S8: NO), the notification unit 606 displays a symbol image indicating that the price of the shelf label and the product name match the registered contents of the PLU file F on the display 20 in accordance with the position of the recognized shelf label. Display (S9), and proceed to S11. If they do not match (S8: YES), the notification unit 606 displays a symbol image indicating that the price of the shelf label and the product name are different from the registered contents of the PLU file F on the display 20 according to the position of the recognized shelf label. (S10), and the process proceeds to S11. In S10, a warning sound may be output by the buzzer 652.

  In S11, the CPU 60 determines whether or not reading for all the shelf labels recognized in S4 is completed. When reading of all the shelf labels is not completed (S11: NO), the CPU 60 returns the process to S5. By returning the processing to S5, the processing of (S5 to S10) is performed for the next stacked shelf label. If all the shelf labels have been read (S11: YES), the CPU 60 advances the process to S12.

  By repeating the processes of S5 to S11 described above, the display 20 displays whether or not the price and product name match the registered contents of the PLU file F for all the shelf labels included in the frame image. Will be. FIG. 6 is an external view of the information processing apparatus 1 showing a display example of the display 20.

  In FIG. 6, a recognition mark M1 indicates a recognized shelf label. The symbol image M2 indicates that the price of the shelf label matches the registered content of the PLU file F. The symbol image M3 indicates that the price of the shelf label is different from the registered content of the PLU file F. The prices displayed in the symbol image M2 and the symbol image M3 are prices registered in the PLU file F. The symbol image M2 and the symbol image M3 are displayed in different colors and shapes. Although FIG. 6 shows an example of price only, it goes without saying that the same applies to product names. Further, the display layout illustrated in FIG. 6 is an example, and the shapes and display positions of the symbol image M2 and the symbol image M3 are not particularly limited to the illustrated examples.

  As shown in FIG. 6, the display 20 corresponds to a position where the symbol image M2 or the symbol image M3 corresponds to the recognition mark M1, that is, the position of the recognized shelf label, in the captured image started to be displayed in S2. It is displayed superimposed on the selected position. Therefore, the clerk H can easily recognize which shelf label price matches the registered content of the PLU file F.

  In S <b> 12, the CPU 60 determines whether or not there is an end of business based on an operation instruction from the touch panel 30 or the operation key 40. When the business is continued (S12: NO), the CPU 60 returns the process to S3 and continues the process. When the business is to be ended (S12: YES), the CPU 60 ends the display of the image captured by the camera 50 on the display 20 (S13). Next, the image data capturing unit 601 outputs an imaging off signal to the camera controller 68 to end the imaging by the camera 50 (S14), and ends the process.

  As described above, the information processing apparatus 1 includes the image data capturing unit 601, the shelf label recognition unit 602, the barcode reading unit 603, the character recognition unit 604, the collation unit 605, and the notification unit 606. . The image data capturing unit 601 captures an image captured by the camera 50. The shelf label recognition unit 602 recognizes a shelf label included in the captured image. The barcode reading unit 603 reads product identification information for identifying a product from the recognized shelf label. The character recognizing unit 604 recognizes a character described on the recognized shelf label. The collation unit 605 refers to product registration information in which information related to a product is registered in advance for each product identification information, information registered corresponding to the read product identification information, a recognized character, Perform the verification. The alerting | reporting part 606 alert | reports according to the result of collation. Therefore, the clerk H can check the shelf label only by confirming the notification according to the result of the collation. For this reason, the information processing apparatus 1 can reduce the burden of the check work performed by the store clerk H.

[Second Embodiment]
In the second embodiment, a store clerk uses an information processing device installed in a cash register or a backyard to check the product name and price on the system and the product name and price printed on the shelf label. The store system to perform is illustrated.

  FIG. 7 is a system configuration diagram schematically showing the overall configuration of the store system 200. The store system 200 shown in FIG. 7 includes an information processing apparatus 201 installed in a cash register stand or a store backyard, a POS terminal 202 installed in the cash register stand, a live camera 203 that captures the inside of the store, and the store system 200 as a whole. A store computer SC as a server to be controlled is provided. In the store system 200, the information processing apparatus 201, the POS terminal 202, the live camera 203, and the store computer SC are connected to each other via a communication network such as a LAN (Local Area Network).

  The information processing apparatus 201 is a PC (Personal Computer) having substantially the same configuration as the information processing apparatus 1. The live camera 203 is installed, for example, suspended from a ceiling or the like for each store in the store. The live camera 203 includes an image sensor such as a CCD image sensor or a CMOS image sensor and a communication interface, and broadcasts the captured image to a communication network. Therefore, the information processing apparatus 201 can capture the image in the store without the store clerk H carrying it around, and can perform the processing illustrated in FIG. 5 from the captured image.

  FIG. 8 is a diagram illustrating a display screen example of the information processing apparatus 201 in the store system 200. As shown in FIG. 8, on the display screen G of the information processing apparatus 201, the symbol image M2 or the symbol image M3 corresponds to the recognition mark M1 in the image captured by the live camera 203, that is, the recognized shelf label. It is displayed superimposed on the position corresponding to the position. Therefore, the store clerk H can easily recognize which shelf label price matches the registered content of the PLU file F without looking around the store.

  A program executed by the information processing apparatus 1 according to the present embodiment is provided by being incorporated in advance in a ROM or the like. A program executed by the information processing apparatus 1 according to the present embodiment is an installable or executable file, and is a computer such as a CD-ROM, a flexible disk (FD), a CD-R, or a DVD (Digital Versatile Disk). The information may be provided by being recorded on a recording medium that can be read by the user.

  Furthermore, the program executed by the information processing apparatus 1 according to the present embodiment may be provided by being stored on a computer connected to a network such as the Internet and downloaded via the network. Further, the program executed by the information processing apparatus 1 of the present embodiment may be configured to be provided or distributed via a network such as the Internet.

  The program executed by the information processing apparatus 1 according to the present embodiment includes the above-described units (image data capturing unit 601, shelf label recognition unit 602, barcode reading unit 603, character recognition unit 604, verification unit 605, and notification unit 606. ), And the actual hardware includes a CPU (processor) that reads the program from the ROM and executes the program to load each unit onto the main storage device, and the image data capturing unit 601. A shelf label recognition unit 602, a barcode reading unit 603, a character recognition unit 604, a collation unit 605, and a notification unit 606 are generated on the main storage device.

1,201 Information processing apparatus 10 Case 20 Display 30 Touch panel 50 Camera 60 CPU
62 ROM
63 RAM
69 HDD
200 Store System 202 POS Terminal 203 Live Camera 601 Image Data Capture Unit 602 Shelf Label Recognition Unit 603 Barcode Reading Unit 604 Character Recognition Unit 605 Verification Unit 606 Notification Unit 652 Buzzer F PLU File G Display Screen M1 Recognition Marks M2, M3 Symbol Image PR Program SC Store computer

Japanese Patent No. 3560988

Claims (6)

Image capturing means for capturing an image captured by the image capturing means;
Shelf label recognition means for recognizing a shelf label included in the captured image;
Reading means for reading product identification information for identifying a product from the recognized shelf label;
Character recognition means for recognizing characters written on the recognized shelf label;
Refer to the product registration information in which information related to the product is registered in advance for each product identification information, and check the information registered corresponding to the read product identification information and the recognized character Matching means for performing
An informing means for informing according to the result of the collation;
An information processing apparatus comprising: The character recognition means recognizes at least one of a product name and a price described on the recognized shelf label,
The collation means collates the product name and price registered corresponding to the read product identification information with at least one of the recognized product name and price,
The notifying means notifying the matching or mismatching of the collation results;
The information processing apparatus according to claim 1. It further comprises display means for displaying the captured image,
The notification means performs a display corresponding to the result of the collation on the display means;
The information processing apparatus according to claim 1 or 2. The notification means superimposes and displays the result of the collation for the shelf label at a position corresponding to the shelf label recognized in the captured image;
The information processing apparatus according to claim 3. An imaging device for imaging the store;
Image capturing means for capturing an image captured by the imaging device;
Shelf label recognition means for recognizing a shelf label included in the captured image;
Reading means for reading product identification information for identifying a product from the recognized shelf label;
Character recognition means for recognizing characters written on the recognized shelf label;
Refer to the product registration information in which information related to the product is registered in advance for each product identification information, and check the information registered corresponding to the read product identification information and the recognized character Matching means for performing
An information processing apparatus comprising notification means for performing notification according to a result of the verification;
The store system characterized by having. The computer of the information processing device
Image capturing means for capturing an image captured by the image capturing means;
Shelf label recognition means for recognizing a shelf label included in the captured image;
Reading means for reading product identification information for identifying a product from the recognized shelf label;
Character recognition means for recognizing characters written on the recognized shelf label;
Refer to the product registration information in which information related to the product is registered in advance for each product identification information, and check the information registered corresponding to the read product identification information and the recognized character Matching means for performing
An informing means for informing according to the result of the collation;
A program characterized by making it function.

Images