JP2013054631A - Information processor and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To simplify operation of an operator in confirming sales information of a commodity.SOLUTION: An information processor has image display means, commodity identification means and sales display means. The image display means displays an image of a commodity sales floor taken by an imaging device on a display part. The commodity identification means identifies a commodity from the image included in a range specified on the image of the commodity sales floor displayed on the display part by the image display means. The sales display means displays sales information on sales of a commodity identified by the commodity identification means on the display part.

Description

  Embodiments described herein relate generally to an information processing apparatus and a program.

  Conventionally, in stores that sell products such as supermarkets, sales information related to sales of products corresponding to the entered product code can be confirmed by entering the product code on a store server that is installed at the store office. It is possible to do.

  However, conventionally, there is a problem that the operation of the operator when confirming the sales information of the product is complicated.

  The information processing apparatus according to the present embodiment includes image display means, product specifying means, and sales display means. The image display means displays on the display unit the image of the merchandise store taken by the imaging device. The product specifying unit specifies a product from an image included in a range specified on the image of the product sales floor displayed on the display unit by the image display unit. The sales display means displays sales information relating to sales of the product specified by the product specifying means on the display unit.

FIG. 1 is a system configuration diagram showing a schematic configuration of a POS system. FIG. 2 is a block diagram showing a hardware configuration of the control system of the information terminal. FIG. 3 is a block diagram showing a functional configuration of the information terminal. FIG. 4 is a schematic diagram illustrating a configuration example of a sales display screen displayed on the liquid crystal display. FIG. 5 is a schematic diagram illustrating a configuration example of a sales display screen displayed on the liquid crystal display. FIG. 6 is a schematic diagram illustrating a configuration example of a sales display screen displayed on the liquid crystal display. FIG. 7 is a flowchart showing the flow of processing operations in the information terminal. FIG. 8 is a schematic diagram showing a modification of the sales display screen shown in FIG.

  Hereinafter, embodiments will be described with reference to the accompanying drawings.

  Note that the information processing apparatus and the program according to the present embodiment schematically display the product sales floor image captured by the imaging device on the display unit, and are specified on the product sales floor image displayed on the display unit. It is an apparatus and a program capable of specifying a product from an image included in a range and displaying sales information related to sales of the specified product on a display unit.

  First, the configuration of a POS (Point Of Sales) system 1 to which an information processing apparatus and a program according to the present embodiment are applied will be described with reference to FIG. FIG. 1 is a system configuration diagram showing a schematic configuration of the POS system 1.

  As shown in FIG. 1, the POS system 1 includes a store server device 2 and a recording device 3 installed in a store office and a plurality of POS terminals 4 (shown in FIG. 1) installed in a store accounting location. In the example, there are two), a wireless hub 5 installed in the product sales floor, and a plurality of cameras (imaging devices) (three in the example of FIG. 1) 6.

  In the POS system 1, the store server device 2, the recording device 3, the POS terminal 4, the wireless hub 5, and the camera 6 are connected to each other via a communication network N such as a LAN (Local Area Network). . In the POS system 1, the wireless hub 5 and the portable information terminal 7 are connected to each other via a wireless communication network so as to be able to communicate with each other.

  Here, the store server device 2 is a server device as a host device of the recording device 3, the POS terminal 4, and the camera 6. Specifically, the store server device 2 performs registration processing of products purchased by customers with the POS terminal 4 and Payment processing is performed, and sales information related to sales for each product or department (for example, sales or number of sales for each product or department) is managed by communication with the POS terminal 4.

  The recording device 3 records a captured image captured by each camera 7.

  The POS terminal 4 is a terminal for performing registration and accounting processing of products purchased by customers with the store server device 2 based on the product code input from the scanner. The POS terminal 4 stores a sales file for storing and managing information on products purchased by customers in a predetermined storage unit.

  The wireless hub 5 is a device that mediates wireless communication between the store server device 2 and the information terminal 7.

  The camera 6 is an image pickup device for picking up products displayed on each product shelf, particularly each display shelf installed in the product store, and is composed of, for example, a video camera that captures a live image of the product store.

  The information terminal 7 is a portable information processing device used by a store clerk, and displays various information on a liquid crystal display (display unit) 71.

  Next, the hardware configuration of the control system of the information terminal 7 will be described with reference to FIG. FIG. 2 is a block diagram showing a hardware configuration of the control system of the information terminal 7. As shown in FIG. 2, the information terminal 7 includes a CPU (Central Processing Unit) 70a, a ROM (Read Only Memory) 70b, and a RAM (Random Access Memory) 70c, which are connected via a bus line BL. Connected. The CPU 70a functions as a process control unit 700 (see FIG. 3) that controls the processing operation of the information terminal 7 by developing and executing the program stored in the ROM 70b on the RAM 70c.

  Further, a liquid crystal display 71, a touch panel 72, a nonvolatile memory 73, a communication interface 74, and the like are connected to the CPU 70a via a bus line BL and an interface (not shown). The ROM 70b and the RAM 70c function as storage means, and store display data for displaying various information on the liquid crystal display 71.

  Next, the functional configuration of the information terminal 7 will be described with reference to FIG. FIG. 3 is a block diagram showing a functional configuration of the information terminal 7.

  As illustrated in FIG. 3, the CPU 70a functions as the processing control unit 700 by developing a program stored in the ROM 70b or the like on the RAM 70c and executing the program.

  The processing control unit 700 includes an operation input unit 700a, a display processing unit 700b, a product identification unit 700c, a sales information generation unit 700d, and a pop-up information generation unit 700e.

  Here, the operation input unit 700a receives an operation input from the touch panel 72. In the present embodiment, the operation input unit 700a receives operation inputs on the sales display screens S1, S2, and S3 shown in FIGS.

  The display processing unit 700b displays various types of information on the liquid crystal display 71. In this embodiment, the captured image of the camera 6 acquired via the communication interface 74, a sales information generation unit 700d and pop-up information generation described later. Sales display screens S1, S2, and S3 shown in FIGS. 4 to 6 are displayed on the liquid crystal display 71 based on various information input from the unit 700e.

  4 to 6 are schematic diagrams showing one configuration example of the sales display screen displayed on the liquid crystal display 71. FIG. 4 shows the sales display screen S1 showing a state before the sales information is displayed. FIG. 5 shows a sales display screen S2 displaying sales information related to sales of a certain product, and FIG. 6 shows a sales display screen S3 displaying sales information related to sales of a certain department and products belonging to the department. Show.

  As shown in FIG. 4, the sales display screen S1 is a live image captured by the camera selection button BT for selecting a place to be imaged (that is, the camera 6) and the camera 6 selected by the camera selection button BT. A live image display area E1 for displaying the image P. Note that when the camera selection button BT is touch-selected, for example, a pull-down menu (not shown) is displayed, and one of a plurality of cameras can be selected from the menu.

  On the other hand, the sales display screen S2 shown in FIG. 5 is provided with a camera selection button BT and a live image display area E1 as well as the sales display screen S1, and on the image P displayed in the live image display area E1. And a sales display area E2 for displaying sales information related to sales of the product specified from the image in the range A1 designated by the operator with a finger or a touch pen. In the live image display area E1, a pop-up B1 that displays the name of the product in association with the specified product is displayed. In the sales display area E2, for example, a table T1 including product names (for example, four watermelons), sales (for example, ¥ 4,000), and the number of sales (20) is displayed as sales information. .

  On the other hand, the sales display screen S3 shown in FIG. 6 includes a camera selection button BT, a live image display area E1, and a sales display area E2, similarly to the sales display screen S2. However, in the sales display screen S3, the sales information displayed in the sales display area E2 is different from the sales information displayed in the sales display area E2 of the sales display screen S2. That is, in the sales display area E2 of the sales display screen S3, sales related to a plurality of products specified from the image in the range A2 designated on the image P displayed in the live image display area E1 and a department to which the plurality of products belong. A table T2 showing information is displayed. In the live image display area E1 of the sales display screen S3, as in the sales display screen S2, pop-ups B1, B2, and B3 that display the names of the products in association with the plurality of specified products, respectively. , B4 are displayed.

  The pop-ups B1, B2, B3, and B4 each have a background color that is substantially the same as the associated product or a background color that makes it easy to imagine the product. In the example of FIGS. 5 and 6, when the product is “banana”, the background color of the popup displayed in association with the product is “yellow”, and when the product is “mandarin orange”, the product The background color of the popup displayed in association with is set to “orange”.

  Returning to FIG. 3, the product identification unit 700 c determines the product type (for example, “four” from the images in the ranges A1 and A2 designated by the operator (clerk) on the product sales floor image P displayed on the liquid crystal display 71. The product code uniquely assigned to the specified product is searched from a table (not shown) in which a product code unique to each product type is registered in association with each other.

  Specifically, the product specifying unit 700c reads the state of the product surface (such as the color of the surface, pattern (pattern), unevenness, etc.) as a feature amount from the image in the specified range, and is registered in advance. The product is recognized as a specific object by comparing with the feature value for each product. Note that the product identification unit 700c does not consider the contour or size of the product in order to shorten the processing time.

Note that recognizing a product included in an image as described above is called generic object recognition. Regarding such general object recognition, various recognition techniques are described in the following documents.
Keiji Yanai, “Current Status and Future of General Object Recognition”, IPSJ Journal, Vol. 48, no. SIG16 [Search August 10, 2010], Internet <URL: http://mm.cs.uec.ac.jp/IPSJ-TCVIM-Yanai.pdf>

Further, techniques for performing general object recognition by dividing an image into regions for each object are described in the following documents.
Jamie Shotton et al., “Semantic Texton Forests for Image Categorization and Segmentation”, [August 10, 2010 search], Internet <URL: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1. 1.145.3036 & rep = rep1 & type = pdf>

  When one product and its product code are searched for in the product specifying unit 700c, the sales information generating unit 700d is stored in association with the product code from the store server device 2 based on the product code. Receiving sales information (information indicating the sales amount, sales number, etc. of a certain product), and generating image data for displaying on the liquid crystal display 71 a table T1 (see FIG. 5) indicating the received sales information. To the display processing unit 700b.

  In addition, when the product specifying unit 700c searches for a plurality of products and a plurality of product codes respectively corresponding to the plurality of products, the sales information generation unit 700d stores the store server device based on the plurality of product codes. Sales information stored in association with each product code from 2 (information indicating the sales amount and the number of sales of each product), and a department code (one or more) that is a high-order code of the plurality of product codes Sales information (information indicating the sales amount of the department, the number of sales, etc.) stored in correspondence with the table T2 (see FIG. 6) indicating the received sales information is displayed on the liquid crystal display 71. Is generated and output to the display processing unit 700b.

  The pop-up information generation unit 700e generates image data for displaying the pop-ups B1, B2, B3, and B4 for displaying the names of the products specified by the product specifying unit 700c, and outputs the image data to the display processing unit 700b.

  Next, processing operations in the information terminal 7 will be described with reference to FIG. FIG. 7 is a flowchart showing a flow of processing operations in the information terminal 7, and particularly shows a processing procedure when displaying sales information.

  As shown in FIG. 7, in this process, first, in step S11, the display processing unit 700b of the processing control unit 700 displays the sales display screen S1 shown in FIG. 4 on the liquid crystal display 71 based on a predetermined operation by the operator. To do.

  Thereafter, in step S12, the process control unit 700 monitors whether or not a range designation operation is performed by the operator with a finger or a touch pen. Then, when a range designation operation is performed by the operator in this monitoring state (step S12: Yes), subsequently, in step S13, the product identification unit 700c identifies the product type from the image in the designated range. Then, a process for searching for a unique product code corresponding to the identified product type is performed.

  Thereafter, when one product is specified by the specifying process (step S14: No), subsequently, in step S15, the sales information generating unit 700d corresponds to the one product specified in the step S13. The sales information stored in association with the product code is received from the store server device 2 based on the code, and image data for displaying the table T1 (see FIG. 5) indicating the received sales information is generated. And output to the display processing unit 700b. Thereby, the display processing unit 700b displays the sales display screen S2 (see FIG. 5) including the table T1 indicating the sales information of the product designated by the operator on the liquid crystal display 71.

  Thereafter, it is determined whether or not a predetermined operation for ending the process of displaying the sales information shown in FIG. 7 has been performed by the operator (step S16). When the operation for ending is performed (step S16: Yes), The process control unit 700 ends the process. On the other hand, when the operation to end is not performed, that is, when the operation is not ended (step S16: No), the process control unit 700 returns the process to step S11 and performs the same process.

  On the other hand, when a plurality of products are specified by the specifying process in step S13 (step S14: Yes), in the subsequent step S17, the sales information generation unit 700d sets the plurality of products specified in step S13. Sales information stored in association with each product code from the store server device 2 based on a plurality of corresponding product codes, and one or more department codes that are higher-order codes of the plurality of product codes The sales information stored in the table is received, and image data for displaying the table T2 (see FIG. 6) indicating the received sales information is generated and output to the display processing unit 700b. As a result, the display processing unit 700b displays a sales display screen S3 including a table T2 indicating a plurality of products included in a range (for example, A2) designated by the operator and sales information of a department to which the plurality of products belong (FIG. 6). Reference) is displayed on the liquid crystal display 71. Thereafter, the process control unit 700 shifts the process to step S16 and performs the same process.

  Next, an example of specific processing of FIG. 7 will be described.

  First, for example, in a state where the sales display screen S1 shown in FIG. 4 is displayed on the liquid crystal display 71, the customer is displayed on the image P of the product sales floor displayed in the live image display area E1 of the sales display screen S1. A process when the range A1 including the image of the product “4-cut watermelon” is designated will be described.

  In this case, in step S11, the display processing unit 700b displays the sales display screen S1 shown in FIG. Then, in step S12, it is determined that the range A1 has been designated by the operator (step S12: Yes), and in step S13, the product specifying unit 700c specifies a product called “four cut watermelon” from the image of the range A1. , Search for a unique product code corresponding to the product.

  After that, it is determined No in the determination in step S14, and in the subsequent step S15, the sales information generation unit 700d obtains the product “4-cut watermelon” from the store server device 2 based on the product code searched in step S13. Image data for receiving sales information (for example, sales “¥ 4,000” and sales number “20”) and displaying a table T1 (see FIG. 5) indicating the received sales information on the liquid crystal display 71 is displayed. Generate and output to the display processing unit 700b. Thereby, the display processing unit 700b displays the sales display screen S2 as shown in FIG.

  Next, in a state where the sales display screen S1 shown in FIG. 4 is displayed on the liquid crystal display 71, the customer displays “4” on the image P of the merchandise store displayed in the live image display area E1 of the sales display screen S1. A process when a range A2 including images of a plurality of products such as “Tsumiri Watermelon”, “Tangerine”, and “Banana” is designated will be described.

  In this case, similarly to the above, in step S11, the display processing unit 700b displays the sales display screen S1 shown in FIG. In step S12, it is determined that the range A2 has been designated by the operator (step S12: Yes), and in step S13, the product specifying unit 700c reads “four cut watermelons”, “mandarin oranges” and “ Three products called “banana” are specified, and a unique product code corresponding to each product is searched.

  Thereafter, it is determined Yes in step S14, and in the subsequent step S17, the sales information generation unit 700d receives “four cut watermelon” from the store server device 2 based on the plurality of product codes searched in step S13. Sales information for each of the three products “Mikan” and “Banana” (for example, sales “¥ 4,000” and sales number “20” as sales information for the product “4-cut watermelon”, “Mikan” The sales information “¥ 6,000” and the sales number “40” as the sales information of the product “the sales information“ ¥ 100,000 ”and the sales number“ 80 ”as the sales information of the product“ banana ”, etc.) , Sales information of a department code which is a high-order code of the plurality of product codes (for example, sales “¥ 200,000” as sales information of a department “fruit and vegetables” and the number of sales “ 05 ", etc.) and receives the outputs table T2 shown sales information thus received (see FIG. 6) to generate the image data to be displayed on the display unit 700b in the liquid crystal display 71. Thereby, the display processing unit 700b displays a sales display screen S3 as shown in FIG.

  According to the embodiment described above, an operator can easily confirm desired sales information by selecting a product in a live image without manually inputting a product code. That is, the operation of the operator when confirming the sales information of the product can be facilitated.

  As mentioned above, although demonstrated based on exemplary embodiment, this invention is not limited by above-described embodiment.

  For example, in the above-described embodiment, when a plurality of products are specified in step S14 of FIG. 7, the process proceeds to step S17 to display department sales information. However, the present invention is not limited to this. In the process of step S14, when a product equal to or greater than a preset threshold value (number) (for example, 5) is specified, the process proceeds to the process of step S17 to display department sales information. When the above-mentioned product is not specified, it may constitute so that it may shift to processing of Step S14 and display the sales information on the specified product.

  Although not described in the above embodiment, if the product cannot be specified in the product specifying process in step S13 in FIG. 7, the corresponding camera 6 is set so that the range designated by the operator is as large as possible. The product may be configured to be specified again after zooming automatically. In addition, the range specified by the operator may be divided into a plurality of images, and the product specifying process may be performed on each of the divided images.

  In addition, when the product cannot be specified in the product specifying process in step S13 of FIG. 7, a plurality of product candidates indicating product names and product images are displayed on the sales display screen S1, and then the operator When a product candidate is selected, a product corresponding to the selected product candidate may be specified as a product designated by the operator.

  Further, when displaying the above candidates, products that are considered not to be imaged from the orientation of the camera that captures the live image displayed in the live image display area E1 or the unique ID (channel, etc.) of the camera are selected from the candidates. It is preferable to exclude candidates and narrow down candidates.

  Further, in the above-described embodiment, as shown in FIGS. 5 and 6, the form in which the sales amount and the number of sales of the product or department are displayed as the sales information is not limited to this. For example, FIG. As shown, a graph T3 indicating the number of sales for each time zone of a product or department, that is, time-series sales may be displayed as sales information.

  Further, in the above-described embodiment, the case where the information processing apparatus and the program according to the present embodiment are applied to the information terminal 7 has been described. However, the present invention is not limited to this, for example, other such as the store server apparatus 2 and the POS terminal 4 It is also possible to apply to this information processing apparatus.

  Further, the program executed by the information terminal 7 of the above-described embodiment can be provided by being incorporated in advance in a storage unit such as a ROM. The program can be provided by being stored in a computer-readable storage medium in an installable or executable format file. Furthermore, the program can be provided or distributed via a network such as the Internet.

  Moreover, although some embodiment of this invention was described, these embodiment is shown as an example and is not intending limiting the range of invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

1 POS system 2 Store server device (information processing device)
3 Recording device 4 POS terminal (information processing device)
5 Wireless hub 6 Camera 7 Information terminal (information processing device)
N Communication network (LAN)
70a CPU
700 processing control unit 700a operation input unit 700b display processing unit (image display means, sales display means, pop-up display means)
700c Product specifying unit (product specifying means)
700d Sales information generator 700e Pop-up information generator

Japanese Patent No. 3223929

Claims (6)

Image display means for displaying on the display unit an image of the merchandise store imaged by the imaging device;
Product specifying means for specifying a product from an image included in a range specified on an image of a product sales floor displayed on the display unit by the image display means;
Sales display means for displaying sales information on sales of the product specified by the product specifying means on the display unit;
An information processing apparatus.   When the product specifying unit specifies a plurality of products from the image included in the specified range, the sales display unit displays sales information regarding sales of a department to which the specified products belong to the display unit. The information processing apparatus according to claim 1, which displays the information processing apparatus.   When the product specifying unit specifies a plurality of products belonging to the same department from the image included in the specified range, the sales display unit displays sales information related to sales of one specified department. The information processing apparatus according to claim 2, displayed on the screen.   When the product specifying unit specifies a plurality of products belonging to different departments from the image included in the designated range, the sales display unit displays sales information regarding sales of each of the plurality of specified departments. The information processing apparatus according to claim 2, displayed on the screen. Pop-up display means for pop-up displaying the product specified by the product specifying means or the name of the department to which the product belongs in association with the image of the corresponding product,
The information processing apparatus according to any one of claims 1 to 4, further comprising: Computer
Image display means for displaying on the display unit an image of the merchandise store imaged by the imaging device;
Product specifying means for specifying a product from an image included in a range specified on an image of a product sales floor displayed on the display unit by the image display means;
Sales display means for displaying sales information on sales of the product specified by the product specifying means on the display unit;
Program to make it work.

Images