JP2020177705A - Information processing apparatus and program - Google Patents

Abstract

To provide an information processing apparatus and a program that enable an operator to easily register an article.SOLUTION: An information processing apparatus comprises: designation means, registration means, and display control means. The designation means designates an article. The registration means registers information on the article designated by the designation means. The display control means displays, on a display part, an operation part for indicating the registration means on the basis of the information registered by the registration means.SELECTED DRAWING: Figure 4

Description

Embodiments of the present invention relate to information processing devices and programs.

Conventionally, a technique related to general object recognition (object recognition) that recognizes the type of an article according to the degree of similarity between a feature amount of a target article extracted from captured image data and a feature amount for collation prepared in advance. There is.

By the way, the information processing device that recognizes the target article by the above-mentioned general object recognition selects the target article from the candidate articles in order to prevent an error in the general object recognition when the similarity is lower than the threshold value. Seeking.

The information processing device individually identifies the corresponding article for each target article. Therefore, even when a plurality of articles of the same type are registered, the operator has to repeat the same operation, and there is room for improvement in the operability of the information processing device.

An object to be solved by the present invention is to provide an information processing device and a program capable of easily registering an article.

The information processing apparatus of the embodiment includes a designation means, a registration means, and a display control means. The designation means designates an article. The registration means registers the information of the article designated by the designated means. The display control means causes the display unit to display an operation unit for instructing the registration means based on the information registered by the registration means.

FIG. 1 is a perspective view showing an example of a checkout system according to the first embodiment. FIG. 2 is a block diagram showing an example of the hardware configuration of the POS terminal and the product reader. FIG. 3 is an explanatory diagram showing an example of the data structure of the PLU file. FIG. 4 is a block diagram showing a characteristic functional configuration of the POS terminal. FIG. 5 is an explanatory diagram showing an example of the data structure of the image data table. FIG. 6 is an explanatory diagram showing an example of the data structure of the sales registration table. FIG. 7 is an explanatory diagram showing an example of the product registration screen. FIG. 8 is an explanatory diagram showing an example of the product registration screen. FIG. 9 is a flowchart showing an example of the product registration process executed by the POS terminal of the first embodiment. FIG. 10 is an explanatory diagram showing an example of the data structure of the sales registration table according to the second embodiment. FIG. 11 is an explanatory diagram showing an example of the product registration screen. FIG. 12 is a flowchart showing an example of the product registration process executed by the POS terminal of the second embodiment. FIG. 13 is a perspective view showing an example of the configuration of the self-POS. FIG. 14 is a block diagram showing an example of a hardware configuration of self-POS.

Hereinafter, the information processing apparatus and the program according to the embodiment will be described in detail with reference to the drawings. The embodiment described below is an embodiment of the information processing device and the program, and does not limit the configuration, specifications, and the like. This embodiment is an application example to a checkout system introduced in a store such as a supermarket.

(First Embodiment)
FIG. 1 is a perspective view showing an example of the checkout system 1 according to the first embodiment. The checkout system 1 includes a product reading device 101 that reads information about products, and a POS (Point Of Sales) terminal 11 that registers and setstles products related to one transaction. Hereinafter, an example in which the POS terminal 11 is applied as the information processing device according to the first embodiment will be described. Further, as an example of the article subject to general object recognition, the case where the target article is a product will be described, but the target article may be an article other than the product.

The POS terminal 11 is placed on the upper surface of the drawer 21 on the checkout table 41. The opening operation of the drawer 21 is controlled by the POS terminal 11. The POS terminal 11 is provided with a keyboard 22 operated by an operator (clerk) on the upper surface. The POS terminal 11 includes a first display unit 23 that displays various information toward the operator behind the keyboard 22 when viewed from the operator who operates the keyboard 22. The first display unit 23 includes a touch panel 26 that receives various operations. The POS terminal 11 includes a second display unit 24 rotatably erected further behind the first display unit 23. The second display unit 24 shown in FIG. 1 is directed toward the front side in FIG. 1, but is rotated so as to be directed toward the back side in FIG. 1 to display various information to the customer.

The horizontally long table-shaped counter stand 151 is arranged so as to form an L shape with the checkout stand 41 on which the POS terminal 11 is placed. A shopping cart 153 for storing products is placed on the counter stand 151. The shopping basket 153 is not limited to the so-called basket shape, and may be a tray or the like. Alternatively, the shopping cart 153 may be in the shape of a box, a bag, or the like. The shopping cart 153 includes a first shopping cart 154 brought in by the customer and a second shopping cart 155 placed at a position sandwiching the product reading device 101 from the first shopping cart 154.

A product reading device 101 connected to the POS terminal 11 so as to be able to transmit and receive data is installed on the counter stand 151. The product reading device 101 is built in the thin rectangular housing 102. The housing 102 is provided with a reading window 103 on the front surface. A display operation unit 104 is attached to the upper portion of the housing 102.

The display operation unit 104 includes a first display unit 106 on which the touch panel 105 is laminated on the surface. The product reading device 101 includes a keyboard 107 on the right side of the first display unit 106. The product reading device 101 includes a card reading groove 108 of the card reader 111 (see FIG. 2) on the right side of the keyboard 107. The product reading device 101 includes a second display unit 109 that provides information to the customer on the back left back side of the display operation unit 104 when viewed from the operator.

The product reading device 101 includes a product reading unit 110 (see FIG. 2) inside the housing 102. The product reading unit 110 includes an imaging unit 164 (see FIG. 2) on the back side of the reading window 103.

In the first shopping cart 154, products related to one transaction are stored. The operator who operates the product reading device 101 moves the products in the first shopping cart 154 to the second shopping cart 155. In this moving process, the operator holds the product over the reading window 103 of the product reading device 101. At this time, the imaging unit 164 (see FIG. 2) arranged in the reading window 103 images the product.

FIG. 2 is a block diagram showing an example of the hardware configuration of the POS terminal 11 and the product reading device 101. The POS terminal 11 includes a CPU (Central Processing Unit) 61, a ROM (Read Only Memory) 62, a RAM (Random Access Memory) 63, a storage unit 64, a communication interface 65, a keyboard 22, and a first display unit. A 23, a touch panel 26, a second display unit 24, a connection interface 66, a drawer 21, and a printer 67 are provided. CPU 61, ROM 62, RAM 63, storage interface 64, communication interface 65, keyboard 22, first display unit 23, touch panel 26, second display unit 24, connection interface 66, drawer 21 and so on. The printer 67 is connected to the printer 67 via a bus.

The CPU 61 comprehensively controls the operation of the POS terminal 11. The ROM 62 stores various programs and data. The RAM 63 temporarily stores various programs and rewritably stores various data. Further, the RAM 63 stores an image data table 72 described later and a sales registration table 73 described later. The image data table 72 and the sales registration table 73 are not limited to the RAM 63, and may be stored in another storage medium such as the storage unit 64, or may be stored in another device such as a store server. ..

The keyboard 22 includes various keys for operating the POS terminal 11. For example, the keyboard 22 includes a closing key or the like for ending the product registration process for registering the product related to one transaction.

The storage unit 64 is a non-volatile storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive). The storage unit 64 stores the control program 70 and the PLU file 71. The PLU file 71 is not limited to the storage unit 64, and may be stored in another storage medium, or may be stored in another device such as a store server.

The control program 70 is a program for exerting the functions provided in the operating system and the POS terminal 11. The control program 70 includes a program that exerts a characteristic function according to the present embodiment.

The PLU file 71 is a product file that stores information related to product sales registration for each of the various products displayed and sold in the store. Here, FIG. 3 is an explanatory diagram showing an example of the data structure of the PLU file 71. The PLU file 71 stores the product code, the product information, the illustration image, and the matching feature amount for each product in association with each other. The product code is identification information that can identify the product. The product information is information such as the product classification, product name, and unit price to which the product belongs. The illustration image is an image showing the product. The collation feature amount is a parameterized appearance feature such as a standard shape of a product, surface color, pattern, and unevenness. The collation feature amount is collation data used for determining the degree of similarity, which will be described later.

Returning to FIG. 2, the communication interface 65 is connected to the CPU 61 of the POS terminal 11. The communication interface 65 is an interface for communicating with an external device such as a store computer via a network.

The connection interface 66 is an interface for connecting to the product reading device 101. The printer 67 prints the transaction contents in one transaction on the receipt paper.

The product reading unit 110 of the product reading device 101 includes a CPU 161, a ROM 162, a RAM 163, an imaging unit 164, an audio output unit 165, and a connection interface 175. The CPU 161, the ROM 162, the RAM 163, the image pickup unit 164, the audio output unit 165, and the connection interface 175 are connected via a bus.

The CPU 161 comprehensively controls the operation of the product reading device 101. The ROM 162 stores various programs and data. The RAM 163 temporarily stores various programs and rewritably stores various data.

The imaging unit 164 includes an image sensor (not shown) such as a color CCD image sensor or a color CMOS image sensor, and a light source (not shown) such as an LED. Then, the image pickup unit 164 takes an image from the inside of the reading window 103 under the control of the CPU 161. For example, the image pickup unit 164 captures image data at a frame rate such as 30 fps (Frame Per Second). The image capturing unit 164 stores the sequentially captured image data in the RAM 163.

The voice output unit 165 is a voice circuit, a speaker, or the like for generating a preset warning sound or the like. The voice output unit 165 gives a warning sound or voice notification under the control of the CPU 161.

The connection interface 175 is an interface for connecting to the POS terminal 11 and the display operation unit 104.

The display operation unit 104 of the product reading device 101 includes a connection interface 176, a keyboard 107, a first display unit 106, a touch panel 105, a second display unit 109, and a card reader 111. The connection interface 176, the keyboard 107, the first display unit 106, the touch panel 105, the second display unit 109, and the card reader 111 are connected via a bus.

The connection interface 176 is an interface for connecting to the POS terminal 11 and the product reading unit 110.

The card reader 111 is a device that reads information stored in a storage medium of a card used for payment such as a credit card. The card reader 111 reads the information stored in the storage medium of the card inserted in the card reading groove 108.

The operation of the display operation unit 104 is controlled by the CPU 161 of the product reading unit 110 or the CPU 61 of the POS terminal 11.

Next, the characteristic functions of the POS terminal 11 will be described. Here, FIG. 4 is a block diagram showing a characteristic functional configuration of the POS terminal 11. The CPU 61 expands the control program 70 of the storage unit 64 into the RAM 63 and operates according to the control program 70 to generate each functional unit shown in FIG. 4 in the RAM 63. Specifically, the CPU 61 includes an image acquisition unit 611, a product detection unit 612, a feature amount extraction unit 613, a similarity calculation unit 614, a memory control unit 615, and a product identification unit 616 as functional units. A product registration unit 617, a display control unit 618, and an operation control unit 619 are provided.

The image capturing unit 611 outputs an imaging request to the product reading device 101, and causes the imaging unit 164 to start the imaging operation. The image pickup unit 164 stores the sequentially captured image data in the RAM 163. The product reading device 101 sequentially outputs the image data stored in the RAM 163 to the image capturing unit 611 of the POS terminal 11. Then, the image acquisition unit 611 sequentially receives the input of the image data captured by the image pickup unit 164.

The product detection unit 612, which is a detection means, detects the product included in the image data received by the image capture unit 611. For example, the product detection unit 612 detects all or a part of the product included in the image data by using a pattern matching technique or the like. Specifically, the product detection unit 612 extracts contour lines and the like from the binarized image of the image data. The product detection unit 612 detects the product based on the difference between the contour line extracted from the image data prepared in advance and the contour line extracted from the image data to be detected. Here, the image data prepared in advance is image data captured by the imaging unit 164 on a background that does not include a product or the like. Therefore, the product detection unit 612 extracts the contour line that is not included in the contour line extracted from the image data prepared in advance from the image data to be detected, provided that the image data to be detected includes the product. It can be detected from the contour line. Then, the product detection unit 612 detects the product based on the contour line that is not included in the contour line extracted from the image data prepared in advance.

The feature amount extraction unit 613 extracts the surface state such as the color tone of the product and the unevenness of the surface detected by the product detection unit 612 from the image data captured by the image pickup unit 164 as the feature amount.

The similarity calculation unit 614 compares the feature amount for collation of the product registered in the PLU file 71 with the feature amount of the product extracted by the feature amount extraction unit 613, so that the feature amount of the product registered in the PLU file 71 can be compared. Calculate the similarity with each. Here, the similarity is how similar all or part of the products are when the collation feature amount of the products stored in the PLU file 71 is 100% = "similarity: 1.0". It indicates whether or not. In addition, the similarity calculation unit 614 may calculate the similarity by changing the weighting depending on, for example, the hue and the unevenness of the surface.

Recognizing an object contained in an image in this way is called generic object recognition. Regarding such general object recognition, various recognition techniques are described in the following documents.
Keiji Yanai, "Current Status and Future of General Object Recognition", IPSJ Journal, Vol. 48, No. SIG16 [Searched April 29, 2016], Internet <URL: http://mm.cs.uec.ac.jp/IPSJ-TCVIM-Yanai.pdf>

Further, a technique for performing general object recognition by dividing an image into regions for each object is described in the following documents.
Jamie Shotton et al., “Semantic Texton Forests for Image Categorization and Segmentation”, [Searched April 29, 2016], Internet <URL: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1. 1.145.3036 & rep = rep1 & type = pdf>

It should be noted that the method of calculating the degree of similarity between the collation feature amount of various products registered in the PLU file 71 and the feature amount of the product extracted by the feature amount extraction unit 613 is not particularly limited. For example, the similarity calculation unit 614 may calculate the similarity between the matching feature amount of the product registered in the PLU file 71 and the feature amount of the product extracted by the feature amount extraction unit 613 as an absolute evaluation. , May be calculated as a relative evaluation.

The storage control unit 615 stores the image data or the like captured by the image pickup unit 164 in the image data table 72. Here, FIG. 5 is an explanatory diagram showing an example of the data structure of the image data table 72. The image data table 72 stores one or more of the identification number, the image data, the feature amount, the similarity, and the product code in association with each other. The identification number is identification information that can identify the information stored in the image data table 72. The image data is the image data received by the image capturing unit 611. That is, the image data is image data used for recognizing a general object of a product included in the image data. The feature amount is a feature amount extracted by the feature amount extraction unit 613 for the product included in the associated image data. The similarity is the highest degree of similarity between the feature amount of the product included in the associated image data and the collation feature amount of various products registered in the PLU file 71. The product code is the product code of the product with the highest degree of similarity. Then, the storage control unit 615 deletes all the image data and the like stored in the image data table 72 on condition that the product has been registered as sales. The storage control unit 615 may store the image data, the feature amount, the similarity, and the product code not only in the same data table but also in different data tables.

Returning to FIG. 4, the product identification unit 616, which is the extraction means, extracts the product information of the product corresponding to the product included in the image data from the PLU file 71 based on the similarity stored in the image data table 72. That is, the product identification unit 616 extracts the product information of the product imaged by the image pickup unit 164. In the present embodiment, the product identification unit 616 extracts the confirmed product, the product candidate, and the non-identifiable stage according to the similarity stored in the image data table 72. The classification according to the degree of similarity is an example, and may be classified by another method.

As the first condition, the product identification unit 616 determines whether or not a predetermined number or more of similarities of the first threshold value or more are registered for the same product in the image data table 72. The product identification unit 616 extracts the product corresponding to the product detected by the product detection unit 612 from the PLU file 71 as a confirmed product on condition that the first condition is satisfied. Then, the product identification unit 616 extracts the product information of the confirmed product from the PLU file 71. The confirmed product indicates that the product can be automatically registered based on the similarity stored in the image data table 72 without the confirmation operation by the operator. The automatic registration is a process of registering a confirmed product as a product to be sold without requesting a confirmation operation by an operator.

Further, the product identification unit 616 determines whether or not a predetermined number or more of similarities of the second threshold value or more are registered in the image data table 72 under the second condition that the first condition is not satisfied. To judge. The second threshold value is lower than the first threshold value. The product identification unit 616, which is an extraction means, extracts one or more products corresponding to the products detected by the product detection unit 612 from the PLU file 71 as product candidates on condition that the second condition is satisfied. Then, the product identification unit 616 extracts the product information of the product candidate from the PLU file 71. The product candidate indicates that the product candidate corresponds to the product detected by the product detection unit 612 based on the similarity stored in the image data table 72. That is, the product candidate indicates that the product candidate is a product candidate imaged by the imaging unit 164. Then, the product candidate is registered as a product to be sold by an operator's confirmation operation such as selecting a corresponding product from one or a plurality of product candidates.

Further, the product identification unit 616 determines that the product cannot be identified on the condition that the second condition is not satisfied. The product identification unit 616 does not extract the product information from the PLU file 71 on condition that it is determined that the product cannot be identified.

The product registration unit 617 registers the product designated by the designated means such as the product identification unit 616 as a product to be sold. That is, the product registration unit 617 registers the product information of the product to be sold as sales. When the product identification unit 616 determines that the product is a confirmed product, the product registration unit 617 registers the corresponding product as a product to be sold. Alternatively, when the product identification unit 616 extracts the product as a product candidate, the product registration unit 617 registers the product selected from the product candidates as a product to be sold. The product registration unit 617 registers the product specified by the product code indicated by the code symbol as a product to be sold even when the imaging unit 164 or the like reads the code symbol or the like.

More specifically, the product registration unit 617, which is a registration means, registers various information about the product designated as the product to be sold by the product identification unit 616 or the like in the sales registration table 73. Here, FIG. 6 is an explanatory diagram showing an example of the data structure of the sales registration table 73. The sales registration table 73 stores the identification number, the product code, and the product information in association with each other. The identification number is identification information that can identify the information stored in the sales registration table 73. The product code is the product code of the product for which sales are registered. The product information is the product information of the product of the associated product code. The product registration unit 617 sends the information stored in the sales registration table 73 to the store server or the like when the product registration process for registering the product related to one transaction is completed. Further, the product registration unit 617 deletes the information stored in the sales registration table 73.

The display control unit 618 controls all or part of the first display unit 23, the second display unit 24, the first display unit 106, and the second display unit 109 to display various screens. For example, the display control unit 618 causes the first display unit 106 to display the product registration screen 1000. Then, the display control unit 618 provides an operation unit for registering the product specified by the product code in the product registration unit 617 on the product registration screen 1000 based on various information registered in the sales registration table 73 by the product registration unit 617. Display it.

Here, FIG. 7 is an explanatory diagram showing an example of the product registration screen 1000. The product registration screen 1000 is a screen for registering the product code of the product to be sold. The product registration screen 1000 includes a captured image display area 1100, a registration screen button 1200, a product candidate display area 1300, and a registration history display area 1400. The captured image display area 1100 is an area for displaying the image data for which the image acquisition unit 611 has received the input. That is, the captured image display area 1100 is an area for displaying the image data captured by the imaging unit 164. The registration screen button 1200 is a button for displaying a screen for registering sales of a product to be sold by pressing a button of a product selected from the product list. The product candidate display area 1300 is an area for displaying the product candidate button 1301. The product candidate button 1301 is an operation unit that instructs the product identification unit 616 to register the product of the product candidate designated as the product to be sold. The registration history display area 1400 is an area for displaying the registration history button 1401 (see FIG. 8). The registration history button 1401 is an operation unit for instructing sales registration of a product specified by a product code stored in the sales registration table 73 as a product to be sold. Here, the product registration screen 1000 shown in FIG. 7 does not display the registration history button 1401 because it is assumed that the product code is not stored in the sales registration table 73.

Here, FIG. 8 is an explanatory diagram showing an example of the product registration screen 1000 in a state where the product code is stored in the sales registration table 73. The display control unit 618, which is a display control means, causes the product registration unit 617 to register the product specified by the product code on the product registration screen 1000 based on various information registered in the sales registration table 73 by the product registration unit 617. The registration history button 1401 which is the operation unit is displayed. Since the product registration screen 1000 shown in FIG. 8 assumes a state in which the product code is stored in the sales registration table 73, the registration history button 1401 is displayed. Here, the display control unit 618 displays the registration history button 1401 for registering the sales of the product specified by the product code as the product to be sold, provided that the product code is stored in the sales registration table 73. That is, the display control unit 618 displays the registration history button 1401 regardless of whether or not the product identification unit 616 extracts product candidates. Therefore, on the product registration screen 1000 shown in FIG. 8, the registration history button 1401 is pressed even when the product is not imaged in the captured image display area 1100 and the product candidate button 1301 is not displayed in the product candidate display area 1300. it's shown.

The display control unit 618 displays the registration history button 1401 for registering sales of all or part of the products stored in the sales registration table 73 as the products to be sold in the registration history display area 1400. For example, when the registration history display area 1400 is too small to display all the registration history buttons 1401, the display control unit 618 makes the registration history display area 1400 scrollable so that all the registration history buttons 1401 can be displayed. Is displayed. Alternatively, the display control unit 618 displays the remaining registration history buttons 1401 on a separate screen. Further, the display control unit 618 may display the products selected from the products stored in the sales registration table 73 in the registration history display area 1400. For example, the display control unit 618 displays the registration history button 1401 of the recently registered product. Alternatively, the display control unit 618 displays the registration history button 1401 selected by an algorithm such as leaving the recently used registration history button 1401 and deleting the unused registration history button 1401 in the registration history display area 1400.

The operation control unit 619 controls all or a part of the keyboard 22, the touch panel 26, the touch panel 105, and the keyboard 107, and accepts various operations.

Next, the product registration process executed by the POS terminal 11 will be described. Here, FIG. 9 is a flowchart showing an example of the product registration process executed by the POS terminal 11 of the first embodiment.

First, the display control unit 618 of the POS terminal 11 displays the product registration screen 1000 (step S1).

Next, the image acquisition unit 611 of the POS terminal 11 requests the product reading unit 110 to start imaging by the image capturing unit 164 (step S2).

Next, the display control unit 618 of the POS terminal 11 determines whether or not the product code is stored in the sales registration table 73 (step S3). The POS terminal 11 proceeds to step S5 on condition that the product code is not stored in the sales registration table 73 (step S3; No).

On the other hand, on condition that the product code is stored in the sales registration table 73 (step S3; Yes), the display control unit 618 of the POS terminal 11 displays the registration history button 1401 in the registration history display area 1400 (step). S4). That is, the display control unit 618 displays the registration history button 1401 for registering the sales of the product specified by the product code registered in the sales registration table 73.

Next, the image acquisition unit 611 of the POS terminal 11 receives the input of the image data captured by the image pickup unit 164 from the product reading unit 110 (step S5). Next, the feature amount extraction unit 613 of the POS terminal 11 extracts the feature amount of the product detected by the product detection unit 612 (step S6). Next, the similarity calculation unit 614 of the POS terminal 11 compares the feature amount extracted by the feature amount extraction unit 613 with the matching feature amount of each product in the PLU file 71, and calculates the similarity for each product. (Step S7).

Next, the storage control unit 615 of the POS terminal 11 stores the similarity, the feature amount, the image data, and the product code of the product having the highest similarity in the image data table 72 (step S8).

Next, the product identification unit 616 of the POS terminal 11 determines whether or not a confirmed product can be extracted based on the degree of similarity (step S9). On condition that the confirmed product can be extracted (step S9; Yes), the product registration unit 617 of the POS terminal 11 registers the extracted confirmed product as a product to be sold in the sales registration table 73 (step S10). ..

On the other hand, on condition that the confirmed product cannot be extracted (step S9; No), the product identification unit 616 of the POS terminal 11 determines whether or not the product candidate can be extracted based on the similarity. (Step S11). The POS terminal 11 proceeds to step S3 on condition that the product candidate cannot be extracted (step S11; No).

On the other hand, on condition that the product candidates can be extracted (step S11; Yes), the display control unit 618 of the POS terminal 11 displays the product candidate button 1301 for registering the extracted product candidates as sales in the product candidate display area 1300. (Step S12).

Next, the product registration unit 617 of the POS terminal 11 determines whether or not the product candidate button 1301 has been pressed (step S13). The POS terminal 11 proceeds to step S3 on condition that the product candidate button 1301 is not pressed (step S13; No).

On the other hand, on condition that the product candidate button 1301 is pressed (step S13; Yes), the product registration unit 617 of the POS terminal 11 sets the product associated with the product candidate button 1301 as a product to be sold and sets the sales registration table 73. Register the sales in (step S14).

Next, the operation control unit 619 of the POS terminal 11 determines whether or not the pressing of the tightening key is detected (step S15). The POS terminal 11 proceeds to step S3 on condition that the pressing of the closing key is not detected (step S15; No).

On the other hand, on condition that the pressing of the closing key is detected (step S15; Yes), the POS terminal 11 ends the product registration process.

As described above, the POS terminal 11 ends the product registration process.

As described above, according to the POS terminal 11 according to the first embodiment, the product registration unit 617 registers the product code of the product selected from the product candidates extracted by the product identification unit 616 and the like in the sales registration table 73. To do. Then, the display control unit 618 displays the product registration screen 1000 for registering the sales of the product based on the product code registered in the sales registration table 73. More specifically, the display control unit 618 displays the registration history button 1401 for registering the product specified by the product code registered in the sales registration table 73 on the product registration screen 1000. Therefore, when the operator of the POS terminal 11 re-registers the sales of the registered product, the operator can register the product as sales without holding the product over the imaging unit 164. Therefore, the POS terminal 11 can easily register the article.

(Second Embodiment)
Next, the second embodiment will be described. The differences from the first embodiment will be mainly explained, and the components having the same functions as those of the first embodiment will be given the same names and codes as those of the first embodiment. The explanation is omitted.

The POS terminal 11 according to the second embodiment has the product candidate display area 1300 at the same position as the previous general object recognition, provided that the same type of product has already been registered for sales by general object recognition. The candidate button 1301 is displayed.

The product registration unit 617 stores various information of the registered products in the sales registration table 74 on the condition that the product identification unit 616 has registered the sales of the products selected from the product candidates identified by the general object recognition. Here, FIG. 10 is an explanatory diagram showing an example of the data structure of the sales registration table 74 according to the second embodiment. The sales registration table 74 stores the identification number, the product code, the product information, and the location information in association with each other. The identification number, the product code, and the product information of the sales registration table 74 according to the second embodiment are the same information as the identification number, the product code, and the product information of the sales registration table 73 according to the first embodiment. Is. The position information is information indicating the position where the product candidate button 1301 is displayed in the product candidate display area 1300.

The display control unit 618 displays the product registration screen 1000. The display control unit 618 is an operation unit that causes the product registration unit 617 to register the product specified by the product code in the product candidate display area 1300 based on various information registered in the sales registration table 74 by the product registration unit 617. The product candidate button 1301 is displayed. More specifically, the display control unit 618 indicates the position information associated with the corresponding product code, provided that the product code of the product candidate extracted by the product identification unit 616 is included in the sales registration table 74. The product candidate button 1301 is displayed at the position. On the other hand, the display control unit 618 positions the product candidate button 1301 at a position based on a predetermined rule or the like on the condition that the product code of the product candidate extracted by the product identification unit 616 is not included in the sales registration table 74. Is displayed. Here, the predetermined rule is, for example, the order of similarity of the extracted product candidates.

Here, FIG. 11 is an explanatory diagram showing an example of the product registration screen 1000. The product registration screen 1000 shown in FIG. 11 displays the product candidate buttons 1301 for the product E, the product F, the product G, and the product D in the product candidate display area 1300. Further, the product code of the product D is registered in the sales registration table 74. The degree of similarity is in the order of product D, product E, product F, and product G. Therefore, if the product D is not registered in the sales registration table 74, the display control unit 618 places the product D at the position of the product E shown in FIG. 11, the product E at the position of the product F, and the product at the position of the product G. The product G is displayed at the positions of F and product D. However, the display control unit 618 displays the product D at the position shown in FIG. 11 on condition that the product code of the product D is registered in the sales registration table 74. The product registration screen 1000 according to the second embodiment does not have to have the registration history display area 1400.

Next, the product registration process executed by the POS terminal 11 according to the second embodiment will be described. Here, FIG. 12 is a flowchart showing an example of the product registration process executed by the POS terminal 11 of the second embodiment.

First, the display control unit 618 of the POS terminal 11 displays the product registration screen 1000 (step S21).

Next, the image acquisition unit 611 of the POS terminal 11 requests the product reading unit 110 to start imaging by the image capturing unit 164 (step S22).

The processing from step S23 to step S29 is the same as the processing from step S5 to step S10 according to the first embodiment.

Next, the display control unit 618 of the POS terminal 11 determines whether or not the product code of the extracted product candidate is included in the sales registration table 74 (step S30). On condition that the product code of the extracted product candidate is included in the sales registration table 74 (step S30; Yes), the display control unit 618 of the POS terminal 11 displays the product candidate button 1301 at the position indicated by the position information. (Step S31).

On the other hand, on condition that the product code of the extracted product candidate is not included in the sales registration table 74 (step S30; No), the display control unit 618 of the POS terminal 11 sets the product candidate button 1301 as a predetermined rule. Is displayed at a position based on (step S32).

Next, the operation control unit 619 of the POS terminal 11 determines whether or not the product candidate button 1301 has been pressed (step S33). The POS terminal 11 proceeds to step S23 on condition that the product candidate button 1301 is not pressed (step S33; No).

On the other hand, on condition that the product candidate button 1301 is pressed (step S33; Yes), the product registration unit 617 of the POS terminal 11 sets the product associated with the product candidate button 1301 as a product to be sold in the sales registration table 74. Register the sales in (step S34).

Next, the operation control unit 619 of the POS terminal 11 determines whether or not the pressing of the tightening key is detected (step S35). On condition that the pressing of the closing key is not detected (step S35; No), the POS terminal 11 proceeds to step S23.

On the other hand, on condition that the pressing of the closing key is detected (step S35; Yes), the POS terminal 11 ends the product registration process.

As described above, the POS terminal 11 ends the product registration process.

As described above, according to the POS terminal 11 according to the second embodiment, the product registration unit 617 has registered the product code selected from the product candidates extracted by the product identification unit 616 as sales. And the product information and the location information are registered in the sales registration table 74. Then, the display control unit 618, when displaying the product candidate button 1301 for registering the product candidate extracted by the product identification unit 616, on condition that the product code of the product candidate is included in the sales registration table 74. The product candidate button 1301 is displayed at the position indicated by the position information. That is, the display control unit 618 displays the product candidate button 1301 at the position previously displayed. Therefore, when the same product is registered for sales, the POS terminal 11 displays the product candidate button 1301 at the same position, so that it is possible to reduce the mistake of pressing the product candidate button 1301. Therefore, the POS terminal 11 can easily register the article.

Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other embodiments, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are also included in the scope of the invention described in the claims and the equivalent scope thereof.

Further, in the above embodiment, characteristic functions (image acquisition unit 611, product detection unit 612, feature amount extraction unit 613, similarity calculation unit 614, memory control unit 615, product identification unit 616, product registration unit 617, The POS terminal 11 was applied as an information processing device having a display control unit 618 and an operation control unit 619). However, the present invention is not limited to these, and these characteristic functions (image capture unit 611, product detection unit 612, feature amount extraction unit 613, similarity calculation unit 614, memory control unit 615, product identification unit 616, product registration unit 617). , The product reading device 101 may be applied as an information processing device having the display control unit 618 and the operation control unit 619). Further, these characteristic functions may be distributed to the POS terminal 11 and the product reading device 101.

Further, in the above embodiment, the POS terminal 11 describes the case where the product identified by the general object recognition (object recognition) is registered as sales as an example. However, the POS terminal 11 may identify the product by a method other than general object recognition (object recognition). For example, the POS terminal 11 may register sales of a product designated by a code symbol such as a bar code, or may register sales of a product designated by a product code input by a number key or the like. For example, when the POS terminal 11 registers a product with a product code specified by a barcode or the like as sales, the POS terminal 11 stores the specified product code in the sales registration table 73. Then, the POS terminal 11 displays the registration history button 1401 for registering the sales of the product having the product code stored in the sales registration table 73. As a result, the POS terminal 11 can register sales of a plurality of products without having to read the barcode of the same product a plurality of times.

Further, in the above embodiment, the collation data has been described as a feature amount, but the collation data may be an captured product image (reference image).

Further, in the above embodiment, the display control unit 618 displays the product candidate button 1301 in the product candidate display area 1300. Further, the display control unit 618 explains that the registration history button 1401 is displayed in the registration history display area 1400. However, the display control unit 618 does not have to distinguish between the product candidate display area 1300 and the registration history display area 1400 on the product registration screen 1000. That is, the display control unit 618 may display the product candidate button 1301 and the registration history button 1401 in the same display area.

Further, in the above embodiment, the product registration unit 617 stores the position information or the like indicating the position of the product candidate button 1301 in the sales registration table 74 on condition that the product candidate button 1301 is pressed. Then, the display control unit 618 explains that the product is displayed at the same position as the position information on condition that the product of the product candidate extracted by the product identification unit 616 is included in the sales registration table 74. However, the product registration unit 617 may store the position information or the like indicating the position of the registration button in the sales registration table 74 even when the registration button for registering the product other than the product candidate button 1301 is pressed. .. For example, the product registration unit 617 may store the position information or the like indicating the position of the registration history button 1401 in the sales registration table 74 on condition that the registration history button 1401 is pressed. In this case, the display control unit 618 displays the registration history button 1401 at the position of the position information stored in the sales registration table 74.

Further, in the above embodiment, it is applied to the checkout system 1 including the POS terminal 11 and the product reading device 101. However, the present invention is not limited to this, and may be applied to a single device having the functions of the POS terminal 11 and the product reading device 101. Examples of the one-unit configuration device having the functions of the POS terminal 11 and the product reading device 101 include a self-checkout device (hereinafter, simply referred to as self-POS) installed and used in a store such as a supermarket.

Here, FIG. 13 is a perspective view showing an example of the configuration of the self-POS200. FIG. 14 is a block diagram showing an example of the hardware configuration of the self-POS200. In the following, similar configurations shown in FIGS. 1 and 2 will be designated by the same reference numerals, and duplicate description thereof will be omitted. As shown in FIGS. 13 and 14, the main body 202 of the self-POS200 is a product that reads a product image in order to recognize (detect) the first display unit 106 on which the touch panel 105 is arranged on the surface, the type of product, and the like. A reading unit 110 is provided.

The first display unit 106 is, for example, a liquid crystal display. The first display unit 106 displays the product registration screen 1000. Then, the first display unit 106 displays the registration history button 1401 for registering the sales of all or a part of the products stored in the sales registration table 73 as the products to be sold in the registration history display area 1400. Alternatively, the first display unit 106 is the position indicated by the position information associated with the corresponding product code, provided that the product code of the product candidate extracted by the product identification unit 616 is included in the sales registration table 73. Display the product candidate button 1301. Alternatively, the first display unit 106 displays a guidance screen for notifying the customer of the operation method of the self-POS200, various input screens, and a registration screen for displaying the product information read by the product reading unit 110. Alternatively, the first display unit 106 displays the total amount of goods, the deposit amount, the change amount, and the like, and displays a settlement screen or the like for selecting a payment method.

The product reading unit 110 reads the product image by the image pickup unit 164 by the customer holding the code symbol attached to the product over the reading window 103 of the product reading unit 110.

In addition, the self-POS200 is provided with a product mounting stand 203 on the right side of the main body 202 for placing unpaid products in the basket. The self-POS200 is provided with a product mounting stand 204 on the left side of the main body 202 for placing the settled products. The product mounting stand 204 includes a bag-hanging hook 205 for hanging a bag for storing the settled products, and a temporary stand 206 for temporarily placing the settled products before putting them in the bag. The product mounting tables 203 and 204 are provided with measuring instruments 207 and 208, respectively, and have a function of confirming that the weights of the products are the same before and after the settlement.

In addition, the main body 202 of the self-POS200 includes a change device 201 for depositing banknotes for settlement and receiving fishing banknotes.

When the self-POS200 having such a configuration is applied to the checkout system 1, the self-POS200 functions as an information processing device.

The program executed by each device of the above-described embodiment or modification shall be provided by being incorporated in a storage medium (ROM or storage unit) provided in each device in advance, but the program is not limited to this. For example, a file in an installable format or an executable format should be recorded and provided on a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, or a DVD (Digital Versatile Disk). It may be configured as. Further, the storage medium is not limited to a medium independent of a computer or an embedded system, but also includes a storage medium in which a program transmitted by a LAN, the Internet, or the like is downloaded and stored or temporarily stored.

Further, the program executed by each device of the above-described embodiment or modification may be stored on a computer connected to a network such as the Internet and provided by downloading via the network, or the Internet. It may be configured to be provided or distributed via a network such as.

1 Checkout system 11 POS terminal 101 Product reader 72 Image data table 73, 74 Sales registration table 611 Image capture unit 612 Product detection unit 613 Feature extraction unit 614 Similarity calculation unit 615 Storage control unit 616 Product identification unit 617 Product Registration unit 618 Display control unit 619 Operation control unit 1000 Product registration screen 1300 Product candidate display area 1301 Product candidate button 1400 Registration history display area 1401 Registration history button

Japanese Unexamined Patent Publication No. 2013-89083

The information processing apparatus of the embodiment includes an extraction means, a designation means, a registration means, a storage control means, and a display control means. The extraction means extracts one or more candidates for articles. The designation means designates the article from one or more of the candidates extracted by the extraction means . The registration means registers the information of the article designated by the designated means. The storage control means stores the information of the article registered by the registration means in a storage unit. The display control means is a first display area for selecting an article to be registered in the registration means from the candidates extracted by the extraction means, and the registration means from the article specified by the information stored in the storage unit. A screen having a second display area for selecting an article to be registered in is displayed on the display unit.

Claims (6)

Designation means to specify goods and
A registration means for registering information on the article designated by the designated means, and
A display control means for displaying an operation unit for instructing the registration means on the display unit based on the information registered by the registration means.
Information processing device equipped with. The display control means causes the registration screen for displaying the operation unit for registering the article designated by the designated means to display the operation unit for registering the article of the same type as the article registered by the registration means. ,
The information processing device according to claim 1. The registration means includes position information indicating the position where the operation unit is displayed, and the information of the article specified by the operation unit, on condition that the article instructed by the operation unit is registered. To associate and register,
The display control means causes the operation unit to be displayed at a position indicated by the position information.
The information processing device according to claim 1 or 2. The display control means causes one or a plurality of the operation units for registering the article designated by the designated means to be displayed at a position indicated by the position information.
The information processing device according to claim 3. A detection means for detecting the article contained in the image data captured by the image pickup unit, and
It corresponds to the article detected by the detection means based on the similarity between the feature amount of the article extracted from the image data of the article detected by the detection means and the feature amount of various articles for collation. Further provided with an extraction means for extracting the article,
The designation means designates the article extracted by the extraction means.
The information processing device according to any one of claims 1 to 3. Computer,
Designation means to specify goods and
A registration means for registering information on the article designated by the designated means, and
A display control means for displaying an operation unit to be registered in the registration means based on the information registered by the registration means.
A program to make it work.