JP2023171433A - Information processing system, control method, and program - Google Patents

Abstract

To provide technology to accurately determine the validity of articles recognized by a device that registers articles to be settled.SOLUTION: An information processing device having a function for registering an article to be settled includes: first acquisition means for acquiring article information relating to the article to be settled; second acquisition means for acquiring image information including the article; and determination means for determining, based on the output of the first and second acquisition means, whether or not there is any irregularity with respect to the article registration. The image information includes at least one of an image of the article when the article is placed on the registered article placement site or an image of the article placed on the registered article placement site.SELECTED DRAWING: Figure 1

Description

The present invention relates to a technique for determining whether a product recognized by a device is a correct product.

In a device that registers products to be paid for, when the product to be paid for is recognized using a code that identifies the product, such as a so-called barcode, a weight sensor is installed at the location where the recognized product is temporarily placed. There is a technology for determining the validity of products recognized by the device. "Valid" here means that the correct product is recognized as the product the customer is purchasing in the device that registers the product to be paid. In this technique, it is determined whether the weight measured by the weight sensor and the weight of the product linked to the product identification code match based on the weight measured by the weight sensor.

In addition, an example of a technique for appropriately determining the validity of products recognized by a device that registers products to be paid for is described in Patent Document 1, for example. Patent Document 1 discloses that in response to reading a barcode that identifies a product, an image of an object (product) to which the barcode is attached and a basic shape of the product associated with the barcode are acquired, A technique has been disclosed that determines the validity of a recognized product based on a correlation value calculated using the shape of an object recognized from an image and a basic figure linked to a read barcode.

Patent Document 2 and Patent Document 3 disclose techniques for identifying products through image-based object recognition. Specifically, in Patent Document 2 and Patent Document 3, a product image is linked with a product ID, and the product image and the object included in the captured image are identified by their surface color, pattern, and surface irregularities. A technology has been disclosed that performs matching based on feature quantities such as surface conditions and shapes, and identifies product candidates corresponding to objects in images based on the degree of similarity. Further, Patent Document 3 discloses a technology that uses a thermo camera that uses far infrared rays to detect an area corresponding to a person's hand from an image, and performs the above-mentioned feature amount matching by excluding the area. ing.

Japanese Patent Application Publication No. 2014-132501 Japanese Patent Application Publication No. 2013-182323 JP2014-052806A

As described above, the higher the accuracy of the technique for determining the validity of the product recognized by the device for registering the product to be paid, the better.

An object of the present invention is to provide a technique for accurately determining the validity of products recognized by a device that registers products to be paid for.

According to the invention,
a first acquisition means provided in a container for storing and transporting products and acquiring product information regarding the products to be settled;
a second acquisition means provided in the container and configured to acquire image information of a product that a customer puts in the container;
Output means that outputs a warning when the product that the customer has placed in the container is determined to be inappropriate, using the information acquired by the first and second acquisition means;
Equipped with
The first acquisition means is different from the second acquisition means,
An information processing system is provided.

According to the invention,
A computer installed in a container for storing and transporting products,
a first acquisition step of acquiring product information regarding the product to be paid;
a second acquisition step of acquiring image information of the product that the customer puts in the container;
an output step of outputting a warning when the product placed in the container by the customer is determined to be inappropriate using the information acquired in the first and second acquisition steps;
Run
The first acquisition step is provided with a different control method from the second acquisition step.

According to the invention,
A computer installed in a container for storing and transporting products,
a first acquisition means for acquiring product information regarding the product to be settled;
a second acquisition means for acquiring image information of a product that a customer puts in the container;
Output means that outputs a warning when the product that the customer has placed in the container is determined to be inappropriate, using the information acquired by the first and second acquisition means;
function as
The first acquisition means is provided with a different program from that of the second acquisition means.

According to the present invention, it is possible to accurately judge the validity of a product recognized by a device that registers a product to be paid for.

1 is a diagram conceptually showing a processing configuration of an information processing device in a first embodiment. FIG. It is a figure which shows an example of the information which a size/shape information storage part stores. 1 is a diagram conceptually showing a hardware configuration of an information processing device. 3 is a flowchart showing the flow of processing of the information processing apparatus according to the first embodiment. FIG. 7 is a diagram illustrating an example of how an information processing device is attached in a second embodiment. FIG. 7 is a diagram conceptually showing a processing configuration of an information processing device in a third embodiment. 12 is a flowchart showing the flow of processing of an information processing apparatus according to a third embodiment. FIG. 7 is a diagram conceptually showing a processing configuration of an information processing device in a fourth embodiment. It is a figure which shows an example of the information which a weight information storage part stores. It is a flowchart which shows the flow of processing of the information processing device of a 4th embodiment. FIG. 12 is a diagram conceptually showing a processing configuration of an information processing device in a fifth embodiment. It is a flowchart which shows the flow of processing of the information processing device of a 5th embodiment. It is a figure showing conceptually the system configuration of the settlement system in a 6th embodiment. It is a sequence diagram which shows the flow of processing of the payment system of 6th Embodiment.

Embodiments of the present invention will be described below with reference to the drawings. Note that in all the drawings, similar components are denoted by the same reference numerals, and descriptions thereof will be omitted as appropriate.

[First embodiment]
[Processing configuration]
FIG. 1 is a diagram conceptually showing a processing configuration of an information processing apparatus 10 in the first embodiment. The information processing device 10 is a product registration device such as a so-called POS (Point of Sales) terminal, which has at least a function of registering products to be paid for. The information processing device 10 may further have a function of settling the price of registered products. The information processing device 10 of this embodiment includes an image recognition unit 110, a product recognition unit 120 (code acquisition unit 122, imaging unit 124, and thermography information acquisition unit 126), a size/shape information acquisition unit 130, a size/shape information acquisition unit 130, and a size/shape information acquisition unit 130. It includes an information storage section 132 and a determination section 140.

<Registration of products to be paid>
In this embodiment, the products to be paid for are registered as follows.

The item to be paid for is moved as shown by the dotted line in FIG. That is, a product that has not yet been registered as a product to be paid (hereinafter, such a product will be referred to as an "unregistered product") is removed from the place where unregistered products are temporarily placed, and is removed from the product recognition unit 120. is moved to the position of Here, as will be described in detail later, unregistered products are registered as products to be settled. Then, the products registered as payment targets (hereinafter, such products will be referred to as "registered products") are moved to a place where the registered products are temporarily placed.

For example, unregistered products are registered as products to be settled as follows. First, the code acquisition unit 122 of the product recognition unit 120 acquires the product identification code of the unregistered product held over the product recognition unit 120 as the product identification code of the product to be paid. Here, the product identification code is data formed from letters, numbers, symbols, etc. that are predetermined to identify each product. A PLU (Price Look Up) code (for example, a JAN (Japan Article Number) code, etc.) or a Non-PLU code may be used as the product identification code.

The product identification code is obtained by detecting and analyzing the product information symbol from the product to be paid. The product information symbol is a symbol that specifies information about the product. Here, the symbol is a barcode, a two-dimensional code (QR code (registered trademark), etc.), a character string symbol, or the like. Note that the character string here also includes a numerical string. The product information symbol is a bar code or the like in which information that specifies product information (ID of product information, etc.) is encoded, or a character string symbol that represents information that specifies product information. In this case, for example, by using a code reader equipped with a light source and a light-receiving element such as a photodiode, or a code reader equipped with a lens and an image sensor such as a CCD (Charge Coupled Device) image sensor, the product information symbol can be converted into a product identification code. is obtained. In addition, in the case of a code reader equipped with a lens and an image sensor, features of the product (for example, color, shape, packaging, or a specific mark or character string that identifies the product) are extracted from the image containing the product to be paid. The product identification code can also be obtained by performing matching processing with reference to a storage unit (not shown) that stores the product identification code and the feature amount of the product in association with each other.

Then, the product recognition unit 120 uses the product identification code acquired by the code acquisition unit 122 to link the product identification code and product information (for example, the product name, unit price of the product, discount, presence or absence of a discount, etc.). The product information associated with the obtained product identification code is read by referring to a storage unit (not shown) that is attached and stored. The product recognition unit 120 then adds the read product information as information used in the payment process executed by the information processing device 10. As a result, the pre-registration product is recognized on the information processing device 10 and registered as a product to be paid for.

<Determining the validity of the product recognized by the device>
Furthermore, in the present embodiment, the validity of the product registered as a payment target as described above is determined by each processing unit described below. Note that "valid" in this embodiment means that the information processing device 10 recognizes the correct product as the product that the customer will purchase.

The imaging unit 124 is a so-called 3D camera or the like, and is configured to be able to generate images in association with depth information indicating the distance in the depth direction of products (unregistered products) included in the imaging range. The imaging unit 124 associates and generates an image and depth information using a known method such as a method using parallax in a stereo camera. The imaging unit 124 of this embodiment includes the vicinity of the product recognition unit 120 in its imaging range, and its orientation and position are adjusted so that it can image an unregistered product that has been moved to the product recognition unit 120. Note that the imaging unit 124 may be provided integrally with the information processing device 10, such as a camera used to recognize a product as an object, or may be provided integrally with the information processing device 10, such as an external camera, etc. It may be installed around the processing device 10.

Further, the imaging unit 124 preferably captures an image in accordance with the timing at which the code acquisition unit 122 reads the product identification code. For example, when the code acquisition unit 122 detects the presence of a product information symbol of a product, it sends a notification to that effect to the imaging unit 124. Upon receiving this notification, the imaging unit 124 can know the timing at which the code acquisition unit 122 reads the code. In this way, in the case of a product having the above-mentioned product information symbol, if the imaging unit 124 captures an image at the timing when the code acquisition unit 122 reads the product identification code, the posture of the product included in the image will be stabilized. , the change in posture for each acquired image becomes smaller. As a result, it is expected that the number of templates used in image matching will be reduced, thereby reducing the processing load, and that recognition accuracy will be improved through image matching.

The thermography information acquisition unit 126 is a so-called thermography camera that uses infrared rays, etc., and captures infrared radiation energy emitted by an object included in the imaging range, and generates thermography information indicating the temperature distribution of the object. The thermography information acquisition unit 126 is configured to acquire thermography information in accordance with the imaging timing of the imaging unit 124. Further, the position and orientation of the thermography information acquisition unit 126 are adjusted so that the thermography information acquisition unit 126 can acquire thermography information of an imaging range that is the same as the imaging range of the imaging unit 124 or includes the imaging range of 124. Then, the thermography information acquisition unit 126 matches the positional relationship between the image generated by the imaging unit 124 and the thermography information by, for example, coordinate transformation processing in the image space. In this way, the thermography information acquisition unit 126 acquires thermography information associated with the image generated by the imaging unit 124.

The image recognition unit 110 performs the following processing using the image generated by the imaging unit 124 and the thermography information acquired by the thermography information acquisition unit 126.

The image recognition unit 110 performs known image recognition processing on the image generated by the imaging unit 124 to detect, for example, edges of a product in the image. Then, the image recognition unit 110, for example, matches the shape indicated by the detected edge with template data of a two-dimensional shape or three-dimensional shape set in advance, and recognizes the shape with the highest degree of similarity as the shape of the product. do. Furthermore, the image recognition unit 110 uses the depth information associated with the image generated by the imaging unit 124 and the size of the area occupied by the product in the image to determine, for example, the actual size of the product in the image. (For example, height, width, depth width, area of each surface, etc.) are calculated. Since the actual distance from the imaging device to the product can be determined based on the depth information, the image recognition unit 110 converts the size of the area occupied by the product in the image into the actual size of the product based on the distance. be able to.

Furthermore, the image recognition unit 110 uses the thermography information acquired by the thermography information acquisition unit 126 to exclude a region that is estimated to be a person from the shape of the product recognized from the image generated by the imaging unit 124. For example, the image recognition unit 110 is preset with information indicating the range of a person's body temperature (for example, 35 to 40°C), and the image recognition unit 110 stores an area indicating the set temperature in the thermography information. Extract from. The image recognition unit 110 then selects the extracted area from the target area of edge detection processing based on the positional relationship between the thermography information acquired by the thermography information acquisition unit 126 and the image generated by the imaging unit 124. The corresponding region is excluded, or the region corresponding to the region extracted from the recognized shape based on the detected edge is excluded. As a result, the image recognition unit 110 can more accurately recognize the area corresponding to the product from the image generated by the imaging unit 124, and as a result, the actual size of the product calculated from the area can be calculated more accurately.

The size/shape information acquisition unit 130 uses the product identification code acquired by the code acquisition unit 122 to refer to the size/shape information storage unit 132 and determines the size/shape associated with the product identification code. Get information. The size/shape information is information indicating the size of each product (for example, the height, width, and depth of the product, and the area of each side calculated from these) and the external shape of each product. The size/shape information storage unit 132 stores a product identification code and size/shape information in association with each other, as shown in FIG. 2, for example. FIG. 2 is a diagram showing an example of information stored in the size/shape information storage section 132. The information stored in the size/shape information storage unit 132 is not limited to the example in FIG. 2 .

The size/shape information acquisition unit 130 searches the size/shape information storage unit 132 to detect a product identification code that is the same as the product identification code acquired by the code acquisition unit 122, and assigns the detected product identification code to the product identification code. Obtain the linked size/shape information. Note that although the example in FIG. 1 shows an example in which the size/shape information storage unit 132 is provided in the information processing device 10, the size/shape information storage unit 132 communicates with the information processing device 10. It may also be provided in other devices where possible. In this case, the size/shape information acquisition unit 130 acquires size/shape information linked to the product identification code acquired by the code acquisition unit 122 by communicating with the other device.

The determination unit 140 determines whether the size or shape recognized from the image generated by the imaging unit 124 and the size/shape information acquired using the product identification code satisfy a predetermined criterion. do. Here, the predetermined standard is a standard regarding the size or shape of the product, and makes it possible to confirm that the product recognized from the image and the product identified from the product identification code are the same product. It is a standard. For example, the determination unit 140 calculates the degree of similarity between the two according to a predetermined similarity calculation algorithm regarding size and shape, and sets the degree of similarity to a predetermined reference value (a value that is set in advance as a value at which products are considered to be the same). (threshold value) or more.

As a result of the determination, if the degree of matching between the two is equal to or higher than a predetermined reference value, there is a possibility that the product moved to the position of the product recognition unit 120 and the product recognized by the information processing device 10 are the same product. is high, and the determination unit 140 can determine that the recognition result by the information processing device 10 is appropriate. On the other hand, if the degree of coincidence between the two is less than a predetermined reference value, there is a high possibility that the product moved to the position of the product recognition unit 120 and the product recognized by the information processing device 10 are different, and it is determined that The unit 140 can determine that the recognition result by the information processing device 10 is inappropriate. In this case, it is considered that some kind of defect has occurred, such as a code reading error or an incorrect code being pasted. Therefore, when the degree of matching between the two is less than a predetermined reference value, the determination unit 140 displays a message urging reloading the product, a predetermined warning message, etc. on the display unit of the information processing device 10, or outputs an audio message. It may be configured to output.

In addition, in this embodiment, the size/shape information storage unit 132 may store a plurality of pieces of size/shape information in association with each product identification code. Specifically, the size/shape information storage unit 132 stores, for example, size/shape information when viewed from the front, size/shape information when viewed from above, and size/shape information when viewed from diagonally above. Size/shape information, such as size/shape information, indicating the posture of the product viewed from multiple directions may be stored in association with each product identification code. In this case, the determination unit 140 calculates the degree of similarity between each piece of size/shape information linked to each product identification code and the size or shape recognized by the image recognition unit 110, and It is determined whether the degree of matching is greater than or equal to a predetermined reference value. By doing so, it is possible to prevent the accuracy of the determination by the determining unit 140 from decreasing due to the posture of the product included in the image generated by the imaging unit 124.

[Hardware configuration]
FIG. 3 is a diagram conceptually showing the hardware configuration of the information processing device 10. As shown in FIG. As shown in FIG. 3, the information processing device 10 includes a CPU 101, a memory 102, a storage 103, an input/output interface (input/output I/F) 104, a communication module 105, and the like.
The CPU 101, memory 102, storage 103, input/output interface 104, and communication module 105 are connected by a data transmission path for mutually transmitting and receiving data.

The memory 102 is, for example, a RAM (Random Access Memory) or a ROM (Read Only Memory). The storage 103 is, for example, a storage device such as a hard disk, an SSD (Solid State Drive), or a memory card. The storage 103 includes each of the image recognition unit 110, product recognition unit 120, code acquisition unit 122, imaging unit 124, thermography information acquisition unit 126, size/shape information acquisition unit 130, and determination unit 140 of the information processing device 10. It stores program modules for realizing the functions of the processing section. The CPU 101 implements the functions of each processing unit by executing each of these program modules. When executing each of the above modules, the CPU 101 may read these modules onto the memory 102 and then execute them, or may execute the modules without reading them onto the memory 102. The storage 103 also functions as a size/shape information storage unit 132.

The input/output interface 104 is connected to a display device 1041, an input device 1042, an imaging device 1043, a reading device 1044, and the like. The display device 1041 displays a screen corresponding to drawing data processed by the CPU 101, a GPU (Graphics Processing Unit) (not shown), etc., such as an LCD (Liquid Crystal Display) or a CRT (Cathode Ray Tube) display. It is a device. One or more display devices 1041 may be connected to input/output interface 104. The input device 1042 is a device that receives input from an operator, and is implemented as, for example, a keyboard, a mouse, a touch sensor, or the like. The display device 1041 and the input device 1042 may be integrated and realized as a touch panel. The imaging device 1043 is a so-called 3D camera or the like, and includes a monocular imaging module or a twin-lens imaging module (not shown). In this case, the imaging device 1043 can realize the functions of the imaging unit 124 described above. Further, the imaging device 1043 is a so-called thermography camera or the like, and includes an imaging module (not shown) that detects infrared energy emitted from an object and generates an image showing the temperature distribution of the object based on the amount of infrared energy. In this case, the imaging device 1043 can realize the functions of the thermography information acquisition unit 126 described above. The reading device 1044 is a device such as a code reader that includes a light source and a light receiving element such as a photodiode, or a code reader that includes a lens and an imaging device such as a CCD (Charge Coupled Device) image sensor. The reading device 1044 can realize the functions of the code acquisition unit 122 described above.

The communication module 105 is used to transmit and receive data with external devices and the like. Note that there are various methods for connecting the information processing device 10 and an external device via the communication module 105. For example, this connection is a bus connection via a bus line (for example, a USB (Universal Serial Bus) line), a network connection via a network line, or the like. Note that the network line may be a wireless line or a wired line.

Note that the hardware configuration of the information processing device 10 is not limited to the configuration shown in FIG. 3. For example, the input/output interface 104 may be connected to various devices (such as a money insertion device and a change dispensing device) necessary for product payment processing.

[Operation example]
An example of the operation of the information processing device 10 of this embodiment will be described using FIG. 4. FIG. 4 is a flowchart showing the process flow of the information processing device 10 of the first embodiment.

When the product is moved to the position of the product recognition unit 120, the code acquisition unit 122 obtains the product identification code of the product (S101). Further, in response to acquisition of the product identification code, the imaging unit 124 generates an image of the product by associating it with depth information indicating the distance in the depth direction (S102), and the thermography information acquisition unit 126 generates the image of the product by associating it with depth information indicating the distance in the depth direction (S102). Thermography information is acquired in association with the image obtained (S103).

The image recognition unit 110 then recognizes the size or shape of the product using the generated image and the depth information and thermography information associated with the image (S104). For example, the image recognition unit 110 uses the actual distance to the product obtained from the depth information to convert the size of the area occupied by the product in the image into the actual size of the product. For example, the image recognition unit 110 extracts a region of the product from the image using known image recognition processing such as edge processing, and obtains the shape of the region as the shape of the product.

Further, the size/shape information acquisition unit 130 refers to the size/shape information storage unit 132 using the product identification code acquired by the code acquisition unit 122, and uses the product identification code acquired by the code acquisition unit 122 to refer to the size/shape information storage unit 132 and determine the size/shape information associated with the product identification code. Shape information is read from the size/shape information storage unit 132 (S105).

The determination unit 140 compares the size or shape of the product recognized by the image recognition unit 110 with the size/shape information linked to the product identification code, and determines whether a predetermined standard is met. (S106). For example, the determination unit 140 may calculate the degree of similarity between the two according to a predetermined similarity calculation algorithm, and determine whether or not the two satisfy a predetermined criterion based on whether the degree of similarity is greater than or equal to a predetermined threshold. can. If the determination unit 140 determines that the predetermined criteria are met (S106: YES), the determination unit 140 determines that the product has been correctly recognized by the information processing device 10, and does not execute the process of S107, which will be described later. On the other hand, if the determination unit 140 determines that the predetermined criteria are not met (S106: NO), the determination unit 140 outputs a predetermined output (for example, a message prompting the user to reload the product, a warning message, a warning sound, etc.) such as a display or a speaker (not shown). (S107).

[Operations and effects of the first embodiment]
As described above, according to the present embodiment, for a product read by the product recognition unit 120, the actual size or shape of the product calculated from the image and the product identification code of the product are determined. The degree of similarity with size/shape information indicating the size or shape of is calculated. Then, it is determined whether or not both products satisfy a predetermined criterion, depending on whether the degree of similarity between the two products is equal to or greater than a predetermined threshold value (a threshold value preset as a value for considering the products to be the same product). If the sizes or shapes of the two substantially match and meet a predetermined standard, the product moved to the position of the product recognition unit 120 and the product recognized by the information processing device 10 are the same. There is a high possibility that this is the case, and the determination unit 140 determines that the recognition result by the information processing device 10 is appropriate. On the other hand, if the sizes or shapes of the two do not match and do not meet the predetermined criteria, the product moved to the position of the product recognition unit 120 and the product recognized by the information processing device 10 are different. This is likely to be the case, and the determination unit 140 determines that the recognition result by the information processing device 10 is inappropriate. In this way, by comparing the actual size or shape of the product recognized from the image with the theoretical size or shape of the product linked to the product identification code, the information processing device 10 recognizes the product. It is possible to accurately judge the validity of the product.

[Second embodiment]
Another method of operation is to attach a device that reads the items to be paid for to the basket that the customer carries the items they are purchasing, register the items as items to be paid for when they are placed in the basket, and only make payment at the cashier terminal. . In such a case, by providing the basket with a device having the same configuration as the information processing device 10 of the first embodiment, the same effects as the first embodiment can be obtained.

[Processing configuration/hardware configuration]
The information processing device 10 of this embodiment has an image recognition section 110, a product recognition section 120 (code acquisition section 122, imaging section 124, thermography information acquisition section 126), size/shape, and a size/shape similar to the configuration shown in FIG. It includes an information acquisition section 130, a size/shape information storage section 132, and a determination section 140. Further, the information processing device 10 of this embodiment also has the same hardware configuration as that of the first embodiment, and implements each processing unit in the same manner as the first embodiment. However, each hardware component of the information processing device 10 shown in FIG. 4 is made smaller than in the first embodiment in order to be attached to the basket 20.

FIG. 5 is a diagram showing an example of how the information processing device 10 is mounted in the second embodiment. The information processing device 10 is attached to a basket 20, as shown in FIG. As shown in FIG. 5, the unregistered product is registered in the information processing device 10 as a product to be paid for, and then placed in the basket 20.

[Operation example]
The information processing device 10 of this embodiment operates in the same way as the information processing device 10 of the first embodiment, as shown in FIG.

[Operations and effects of the second embodiment]
As described above, the same operations and effects as in the first embodiment can be obtained also in this embodiment.

[Third embodiment]
This embodiment is similar to the first embodiment and the second embodiment, except that the information processing device 10 further includes a number acquisition unit 150.

[Processing configuration]
FIG. 6 is a diagram conceptually showing the processing configuration of the information processing device 10 in the third embodiment. As shown in FIG. 6, the information processing device 10 of this embodiment further includes a number acquisition unit 150.

The number acquisition unit 150 acquires the number of products included in the imaging range of the image generated by the imaging unit 124. Specifically, the number acquisition unit 150 acquires the number of product shapes recognized by the image recognition unit 110 as the number of products. Alternatively, the number acquisition unit 150 acquires the number of objects detected from the image as the number of products by pattern matching of the characteristic amounts of the products stored in advance.

The determination unit 140 of this embodiment determines whether the number of product identification codes acquired by the code acquisition unit 122 and the number of products acquired by the number acquisition unit 150 match. Note that "match" here means complete match. In this embodiment, when a plurality of product information symbols exist in an image, the code acquisition unit 122 detects each product information symbol by performing image recognition processing, for example, and extracts a plurality of product identification codes at approximately the same timing. get. The code acquisition unit 122 of this embodiment further notifies the determination unit 140 of the number of product identification codes acquired here.

[Hardware configuration]
The information processing apparatus 10 of this embodiment also has a hardware configuration as shown in FIG. 4, similarly to the first embodiment and the second embodiment. The storage 103 further stores a program module that implements the function of the number acquisition section 150, and when the CPU 101 executes the program module, the number acquisition section 150 is realized.

[Operation example]
An example of the operation of the information processing device 10 of this embodiment will be described using FIG. 7. FIG. 7 is a flowchart showing the processing flow of the information processing device 10 according to the third embodiment. Note that, below, the steps (S201 to S203) that are different from the first embodiment and the second embodiment will be mainly explained.

When acquiring a product identification code, the code acquisition unit 122 counts the number of acquired product identification codes (S201). Then, the number acquisition unit 150 acquires the number of product shapes recognized by the image recognition unit 110 as the number of products included in the image generated by the imaging unit 124 (S202). Further, the determination unit 140 compares the number of product identification codes tallied in S201 with the number of products acquired in S202, and checks whether the two match (S203). If the number of products and the number of product identification codes do not match (S203: NO), the determination unit 140 determines that some kind of error has occurred, such as a misreading of the product, and prompts, for example, to reread the product. A predetermined message such as a message or a warning message is displayed on the display or output as audio information (S107). On the other hand, if the number of products and the number of product identification codes match (S203: YES), the determination unit 140 determines that no error such as a product misreading has occurred, and the details are explained in the first embodiment. As described above, it is checked whether the size or shape obtained from the image and the size/shape information obtained from the product identification code satisfy predetermined criteria (S106).

[Operations and effects of the third embodiment]
As described above, in this embodiment, it is further determined whether the number of products recognized from the image and the number of product identification codes actually recognized by the information processing device 10 match. Thus, according to the present embodiment, it is possible to determine whether an error such as a misreading of the product has occurred, and if such an error has occurred, the product in the information processing device 10 It is possible to encourage students to recognize information accurately.

[Fourth embodiment]
In this embodiment, a mode will be described in which the information processing device 10 described in each of the above embodiments is combined with a technology using a weight sensor.

When determining the validity of a product recognized by the information processing device 10 based on the weight measured by the weight sensor and the weight of the product linked to the product identification code, if the weight of the product is extremely light. Or, if the same product has variations in weight, correct judgment results may not be obtained. When targeting such a product, the information processing device 10 of this embodiment acquires the size or shape of the product recognized from the image and the product identification code, as described in each of the above embodiments. The validity of the recognition is determined based on the size/shape information.

[Processing configuration]
FIG. 8 is a diagram conceptually showing the processing configuration of the information processing device 10 in the fourth embodiment. Note that the information processing device 10 of the present embodiment includes weight sensors (160 and 162) at a place where unregistered products are placed and a place where registered products are placed, respectively, and basically the weight sensors perform measurement. The validity of the product recognized by the information processing device 10 is determined based on the amount of change in weight.

As shown in FIG. 8, the information processing device 10 of this embodiment further includes a weight information storage section 142. The weight information storage unit 142 stores information that is linked to a product identification code and indicates the weight of each product with the product identification code (hereinafter referred to as weight information), or information that indicates the weight deviation of the product (hereinafter referred to as weight information). , weight deviation information). FIG. 9 shows an example of information stored in the weight information storage section 142. FIG. 9 is a diagram showing an example of information stored in the weight information storage section 142. In the example of FIG. 9, the weight information storage unit 142 stores weight information and weight deviation information in association with the product identification code.

The determination unit 140 of this embodiment acquires a product identification code from the code acquisition unit 122, and uses the product identification code to link the product identification code from the weight information storage unit 142 to the product identification code acquired by the code acquisition unit 122. Obtain weight information or weight deviation information. If the weight information storage unit 142 stores information as shown in FIG. 9, the determination unit 140 can acquire the weight information and weight deviation information. Then, based on the acquired weight information or weight deviation information, the determining unit 140 determines that (1) the weight associated with the acquired product identification code is equal to or greater than the reference value, (2) the acquired product identification code Whether at least one of the following conditions is satisfied: (3) the weight associated with the obtained product identification code is equal to or greater than a predetermined threshold. Determine.

These conditions are conditions for determining whether the product with the product identification code acquired by the code acquisition unit 122 is not suitable for processing by the weight sensor. In (1) above, for example, when a product is very light, a product with a small amount of change in weight that can be detected by a weight sensor is likely to be misrecognized when processed by a weight sensor, so it is difficult to determine whether or not it is such a product. This is the condition to do so. Moreover, the above (2) is a condition for determining whether or not the product cannot be processed by the weight sensor in the first place if the weight is not set. In addition, the above (3) is a condition for determining whether a product with a large variation in weight is likely to be misrecognized when processed by a weight sensor.

When the determination unit 140 of this embodiment determines that any of the above (1) to (3) applies, the determination unit 140 performs the determination described in each of the above embodiments, that is, information processing using the size or shape of the product. A process for determining the validity of recognition by the device 10 is executed. Specifically, if the determination unit 140 of this embodiment determines that any of the above (1) to (3) applies, the determination unit 140 instructs the image recognition unit 110 and the size/shape information acquisition unit 130 to perform the above-described operations. A request to execute the process described in each embodiment is notified. Upon receiving this notification, the image recognition unit 110 sends the size or shape of the product recognized from the image generated by the imaging unit 124 to the determination unit 140. Further, the code acquisition unit 122 that has received this notification sends the size/shape information linked to the product identification code acquired by the code acquisition unit 122 to the determination unit 140. The determination unit 140 uses the size or shape of the product received from the image recognition unit 110 and the size/shape information received from the size/shape information acquisition unit 130, as described in each of the above embodiments. , executes the determination process.

[Hardware configuration]
The information processing apparatus 10 of this embodiment also has a hardware configuration as shown in FIG. 4, similarly to the first embodiment and the second embodiment. The storage 103 further stores a program module that implements the function of the determination unit 140 of this embodiment, and when the CPU 101 executes the program module, the determination unit 140 of this embodiment is implemented. Furthermore, the storage 103 can realize the function of the weight information storage section 142.

[Operation example]
An example of the operation of the information processing device 10 of this embodiment will be described using FIG. 10. FIG. 10 is a flowchart showing the processing flow of the information processing device 10 of the fourth embodiment. Note that in the following explanation, the processes added in this embodiment (S301 to S303) will be mainly explained based on the flowchart in FIG. 4.

Using the product identification code acquired by the code acquisition unit 122 in S101, the determination unit 140 refers to the weight information storage unit 142 and acquires weight information or weight deviation information associated with the product identification code (S301). . Then, using the acquired weight information or weight deviation information, the determination unit 140 determines whether the product with the product identification code acquired by the code acquisition unit 122 in S101 is a product that is not suitable for determination processing using a weight sensor. is determined (S302). Specifically, the determination unit 140 determines whether (1) the weight associated with the acquired product identification code is equal to or greater than a reference value, or (2) the weight associated with the acquired product identification code is not set. (3) The weight deviation associated with the obtained product identification code is greater than or equal to a predetermined threshold.

If the product with the product identification code acquired by the code acquisition unit 122 in S101 is a product that is not suitable for determination processing using a weight sensor (S302: YES), the determination unit 140 uses the image recognition unit 110 and the size / The shape information acquisition unit 130 is notified of this, and the processing described in the first embodiment is executed (S102 to S107). On the other hand, if the product with the product identification code acquired by the code acquisition unit 122 in S101 is a product that does not cause any problem even if the determination process using the weight sensor is performed (S302: NO), the determination process using the weight sensor is performed. It is executed (S303).

[Operations and effects of the fourth embodiment]
As described above, in this embodiment, it is determined whether the product with the product identification code acquired by the code acquisition unit 122 is not suitable for determination processing using a weight sensor. If it is determined that the product with the product identification code acquired by the code acquisition unit 122 is not suitable for determination processing using a weight sensor, the size or shape of the product recognized from the image and the product identification code are Determination processing is performed based on the linked size/shape information. This can prevent erroneous recognition that may occur when a weight sensor or the like is used, and can improve the accuracy of determining the validity of products recognized by the information processing device 10.

[Modification of the fourth embodiment]
In this embodiment, the determination unit 140 basically performs determination processing based on the amount of change in weight measured by the weight sensor, and for products that meet predetermined conditions, the determination unit 140 performs determination processing based on the amount of change in weight measured by the weight sensor. An example of performing judgment processing based on the above is shown. However, the determination unit 140 is not limited to this, and basically performs determination processing based on the size or shape recognized from the image, and uses a weight sensor for products that meet conditions different from the above-mentioned predetermined conditions. The determination processing may be performed based on the amount of change in the measured weight. Here, ``products that meet different conditions'' are products that are not suitable for image recognition processing, such as thin products that have no thickness, because many feature points may be recognized depending on the orientation and posture. . In such a case, the information processing device 10 stores the product identification code of the product that is not suitable for image recognition processing in a storage unit (not shown), and the determination unit 140 stores the product identification code of the product that is not suitable for image recognition processing, and the determination unit 140 combines the product identification code stored in the storage unit with the product identification code stored in the storage unit. When the same product identification code is acquired by the code acquisition unit 122, the validity of the product recognized by the information processing device 10 is determined based on the change in weight measured by the weight sensor.

[Fifth embodiment]
In this embodiment, an information processing apparatus 10 having a minimum configuration capable of obtaining the effects of the present invention will be described. A device (for example, a server device) separate from a device for registering products to be paid (hereinafter referred to as a product registration device) is provided so as to be able to communicate with the product registration device via, for example, a LAN (Local Area Network).

[Processing configuration]
FIG. 11 is a diagram conceptually showing the processing configuration of the information processing device 10 in the fifth embodiment. As shown in FIG. 11, the information processing device 10 of this embodiment includes an image recognition section 110, a code acquisition section 122, a size/shape information acquisition section 130, and a determination section 140.

The image recognition unit 110 recognizes the size or shape of the product in the image using an image associated with depth information indicating the distance in the depth direction of the product included in the imaging range. The code acquisition unit 122 and the size/shape information acquisition unit 130 acquire size/shape information linked to the acquired product identification code. The determining unit 140 determines whether the size or shape of the product recognized from the image and the size/shape information acquired using the product identification code satisfy a predetermined criterion.

[Hardware configuration]
The information processing device 10 of this embodiment has a hardware configuration as shown in FIG. Note that the imaging device 1043 and the reading device 1044 are not connected to the input/output interface 104 of the information processing device 10 of this embodiment, and images and product identification codes can be acquired from other devices via the communication module 105. It may be configured as follows. The image recognition unit 110, code acquisition unit 122, size/shape information acquisition unit 130, and determination unit 140 of this embodiment are program modules in which the CPU 101 realizes the functions of each processing unit, as in each of the above embodiments. This is achieved by executing.

[Operation example]
An example of the operation of the information processing device 10 of this embodiment will be described using FIG. 12. FIG. 12 is a flowchart showing the processing flow of the information processing device 10 of the fifth embodiment.

The code acquisition unit 122 communicates with the product registration device via a network such as a LAN, and obtains the product identification code of the product to be paid for, which is read by the product registration device (S401). Then, the size/shape information acquisition unit 130 uses the product identification code acquired in S401 to read out size/shape information associated with the product identification code (S402). In this embodiment, the size/shape information is stored, for example, in a storage unit (not shown) of another device that can communicate with the information processing device 10, in association with the product identification code.

Further, the image recognition unit 110 communicates with the product registration device via a network such as a LAN, and acquires an image generated by an imaging means provided in the product registration device (S403). The imaging means provided in this product registration device is a so-called 3D camera, and can generate images and depth information in association with each other. Then, the image recognition unit 110 recognizes the shape of the product in the image using known image recognition processing, and also recognizes the shape of the product in the image based on the depth information associated with the image and the size of the area of the product in the image. The actual size of the product is calculated (S404).

The determination unit 140 determines whether the size/shape information acquired in S402 and the size and shape of the product acquired in S404 satisfy predetermined criteria (S405), and sends the results to the product registration device. (S406).

[Operations and effects of the fifth embodiment]
As described above, in this embodiment, the actual size or shape of the product recognized from the image is compared with the theoretical size or shape of the product linked to the product identification code. Thereby, the validity of the product recognized by the product registration device can be accurately determined.

[Sixth embodiment]
In this embodiment, a payment system including a product registration device that performs registration processing of products to be paid for, and a payment device that performs payment processing for the price of products registered as payment targets will be described.

〔System configuration〕
FIG. 13 is a diagram conceptually showing the system configuration of the payment system 1 in the sixth embodiment. As shown in FIG. 13, the payment system 1 includes an information processing device 10 that functions as a product registration device, a payment device 30 that performs payment processing for products registered in the information processing device 10, and a server device 40. ,including. In this case, the information processing device 10, the payment device 30, and the server device 40 are communicably connected via a network such as a LAN. Further, the information processing device 10 has the same configuration as any one of the first to fourth embodiments. In the following description, a case where the information processing apparatus 10 of this embodiment has the configuration of FIG. 1 as in the first embodiment will be exemplified. Note that the image recognition unit 110, code acquisition unit 122, size/shape information acquisition unit 130, and determination unit 140 of the information processing device 10 are not limited to the example of FIG. 13, as described in the fifth embodiment, A device (for example, server device 40) separate from the product registration device may be provided to be able to communicate with the product registration device via, for example, a LAN. In this case, the product registration device includes a processing unit that obtains a product identification code from a product information symbol, etc., separately from the code acquisition unit 122 of the information processing device 10.

[Operation example]
The processing flow of the payment system 1 of this embodiment will be explained using FIG. 14. FIG. 14 is a sequence diagram showing the processing flow of the payment system 1 according to the sixth embodiment.

The information processing device 10 executes the processes of S101 to S105 in the flowchart of FIG. 4, as described in the first embodiment. Then, the determination in S106 of the flowchart in FIG. 4 is performed. If the determination unit 140 determines that the predetermined criteria are not met (S106: NO), the determination unit 140 performs a predetermined output (for example, an output that prompts re-recognition of the product) (S107), and the processes in S101 to S105 are completed. will be executed again. On the other hand, when the determination unit 140 determines that the predetermined criteria are met (S106: YES), the product recognition unit 120 determines that the product has been correctly recognized by the information processing device 10, and the code acquisition unit 122 acquires the product. Product information linked to the product identification code is read (S501). Specifically, the product recognition unit 120 uses the product identification code acquired by the code acquisition unit 122 to obtain the product identification code and product information (for example, product name, unit price of the product, presence or absence of a discount or discount, etc.). The product information associated with the obtained product identification code is read by referring to a storage unit (not shown) that stores the product identification code in a linked manner. Then, the product recognition unit 120 adds the read product information as information (hereinafter referred to as payment information) used in the payment process executed by the information processing device 10 (S502). The payment information includes at least the name of each product to be paid, the unit price and quantity of each product to be paid, the total amount of all the products to be paid, and the like. The processes of S101 to S107, S501, and S502 are repeated until the registration of the product to be settled in one transaction is completed (S503: NO). The information processing device 10 detects, for example, an event such as pressing of a “subtotal” button using an input device provided in the information processing device 10, and detects that registration of products to be settled in one transaction is completed.

If it is detected that the registration of the product to be settled in one transaction has been completed (S503: YES), the information processing device 10 sends the settlement information for the transaction generated in S502 via the server device 40, It is transmitted to the payment device 30 (S504). The information processing device 10 specifies the payment device 30 to which payment information should be sent, for example, through an input operation by an operator of the information processing device 10, and transmits the payment information. In this case, the information processing device 10 transmits the payment information and information specifying the payment device 30 to which the payment information is to be sent to the server device 40 . Then, the server device 40 transmits the payment information to the payment device 30 designated by the information processing device 10, and the payment device 30 performs payment processing using the payment information sent from the server device 40. (S505). The information processing device 10 is not limited to this, and the information processing device 10 transmits the payment information to, for example, the server device 40, and also pays out to the customer a portable medium that includes information that identifies the payment information and is readable by the payment device 30. The portable medium may be read into any payment device 30 by the customer's operation. In this case, the payment device 30 reads payment information from the server device 40 using the identification information included in the portable medium, and performs payment processing (S505).

Although the embodiments of the present invention have been described above with reference to the drawings, these are merely examples of the present invention, and various configurations other than those described above may also be adopted.

For example, in the first embodiment, the imaging unit 124 acquires an image when a product is taken out from a place where unregistered products are placed (for example, a basket) in order to hold the product before registration over the product recognition unit 120. However, the image recognition unit 110 may be configured to recognize the size or shape of the product from the image. In this case, it can be determined whether the unregistered product taken out from the placement location matches the product actually registered as a payment target. Further, for example, the imaging unit 124 may be configured to acquire an image of the registered product being placed at the placement location, and the image recognition unit 110 may be configured to recognize the size or shape of the product from the image. . In this case, it can be determined whether the registered product placed at the placement location matches the product actually registered as a payment target. By doing so, it is possible to discover fraudulent acts such as product substitution during product registration, and to determine whether or not the recognized product is the correct product.

In addition, in the multiple flowcharts or sequence diagrams used in the above explanation, multiple steps (processes) are described in order, but the execution order of the steps executed in each embodiment is the order in which they are described. not limited to. In each embodiment, the order of the illustrated steps can be changed within a range that does not affect the content. Furthermore, the above-described embodiments can be combined as long as the contents do not conflict with each other.

Below, examples of reference forms will be added.
1.
an image recognition unit that recognizes the size or shape of the product in the image using an image associated with depth information indicating the distance in the depth direction of the product included in the imaging range;
a code acquisition means for acquiring a product identification code of a product to be paid;
Size/shape information acquisition means for acquiring size/shape information linked to the acquired product identification code;
determining means for determining whether the size or shape of the product recognized from the image and the size/shape information acquired using the product identification code meet a predetermined standard;
A product registration device comprising:
2.
Further comprising thermography information acquisition means for acquiring thermography information in association with the image,
The image recognition means includes:
using the acquired thermography information to exclude a portion presumed to be a person from the shape of the product recognized from the image;
1. Product registration device described in .
3.
A plurality of pieces of size/shape information are linked to the product identification code,
The determining means determines whether the recognized size or shape and each of the plurality of size/shape information satisfy the predetermined criteria.
1. or 2. Product registration device described in .
4.
further comprising an imaging device that captures the image in accordance with the timing at which the code acquisition device reads the product identification code;
1. From 3. A product registration device according to any one of the above.
5.
Further comprising a number acquisition means for acquiring the number of products included in the imaging range,
The determining means determines whether the number of the acquired product identification codes and the number of the acquired products match.
1. From 4. A product registration device according to any one of the above.
6.
The product identification code is further associated with the weight or weight deviation of the product with the product identification code,
The determination means determines whether the weight associated with the acquired product identification code is equal to or greater than a reference value, when the weight associated with the acquired product identification code is unset, or when the weight associated with the acquired product identification code is unset. performing the determination in at least one of the following cases: the weight deviation associated with the product identification code is equal to or greater than a predetermined threshold;
1. From 5. A product registration device according to any one of the above.
7.
The computer is
Recognizing the size or shape of the product in the image using an image associated with depth information indicating the distance in the depth direction of the product included in the imaging range,
Obtain the product identification code of the product to be paid,
Obtaining size/shape information linked to the obtained product identification code,
Calculating the difference between the size or shape of the product recognized from the image and the size/shape information obtained using the product identification code, and determining whether the difference satisfies a predetermined standard. judge,
A control method including:
8.
The computer,
acquiring thermography information in association with the image;
using the acquired thermography information to exclude a portion presumed to be a person from the shape of the product recognized from the image;
7. Control method described in .
9.
A plurality of pieces of size/shape information are linked to the product identification code,
The computer,
determining whether the recognized size or shape and each of the plurality of size/shape information satisfy the predetermined criteria;
7. or 8. Control method described in .
10.
The computer,
capturing the image according to the timing of reading the product identification code;
7. From 9. The control method according to any one of.
11.
The computer,
Obtaining the number of products included in the imaging range,
determining whether the number of the obtained product identification codes matches the number of the obtained products;
7. From 10. The control method according to any one of.
12.
The product identification code is further associated with the weight or weight deviation of the product with the product identification code,
The computer,
If the weight linked to the obtained product identification code is equal to or higher than a reference value, if the weight linked to the obtained product identification code is not set, or if the obtained product identification code performing the determination in at least one of the following cases: when the weight deviation associated with the weight deviation is equal to or greater than a predetermined threshold;
7. From 11. The control method according to any one of.
13.
computer,
image recognition means for recognizing the size or shape of the product in the image using an image associated with depth information indicating the distance in the depth direction of the product included in the imaging range;
code acquisition means for acquiring the product identification code of the product to be paid;
size/shape information acquisition means for acquiring size/shape information linked to the acquired product identification code;
Calculating the difference between the size or shape of the product recognized from the image and the size/shape information obtained using the product identification code, and determining whether the difference satisfies a predetermined standard. determination means for determining;
A program to function as
14.
The computer,
thermography information acquisition means for acquiring thermography information in association with the image;
The image recognition means,
means for excluding a portion presumed to be a person from the shape of the product recognized from the image using the acquired thermography information;
13. The program described in.
15.
A plurality of pieces of size/shape information are linked to the product identification code,
The computer,
The determination means determines whether the recognized size or shape and each of the plurality of size/shape information satisfy the predetermined criteria;
13. or 14. The program described in.
16.
The computer,
imaging means for capturing the image in accordance with the timing at which the code acquisition means reads the product identification code;
13. From 15. A program described in any one of the following.
17.
The computer,
a number acquisition means for acquiring the number of products included in the imaging range;
The determination means determines whether the number of the acquired product identification codes matches the number of the acquired products;
13. From 16. A program described in any one of the following.
18.
The product identification code is further associated with the weight or weight deviation of the product with the product identification code,
The computer,
The determining means determines that when the weight associated with the acquired product identification code is equal to or higher than a reference value, when the weight associated with the acquired product identification code is not set, or means for executing the determination in at least one of the following cases: the weight deviation linked to the acquired product identification code is equal to or greater than a predetermined threshold;
13. From 17. A program described in any one of the following.

1 Settlement system 10 Information processing device 101 CPU
102 Memory 103 Storage 104 Input/output interface 1041 Display device 1042 Input device 1043 Imaging device 1044 Reading device 105 Communication module 110 Image recognition section 120 Product recognition section 122 Code acquisition section 124 Imaging section 126 Thermography information acquisition section 130 Size/shape information acquisition Section 132 Size/shape information storage section 140 Judgment section 142 Weight information storage section 150 Quantity acquisition section 170 Weight information acquisition section 20 Basket 30 Settlement device 40 Server device

Claims (10)

a first acquisition means provided in a container for storing and transporting products and acquiring product information regarding the products to be settled;
a second acquisition means provided in the container and configured to acquire image information of a product that a customer puts in the container;
Output means that outputs a warning when the product that the customer has placed in the container is determined to be inappropriate, using the information acquired by the first and second acquisition means;
Equipped with
The first acquisition means is an information processing system different from the second acquisition means. Further comprising a determination means for determining the validity of the product placed in the container by the customer using the product information and the image information,
The output means outputs a warning when the determination means determines that the product placed in the container by the customer is inappropriate.
The information processing system according to claim 1. The determination means determines that the product that the customer has placed in the container is inappropriate if the product that the customer has placed in the container is different from the product for which the first acquisition device has acquired product information. ,
The information processing system according to claim 2. a first appearance information acquisition means for acquiring appearance information linked to the product information;
further comprising a second appearance information acquisition means for acquiring appearance information of the product using the image information,
The determination means uses the information acquired by the first appearance information acquisition means and the second appearance information acquisition means to determine the validity of the product placed in the container by the customer.
The information processing system according to claim 2 or 3. further comprising a number acquisition means provided in the container and configured to acquire the number of products included in the image information;
The determination means determines the validity of the product placed in the container using the number of products for which the first acquisition means has acquired product information and the number of products obtained by the number acquisition means. judge,
The information processing system according to any one of claims 2 to 4. Further comprising a third acquisition means provided in the container and configured to acquire weight information of the product placed in the container,
The determining means uses the weight information to determine the validity of the product placed in the container.
The information processing system according to any one of claims 2 to 5. The information acquired by the first acquisition means is further associated with the weight or weight deviation of the product,
The determination means determines that when the weight linked to the information acquired by the first acquisition means is less than or equal to a reference value, the weight linked to the information acquired by the first acquisition means is unset. or, when the weight deviation linked to the information acquired by the first acquisition means is equal to or greater than a predetermined threshold, the product placed in the container is determine validity;
An information processing system according to any one of claims 2 to 6. The first acquisition means has a code reader that reads a code attached to a product, and acquires the code or a processing result of the code as the product information,
The second acquisition means includes an imaging device that captures an image of the product, and acquires the image or a processing result of the image as the image information.
An information processing system according to any one of claims 1 to 7. A computer installed in a container for storing and transporting products,
a first acquisition step of acquiring product information regarding the product to be paid;
a second acquisition step of acquiring image information of the product that the customer puts in the container;
an output step of outputting a warning when the product placed in the container by the customer is determined to be inappropriate using the information acquired in the first and second acquisition steps;
Run
The first acquisition step uses a different control method from the second acquisition step. A computer installed in a container for storing and transporting products,
a first acquisition means for acquiring product information regarding the product to be settled;
a second acquisition means for acquiring image information of a product that a customer puts in the container;
Output means that outputs a warning when the product that the customer has placed in the container is determined to be inappropriate, using the information acquired by the first and second acquisition means;
function as
The first acquisition means is a different program from the second acquisition means.

Images