JP6008405B2 - Self-POS device and operation method thereof - Google Patents

Description

  The present invention relates to a self-POS device, and more particularly, to a self-POS device that supports self-POS barcode reading, appropriately guides customer information reading, and prevents customer fraud.

  Conventionally, in stores such as superstores (supermarkets) and convenience stores (convenience stores), POS terminals have been installed to expedite customer checkout, improve efficiency, and achieve efficient sales, ordering, and inventory information management. Developed and widely used.

  In order to improve the management efficiency of such a store, in order to reduce labor costs, the customer himself / herself operates the POS to input the product information desired to be purchased and performs a checkout process (hereinafter simply referred to as “self-service”). POS) is also being proposed and is becoming popular. In such a self-POS, it is necessary to input merchandise information correctly by improving the operability that allows the customer to operate easily, quickly and accurately, and preventing fraud by the customer.

  In a self-POS system installed in a store, it is common to place a store clerk for the purpose of guidance and monitoring at the rate of one person in several self-POS systems for operational guidance (support) to customers and fraud prevention. is there.

  Conventional techniques related to such self-POS are disclosed in various patent documents. By providing a product-specific data storage unit that stores product data such as product name and unit price, as well as data related to the appearance characteristics of the product, by product, it enables sales registration processing of products without complicated key operations. A merchandise sales registration data processing apparatus is disclosed (for example, refer to Patent Document 1).

  Also, it is troublesome to register the weight of each product in the product data file by capturing the weight data so that the association between the product weight and the barcode is stored and learned when the product barcode is read. A self-checkout terminal that solves this problem is disclosed (for example, see Patent Document 2).

  Furthermore, a self-checkout terminal is disclosed that includes an imaging unit such as a product barcode reader, an alarm device, and a monitoring camera that outputs image data, and prevents unauthorized take-out of products by customers (for example, Patent Document 3). reference.).

  Furthermore, a self-checkout terminal is disclosed in which a product weight detection unit is provided so that a customer can perform accounting processing without reading a product barcode and prevent the product from being illegally taken home (for example, a patent). Reference 4).

JP 2002-74511 A JP 2007-293765 A JP 2008-27427 A JP 2008-276540 A

  However, the conventional techniques as described above have problems to be improved in terms of operability and detection and prevention of fraud by customers. For example, there are many products that change the volume of products without notice due to price increases of raw materials. In such a case, it is necessary to update weight data and cause troubles with customers. Therefore, the monitoring method using the weight data of the product is not practical, and the weight detection unit obstructs the reduction in size and weight of the apparatus.

  In addition, when the shape recognition technique is used, it is necessary to register the basic figure of the product and the barcode in association with each other, so that a large number of man-hours are required particularly in stores that handle a wide variety of products. In addition, in the method of extracting feature points of photographed products and storing them in association with barcodes, strict judgments are made even for low-priced products, and there is a risk that sweet judgments may be made for expensive products. ineffective.

  Furthermore, it is necessary to arrange one store clerk for at least a plurality of self POSs, and it is not possible to sufficiently obtain the effect of reducing the labor cost, which is the main purpose of introducing the self POS.

  The present invention has been made in view of the above-described problems or disadvantages of the prior art, and overcomes or reduces such problems, and is excellent in operability and capable of effectively preventing fraud by a customer and its The main purpose is to provide a method of operation.

  The self-POS device and its operating method according to the present invention employ the following characteristic configuration.

(1) A self-POS device of the present invention includes a scanner that reads product data of a product that a customer desires to purchase at a store or the like, and an imaging unit that captures an image of the product that is read by the scanner. In the self-POS device that performs the checkout process for products,
A registration unit that extracts a feature point of a product handled in the store and registers a basic graphic; and a determination unit that determines an image of the product photographed by the imaging unit to be compared with the basic graphic.
If a basic figure to be compared with the image of the product is not registered in the registration unit, a self-POS that generates the basic figure based on the first image photographed by the photographing unit and registers it in the registration unit apparatus.
(2) A self-POS that includes a scanner that reads product data of a product that a customer desires to purchase in a store or the like, and an imaging unit that captures an image of the product that is read by the scanner, and performs a checkout process for the product desired by the customer In the device
Registration means for extracting feature points of products handled in the store and registering basic figures;
A self-POS device comprising: determination means for comparing an image of the product taken by the imaging means with the basic figure; and alarm means for generating an alarm when the determination means detects the possibility of unauthorized reading.
(3) A self-POS that includes a scanner that reads product data of a product that a customer desires to purchase in a store or the like, and an imaging unit that captures an image of the product that is read by the scanner, and performs a checkout process for the product desired by the customer In the device
Registration means for extracting feature points of products handled in the store and registering basic figures;
Determination means for determining an image of the product photographed by the imaging means to be compared with the basic figure,
The image pickup means is a self-POS device comprising at least two pieces for picking up images on different sides of a product read by the scanner.
(4) The above (1), (2) or (3) further comprising correlation value calculation means for obtaining a correlation value between the image of the product read by the imaging means and the basic figure of the product registered in the registration means. ) Self POS device.
(5) The self-POS device according to (4), wherein when the correlation value is equal to or greater than a predetermined threshold, the basic figure registered in the registration unit is replaced.
(6) The self-POS device according to (2), further including an operation stopping unit that stops the reading operation of the product by the scanner when the alarm unit issues an alarm.
(7) The self-POS device according to (6), wherein the operation stopping unit can be operated by a store clerk with another device in the store.

The operation method of the self-POS device of the present invention is as follows.
(8) A self-POS that includes a scanner that reads product data of a product that a customer desires to purchase in a store or the like, and an imaging unit that captures an image of the product that is read by the scanner, and performs a checkout process for the product purchased by the customer In the operation method of the device,
A basic image registration step of registering a basic image by extracting feature points of an image of the product corresponding to the product data read by the scanner;
A determination step of determining the image captured by the imaging means to be compared with the basic image;
And a step of creating the basic image based on the product image photographed by the photographing means when the basic image to be compared with the product image photographed by the imaging means does not exist.
(9) A self that performs a checkout process for a product purchased by the customer, including a scanner that reads product data of a product that the customer desires to purchase in a store or the like, and an imaging unit that captures an image of the product read by the scanner. In the operation method of the POS device,
A basic image registration step of registering a basic image by extracting feature points of an image of the product corresponding to the product data read by the scanner;
A determination step of determining an image of the product photographed by the imaging means to be compared with the basic image;
An operation method of the self-POS device, comprising: an alarm step for generating an alarm when the possibility of illegal reading is detected in the determination step.
(10) The operation method of the self-POS device according to (8) or (9), wherein the determination step sets a determination criterion for a product in a high price range by changing a determination threshold value for each price range of the product.
(11) The operation method of the self-POS device according to (9), further including an operation stop step of stopping the reading operation of the product data of the product by a store clerk when the alarm is generated by the alarm step.
(12) The self-POS operation method according to (9), further comprising the step of ignoring the alarm and continuing the reading operation of the product to facilitate the settlement operation when the alarm is generated by the alarm step. .

  According to the self-POS device and its operation method of the present invention adopting the above-described characteristic configuration, there are the following practical specific effects.

  According to the present invention, since it is not necessary to register in advance a non-form figure for extracting feature points of a product, the operation is simple. The reason is that the basic figure is registered by reading the figure with the camera (imaging means) and extracting the feature points when reading the commodity code (barcode) of the commodity with the scanner. This is because a figure can be registered. As the number of samples to be read increases, the accuracy of shape recognition can be improved by the learning function of basic graphics of products handled in a store.

  Further, according to the present invention, illegal reading by a customer can be effectively prevented. This is because fraudulent acts by customers can be determined by referring to basic graphics based on graphic data taken by a camera. Furthermore, the fraudulent act by the customer can be obtained by photographing with a plurality of cameras from opposite sides of the product. Furthermore, when a possibility of fraud is detected, the store clerk is notified by an alarm lamp or the like, and if it is manned POS or the like, the self-POS reading operation can be stopped.

The Example of the self POS used for the self POS apparatus of this invention is shown, (A) is a front view, (B) is a right view, (C) is a perspective view. It is explanatory drawing of the goods code reading operation | movement by the scanner of self POS10 of FIG. 1, (A) is a side view, (B) is a partial expansion perspective view. A specific example of the self-POS shown in FIG. 1 is shown, in which FIG. It is explanatory drawing of the dynamic space carving method applicable to this invention. BRIEF DESCRIPTION OF THE DRAWINGS It is explanatory drawing of the self-POS apparatus by this invention, (A) is an explanatory drawing of correlation with the barcode of goods, and a basic figure, (B) is explanatory drawing of the correlation threshold value according to goods price, (C) is a basic figure learning threshold value. An explanatory diagram and (D) show examples of display screens on the display of the self-POS. It is a system configuration figure of a general store system. It is a perspective view which shows the specific example of manned POS shown in FIG. 3 is a flowchart illustrating a self-POS operation method according to the present invention.

  Next, preferred embodiments of the self-POS device and its operating method according to the present invention will be described in detail with reference to the accompanying drawings. However, it should be noted that such examples are merely illustrative of the present invention and do not limit the present invention in any way.

  First, FIG. 1 shows a schematic overall configuration of a self-POS that is a main part of the present invention. FIG. 1A is a front view of the self-POS 10, FIG. 1B is a right side view of the self-POS 10, and FIG. C) is a perspective view of the self-POS 10.

  The self POS 10 includes a main body 20 and a control unit 30 provided above the main body 20. The main body 20 has a barcode scanner (hereinafter simply referred to as a scanner) 21 that scans product data such as a product code (hereinafter referred to as a barcode) attached to a product (not shown) that is displayed and sold in a store. A camera (imaging means) 22 disposed in the vicinity of the scanner 21 and a housing 23 that houses an electronic circuit, a power supply circuit, and the like that control each unit. On the other hand, the control unit 30 includes a display (display screen) 31 such as a liquid crystal display panel (LCD) and various control units to be described later, which display various characters and image information.

  The self POS device 10 scans and reads a barcode of a product (not shown in FIG. 1) arranged on the front surface of the main body 20 with a scanner 21 and simultaneously takes a three-dimensional image of the product taken with a camera 22. Is extracted and stored in association with the product barcode.

  Here, when the customer scans the barcode of the product with the scanner 21 of the self-POS 10, the output side of the reading laser beam (laser beam) of the scanner 21 is scanned so as not to interfere with the customer's hand. There is a need. Therefore, it is preferable to provide the camera 22 on the scanner 21 side because it facilitates shape recognition.

  FIG. 2 is an explanatory diagram of a product barcode reading operation by the scanner 21 of the self-POS 10 shown in FIG. Further, the self-POS 10 includes an electronic money multi-reader (non-contact IC card reader) 24, a receipt printer 25, and an alarm lamp (see FIG. 2B) 27 used for accounting processing. FIG. 2A shows the scanning range of the reading laser beam 26 by the scanner 21 with diagonal lines. FIG. 2B is an enlarged view of the scanner 21 portion of FIG.

  As shown in FIG. 2B, the scanner 21 scans the barcode 41 by scanning at least a portion of the surface of the product 40 arranged below the barcode 41 with the laser beam 26.

  However, when the customer acts maliciously, for example, a scan is performed with another product hidden behind the product 21 facing the scanner 21 (that is, the product 40 in FIG. 2B). It is also possible to make it appear as if it has gone. In order to prevent such an illegal act, it is better to provide the camera 22 on the side opposite to the scanner 21. Therefore, in FIG. 2B, the camera 22 is shown both on the scanner 21 side and on the lower side thereof. In this way, it is also effective to arrange two cameras 22 at different positions and to monitor the scanning operation from the upper and lower surfaces of the product 40 with these two cameras 22.

  Next, with reference to FIG. 3, a detailed configuration of a preferred embodiment of the self-POS 10 according to the present invention will be described. As shown in FIG. 3A, a receipt printer 25 is installed at the bottom of the main body 20 of the self POS 10, and a scanner (fixed scanner) 21 for reading the barcode 41 of the product 40 is arranged on the top thereof. Has been. The scanner 21 is installed so that the irradiation angle of the laser beam is oblique. Further, an electronic money multi-reader (non-contact IC card reader compliant with a plurality of standards) 24 is installed so as to block the laser light of the scanner 21.

  The controller unit 30 disposed above the main body unit 20 includes a large XGA display (display screen) 31, a magnetic card reader 32, and a speaker 33. The main body 20 and the display 31 are mounted on a metal plate of about 2 mm, and are stably installed and play a role of securing a space for placing a product 40 that the customer plans to purchase at the installation location. . As described above, two cameras 22 are attached in the vicinity of the scanner 21 and at a position where the scanner 21 is looked up from below.

  FIG. 3B shows specifications of specific examples of the control unit 30, the display unit 31, the receipt printer 25, the scanner 21, the electronic money multi-reader 24, the speaker 33, the card reader 32, and the like shown in FIG.

  Next, the operation of the self POS 10 will be briefly described. A customer who has made a purchase at the store selects a product he / she wishes to purchase (40 in FIG. 2B) (or puts it in a shopping cart) and carries it in front of the self-POS 10 for payment. On the sheet metal that forms the foot of the self-POS 10, "Before Scan" is displayed in the yellow frame on the right side, "Scan Area" is displayed in the blue frame on the front of the main unit, and "After Scan" is displayed in the green frame on the left side Has been. Therefore, the product is first placed in the “before scan” area on the right side.

  Thereafter, the products are moved one by one to the scan area, and the barcodes 41 of the products 40 are sequentially read by the scanner 21. In a normal POS (including a self-POS), when the barcode 41 is read, a reading completion sound such as “beep” is issued to notify the reading completion. However, the self-POS 10 of the present invention does not produce a reading completion sound at this point.

  The self-POS 10 inquires the barcode 41 of the read product 40 to a PLU server (described later with reference to FIG. 6) installed in the store. Then, the data of the basic figure and the number of samples stored in association with the barcode 41 are obtained in association with the bar code 41 such as the product name of the product and the unit price of the product.

  In addition, since the two cameras 22 are installed vertically in the scan area of the self POS 10, it is recognized that a human (customer) hand has entered the scan area. Therefore, the shape of the product (object) 40 held by the customer is recognized by reading the barcode 41 simultaneously with the inquiry to the PLU server.

  The customer's hand is recognized by a known method (for example, a method disclosed in JP-A-2006-350434). In addition, for example, the known “dynamic space carving method” shown in FIG. 4 may be used to recognize the product (object) 40 that is partially hidden in the customer's hand.

  The self POS 10 checks whether or not the data received from the PLU server has a basic figure stored in association with the barcode. If there is no basic figure stored in association with the barcode, the object whose shape is recognized this time is set as a basic figure, the value of “number of samples” is set to “1”, and the barcode data is transmitted to the PLU server.

  The PLU server stores the basic graphic received from the self POS 10 and the “number of samples” in association with the barcode. FIG. 5 is an explanatory diagram related to the operation of the self-POS 10 described above. FIG. 5A is a diagram for explaining the association between a barcode and a basic figure. That is, it includes a barcode, the number of times each barcode is read, and a basic figure. FIG. 5B shows the correlation value threshold for each product price. FIG. 5C shows the basic figure learning threshold. FIG. 5D shows a specific example of a display screen displayed on the display 31 of the self-POS 10.

  If there is a basic figure stored in the self POS 10 in association with the barcode 41 of the product 40 read by the scanner 21, the correlation value between the product (object) whose shape has been recognized and the basic figure is calculated. The self-POS 10 has a correlation value threshold value table in which the price of a commodity set in advance and the correlation value threshold value are associated (see FIG. 5B). The correlation value calculated from the product (object) 40 and the basic figure and the price of the product obtained from the PLU server are referred to the correlation value threshold value table, and the correlation value exceeds the threshold value, the product 40 is normally read. Judgment is made and a sound such as “beep” indicating completion of barcode reading is generated. Then, the read product name, the price of the product, etc. are stored in a form that can be settled as a purchased product, and the product name, price, etc. are displayed on the display 31.

  The self-POS 10 also has a preset basic figure learning threshold value (see FIG. 5A). If the correlation value calculated from the product (object) and the basic figure exceeds the basic figure learning threshold, if the number of samples of the original basic figure is N times The basic figure is recalculated using the calculation result of the shape recognition as the learning value for the N + 1th time of the basic figure. The recalculated basic figure and the number of samples are sent to the PLU server together with the barcode data.

  The PLU server rewrites the basic figure and sample number data stored in association with the barcode 41 of the product 40 received from the self POS 10 to the data newly received from the self POS 10. If the correlation value does not exceed the basic figure learning threshold, the shape recognition calculation result is discarded.

  Next, referring to the correlation value threshold value table based on the correlation value calculated from the product (object) and the basic figure and the price of the product obtained from the PLU server, the operation when the correlation value does not exceed the threshold value will be described.

  When the threshold is not exceeded, there are two cases where the shape of the product 40 cannot be recognized correctly and when the customer reads the product maliciously. If the threshold is not exceeded, the self-POS 10 issues a message such as “Please place the product in the correct position” or “Please read again” in order to prompt the customer to re-read the product. Then, the barcode 41 of the product 40 is read again and the shape is recognized again to calculate the correlation value.

  If the result of the recalculation exceeds the threshold value, the reading completion sound, screen display, and learning of the basic figure are performed in the same procedure as described above. If the threshold is not exceeded even after recalculation, the self-POS 10 may scan the two products together with a barcode of another product in front of the barcode 41 of the product 40 and impersonate the product as if it were one product. It is possible that this is happening. Therefore, the self-POS 10 informs the store clerk by turning on or blinking the red alarm lamp 27 shown in FIG. Since the clerk is usually in the counter after the self-POS 10, he notices that the alarm lamp 27 is lit. Conversely, since it is difficult for the customer to see the alarm lamp 27 on the rear side of the self-POS 10, the store clerk can monitor the behavior of the customer without being noticed by the customer.

  Here, with reference to FIG.6 and FIG.7, the store system structure containing the general apparatus for stores provided with the self POS10 is demonstrated. FIG. 7 is a perspective view showing an example of a manned POS 120 constituting the store system in FIG.

  A store system 100 shown in FIG. 6 includes a PLU server (or store server) 110, a plurality of manned POSs 120A and 120B, and an ordering terminal (ST 130 and GOT 140) together with one or more self-POSs 10 as described above. Interconnected. Further, the store LAN 150 is preferably connected to the IP line 170 via the router 160.

  A manned POS 120 shown in FIG. 7 includes a main body 122 and a control unit 124. The control unit 124 includes a display 125 such as a liquid crystal display panel, and operation buttons (software) such as “POS stop” 126 for stopping the operation of the self-POS 10 designated by the operator and “ignore” 127 for ignoring the abnormality of the self-POS 10. Key). Further, in the vicinity of these operation buttons, a product name display unit 128 corresponding to the barcode 41 read by the scanner 21 is provided, and for example, a product name such as “glyco almond chocolate” is displayed.

  The self-POS 10 has an “operation stop function”, and when an illegal reading of the product as described above is detected, the operation of the self-POS 10 designated by the store clerk from the manned POS 120, the ordering terminals 130, 140, etc. is stopped, It can be frozen.

  If the customer is suspicious, the store clerk stops operating the self-POS 10. If the self-POS 10 is in an operation stop state (freeze state), the customer cannot call the store clerk because the customer cannot settle the product 40. Therefore, the store clerk guides the customer to the manned POS 120 or goes to the place where the self POS 10 is installed, and cancels the “operation stop state” of the self POS 10 by a hidden operation. As a result, since a settlement process such as scanning of the product 40 is performed under the monitoring of the store clerk, effective fraud prevention can be performed.

  Note that the “operation stop function” of the self POS 10 is to turn off the power of the self POS 10, ignore the input from the outside even though it is actually in the operating state, The screen may be darkened and the customer may think that the self POS 10 has failed. Since the store clerk has knowledge of the hidden operation for activating the self-POS 10 again, the self-POS 10 can be restored.

  The correlation value threshold value and the basic graphic learning threshold value are preferably stored in the PLU server 110 instead of being stored in the self POS 10. These values are sent from the PLU server 110 to the self POS 10 each time an inquiry is made from the self POS 10 after the barcode is read. Further, the correlation value threshold value may be stored in association with the barcode 41 of the product 40 read by the scanner 21 instead of being set for each price. The same applies to the basic figure learning threshold.

  When the scanning of all the products 40 that the customer desires to purchase is completed, the customer presses the “payment button”. As a result, the settlement amount displayed on the display 31 is confirmed, and then the electronic money multi-reader 24 performs settlement. When the customer's payment is completed, the receipt is printed from the receipt printer 25 and output.

  Next, a self POS operation method according to the present invention, that is, the operation of the above-described self POS 10 will be described in detail with reference to the flowchart of FIG.

  First, it is determined whether or not the barcode 41 of the product 40 read by the scanner 21 of the self POS 10 is input, that is, whether or not there is a basic figure registered or stored in association with the barcode 41 (step S1). If the barcode 41 is not input (step S1: NO), the process returns to step S1 and waits for the barcode 41 to be input. When the barcode 41 is input (step S1: YES), the product 40 is photographed by the camera 22 and image recognition is executed (step S2).

  Next, an inquiry is made to the PLU server 110 (step S3). The product name, price (unit price), basic figure, and number of samples (that is, the number of images of the product photographed so far) N are received from the PLU server 110 (step S4). Then, it is determined whether or not the sample number N is greater than 0 (step S5).

  If there is no basic graphic registered in association with the barcode 41 (step S5: NO), the graphic captured this time by the camera 22 is registered as the basic graphic of the product 40, and the “number of samples N” is set to “1”. (Step S17). A reading end sound “beep” of the barcode 41 is generated (step S18). The product name and price of the product 40 are displayed on the display 31 (step S19), and the process proceeds to step S13 described later.

  On the other hand, when the number of samples N> 0 (step S5: YES), the correlation value between the product photographed by the camera 22, that is, the product 40 subjected to shape recognition and the basic figure is calculated (step S6). The self-POS 10 has a correlation value threshold value table (see FIG. 5B) in which a price of a preset product and a correlation value threshold value are associated with each other. Therefore, whether correlation value> correlation value threshold, that is, whether the correlation value exceeds the threshold with reference to the correlation value threshold table from the correlation value calculated from the product and the basic figure and the price of the product obtained from the PLU server 110 (Step S7).

  If correlation value> correlation value threshold (step S7: YES), it is determined that the product 40 has been read correctly, and a completion sound “beep” indicating completion of reading the barcode 41 is generated (step S8). Then, the product name and price of the product 40 are displayed on the display 31 in a form that can be settled as a purchased product (step S9).

  Next, it is determined whether or not the sample number N exceeds the sample number reference value, that is, whether or not the graphic learning of the product is sufficient (step S10). The image of the product 40 taken by the camera 22 is used for learning, the basic figure is updated, the number of samples is set to N = N + 1 (step S11), and the basic figure is recalculated (step S12). Until the sample number reference value is exceeded, the barcode 41, the original figure and the sample number are transmitted to the PLU server 110, and all images of the product 40 input from the camera 22 are used for learning (step S13). . For example, in the case of a product that is packed with a liquid, such as “Assorted oden”, the feature points cannot be extracted even after learning, so the number of samples N exceeded the reference value. At that time, the product is not associated with image recognition and stored in association with the barcode 41 as no reference graphic. Products not applicable to image recognition can be registered in advance.

  Further, the self-POS 10 has basic figure learning values that are set and registered in advance (see FIG. 5C). When the correlation value calculated from the product 40 and the basic figure exceeds the basic figure learning threshold, when the number of samples of the original basic figure is N times, the shape of this time The basic figure is recalculated using the recognition calculation result as the N + 1th learning value of the basic figure. The recalculated basic figure and the number N of samples are sent to the PLU server 110 together with the barcode 41 data read by the scanner 21. The PLU server 110 rewrites the basic figure and sample number N data stored in association with the barcode received from the self POS 10 to the data newly received from the self POS 10.

  Learning of the basic figure related to the product 40 is performed using, for example, a known parametric eigenspace method or the like. It is known that if the number of shape samples is increased, the recognition rate is improved. For example, the recognition rate improves as the number of frames increases, and according to shape recognition by the partial recognition method, for example, if the number of samples N is 10 or more, the recognition rate is improved to about 99%.

  It is desirable that the above-described sample number reference value is the number of samples at which the recognition rate is about 99%. Therefore, the sample number reference value varies depending on the shape recognition method. If the correlation value does not exceed the basic figure learning threshold, the shape recognition calculation result is discarded. Thereafter, it is determined whether or not the customer has pressed the checkout button (step S14). If the settlement button is not pressed (step S14: NO), the process returns to step S1 described above. On the other hand, if the settlement button is pressed (step S14: YES), the customer confirms the product data and the settlement amount displayed on the display 31, and pays with electronic money (not shown) with the electronic money multireader 24. (Step S15), the receipt printer 25 prints and issues a receipt (step S16), and the settlement or accounting process ends.

  Next, the correlation value calculated from the image of the product 40 taken by the camera (imaging means) 22 and the basic figure refers to the price correlation value threshold table (see FIG. 5B) of the product obtained from the PLU server 110. The operation when the value does not exceed the threshold value (step S7: NO) will be described.

  When the threshold value is not exceeded, there are a case where the shape recognition of the product 40 cannot be performed normally and a case where the customer illegally reads the product 40 maliciously. Therefore, in order to cope with the former case, the self-POS 10 prompts the customer to read the product 40 again, such as “Please read again” or “Place the product in the correct position”. A message is issued (step S20). Therefore, the barcode 41 of the product 40 is read again (step S21), the shape is recognized again, and the correlation value between the figure of the product 40 and the basic figure is calculated (step S22).

  Next, the recalculated correlation value> correlation value threshold value is determined (step S23). If the result of the recalculation exceeds the threshold value (step S23: YES), the process proceeds to step S8 described above, and a reading completion sound is generated. Then, product data such as the product name and price of the product 40 is displayed on the display 31 (step S9), and the above-described basic graphic is learned.

  On the other hand, if the threshold is not exceeded even after recalculation (step S23: NO), the barcode of another product is placed in front of the product 40 and the barcode 41 is read illegally or two by the scanner 21. There is a possibility that an illegal reading operation such as scanning the products together to make it appear as one product is performed. Accordingly, the clerk is instructed to perform an appropriate action as described above (without being noticed by the customer), for example, by turning on or blinking the red alarm lamp 27 provided behind the self-POS 10 to notify the clerk (step S24). The “operation stop function” of the self-POS 10 is activated by operating another manned POS 120 or the like.

  In the above-described conventional self POS, for example, when an illegal reading is detected, the reading operation of the product is simply stopped. If there are many clerk in the store, any one of the clerk can handle, but with a small number of clerk, the self POS is arranged at one corner of the store, especially in the case of a small store, self POS If the reading operation is stopped, an unfavorable result is caused in operation. However, in the self-POS device of the present invention, the store clerk can operate the reading operation of the self-POS 10 by the manned POS 120 (or ordering terminal 130, 140) or the like in the store, so a small number of small stores such as convenience stores ( It can be efficiently operated by 1 to 2 salesclerks.

  In the self-POS device of the present invention, when the alarm lamp 27 of the self-POS 10 is turned on, in addition to the above-mentioned “stop operation” instruction function from the manned POS 120 or the like, the “ignore” button 127 shown in FIG. In the case of a customer who is likely to wake up, it is possible to operate so as to ignore the discrepancy in image recognition. With this “ignore” function, unnecessary troubles with customers can be prevented, or in the case where the store clerk cannot be released, fraud can be ignored and priority can be given to facilitating the work.

  The correlation threshold and the basic figure learning threshold described above are not stored in the self POS 10, but are stored in the PLU server 110, and after reading the barcode 41 of the product 40 by the self POS 10, the PLU server responds to a request from the self POS 10 110 may be configured to be sent to the self-POS 10. Furthermore, the correlation value threshold value may be stored in association with a barcode for each product, instead of being set for each price of the product 40. The same applies to the basic figure learning threshold.

  When the payment is completed, the customer goes home with the already-purchased product placed in the vicinity of the self-POS 10 (for example, the left side) and the issued receipt.

  The preferred embodiments of the self-POS device and its operating method according to the present invention have been described in detail above. However, the present invention should not be limited to such an embodiment, and various modifications and changes can be made according to a specific application.

  For example, a dynamic space carving method (see FIGS. 4A to 4E) for recognizing a product (object) partially hidden in the customer's hand actually has a poor recognition rate. In that case, the research of the recognition method which recognizes the movement of the hand is progressing. For example, when the movement of a human hand is recognized using a plurality of cameras, the recognition rate can be considerably improved. For example, the self-POS does not recognize the shape of the product held in the hand, but only recognizes the movement of the hand.

  When it is recognized that the hand has passed in front of the barcode reading section of the product by the self-POS 10, it is confirmed whether or not the barcode has been read. When it is confirmed that the hand has passed and the reading of the bar code is confirmed, a bar code reading completion sound “beep” is output, and the PLU server 110 is inquired about the product data necessary for the settlement of the product name and price of the product 40. . The product name and price obtained from the PLU server 110 are displayed on the display 31 and data is stored so that payment can be made.

  Even when the movement of the hand cannot be recognized and only the reading of the barcode is detected, the barcode reading completion sound “beep” is output, and the PLU server 110 is inquired about data necessary for the settlement of the product name and price, The product name and price obtained from the PLU server 110 are displayed on the display 31, and the product data is stored so as to be settled.

  If it is recognized that the hand has passed but the barcode 41 cannot be read, a message such as “Please read again” is output to prompt the customer to scan the product again. As a result, if the barcode 41 can be read, a reading completion sound “beep” of the barcode 41 is output, and the PLU server 110 is inquired about data necessary for settlement of the product name and price. Then, the product name and price obtained from the PLU server 110 are displayed on the display 31 and data is stored so as to be settled.

  Even after outputting the “Please read again” rescan message, it is provided at the rear of the self-POS 10 to notify the store clerk when the bar code cannot be read by recognizing only the passage of the hand. The alarm lamp 27 is turned on to notify the store clerk of the abnormality.

  Furthermore, another embodiment will be described. In the current image recognition technology, the product (object) 40 hidden behind the hand cannot be completely recognized. However, it is possible to almost certainly recognize whether or not the product (object) 40 is held in the hand.

  Instead of recognizing that the hand has passed through the scan area, the operation as in the above-described embodiment is performed when the hand holding the product (object) 10 passes through the scan area. Thus, it is possible to prevent a re-read message from being output simply by passing the hand.

  The self-POS device and the operation method thereof according to the present invention employ the characteristic configuration as described above and exhibit a specific effect, and thus are suitable for a POS installed in a store having a small store area such as a supermarket or a convenience store.

10 Self POS
20 Main unit 21 Scanner (product code reading means)
22 Camera (imaging means)
27 Alarm lamp (alarm means)
30 Control unit 31 Display (display screen)
40 products 41 barcode (product code)
100 In-store equipment system 110 PLU server 120 Manned POS

Claims (9)

A scanner that reads product data of a product that the customer desires to purchase in a store or the like, and an imaging unit that captures an image of a hand holding the product of the customer when the product is read by the scanner; In the self POS device that performs the scan process for reading the product data of the product and the checkout process for the product to be purchased,
Extracting feature points of products handled in the store, registering basic figures in association with product data of the products, and images of the products photographed by the imaging means of the products read by the scanner A determination means for comparing with the basic figure associated with the product data,
When the imaging means recognizes the passing of the customer's hand holding the product, but the product data cannot be read by the scanner, it outputs a message prompting another product scan, and the imaging means A self-POS device that does not output a message prompting another product scan when it recognizes a hand that does not have a product . After outputting the message, the imaging means recognizes the passage of the customer's hand holding the product, but if the product data cannot be read by the scanner, an alarm lamp is turned on or blinked. The self-POS device according to claim 1.   3. The self-POS device according to claim 2, wherein the alarm lamp is provided at a rear portion of the self-POS device which is difficult for a customer who performs the scanning process to see.   4. The self-operation according to claim 3, further comprising an operation stopping unit that stops the reading operation of the product by the scanner by another device operated by a store clerk in the store when the alarm lamp is turned on or blinking. 5. POS device.   5. The self-POS device according to claim 4, further comprising an operation stop canceling unit that cancels the stop of the reading operation of the product by another device operated by a store clerk in the store. A scanner that reads product data of a product that the customer desires to purchase in a store or the like, and an imaging unit that captures an image of a hand holding the product of the customer when the product is read by the scanner; In the operation method of the self-POS device that performs the scan process for reading the product data of the product and the checkout process for the product to be purchased,
A basic image registration step of extracting feature points of the product image corresponding to the product data read by the scanner and registering a basic image in association with the product data of the product;
A determination step of comparing the image captured by the imaging unit with a basic image associated with product data of a product read by the scanner;
When the imaging means recognizes the passing of the customer's hand holding the product, but the product data cannot be read by the scanner, it outputs a message prompting another product scan, and the imaging means If it recognizes the passage of a hand that doesn't have the product, it will not output a message prompting you to scan the product again ;
A method of operating a self-POS device, comprising: The self-POS device has an alarm lamp at the rear that is difficult to see from the customer performing the scanning process,
After outputting the message, the imaging means recognizes the passage of the customer's hand holding the product, but if the product data cannot be read by the scanner, an alarm step of turning on or flashing the alarm lamp is performed. The operation method of the self-POS device according to claim 6, further comprising:   The operation stopping step of stopping the reading operation of the product data of the product by another device operated by a store clerk in the store when the alarm is generated by the alarm step. Method of self-POS device.   The operation method of the self-POS device according to claim 8, further comprising an operation cancellation release step of canceling cancellation of the reading operation of the product by another device operated by a store clerk in the store.

Images