KR102655145B1 - Unmanned payment system - Google Patents

Abstract

This specification relates to an unmanned payment system. An unmanned payment system according to an embodiment includes a product scanning unit for inputting identification information related to the product; To output an image including distance information by photographing a shelf consisting of a first area where items are placed before inputting identification information, a second area where identification information is input, and a third area where items are placed after inputting identification information. 3D camera; a communication unit for transmitting the identification information output by the product scanning unit to a server and receiving product information including the price and name of the product; and calculating the occupied volume of one or more items placed in the first to third areas of the shelf based on the image, and identifying the change in the occupied volume of the one or more items in the first to third areas and the item through the article scanning unit. It may be configured to include a control unit to determine whether there is an abnormality in scanning the product based on the input.

Description

Unmanned payment system {Unmanned payment system}

This specification relates to an unmanned payment system, and more specifically, to a device and method for preventing loss of goods in an unmanned payment system.

Recently, unmanned payment counters are being installed in large supermarkets to reduce payment waiting lines and improve customer convenience. In addition, the number of unmanned stores that operate only with an unmanned payment system without a manager is increasing.

While this unmanned payment system provides convenience to users, there is a lot of room for loss due to buyers missing part of the product. Efforts are underway to eliminate or reduce these losses. For example, systems that link the weight of an item or a system that uses images of the item are being attempted.

However, the system that uses the weight of the product has difficulties in that a scale device for measuring weight must be installed on the shelf where the product is placed before purchase and the shelf where the product is placed after purchase, the problem of having to measure the weight of the product in advance and store it, and the weight range of the product. Because it is wide, it is not easy to measure weight accurately. In order to measure weight accurately, there are problems such as increasing the price of the weighing device, having to calibrate the weighing device periodically, and easily breaking down. In addition, there is a problem that it cannot be applied to products whose weight is not constant for each product, such as agricultural and marine products or foodstuffs.

Additionally, in a system that uses a camera to capture images of an article, in order to recognize the article well, images must be taken in advance for each article from various directions and the captured images must be trained as a learning model. However, depending on the lighting or shooting angle, the item may not be recognized properly or it takes a long time to recognize the item, making it difficult to properly identify the item in a timely manner. Additionally, as the number of items increases, training a learning model for each item becomes expensive and inefficient. Since it is not valid from the perspective, the method using the camera is not actually applied to the unmanned payment system.

In addition, it is possible to identify loss by recording video and analyzing the recorded video afterwards. However, in this case, the loss can only be confirmed after the loss has occurred, and there is also the problem that it takes a lot of time and money to analyze the recorded video. there is.

This specification takes this situation into consideration, and the purpose of this specification is to provide a method to effectively identify and prevent losses in an unmanned payment system.

An unmanned payment system according to an embodiment of the present invention for realizing the above-described problem includes an article scanning unit for inputting identification information related to the article; To output an image including distance information by photographing a shelf consisting of a first area where items are placed before inputting identification information, a second area where identification information is input, and a third area where items are placed after inputting identification information. 3D camera; a communication unit for transmitting the identification information output by the product scanning unit to a server and receiving product information including the price and name of the product; and calculating the occupied volume of one or more items placed in the first to third areas of the shelf based on the image, and identifying the change in the occupied volume of the one or more items in the first to third areas and the item through the article scanning unit. It is characterized by including a control unit to determine whether there is an abnormality in the scanning of the product based on the input.

The unmanned payment system according to another embodiment of the present invention includes a shelf consisting of a first area where goods before inputting identification information are placed, a second area where identification information is entered, and a third area where goods are placed after inputting identification information. Analyzing an image captured and output to include distance information; calculating the occupied volume of the article in the first to third areas through image analysis; and when a change occurs in the occupied volume of the article in at least one of the first to third areas, determining whether there is an abnormality in the scanning of the article based on the change in occupied volume and input of identification information for the article through the article scanning unit. It is characterized by comprising:

Therefore, in the unmanned payment system, it is easy and convenient to detect when an item is being taken out without scanning for payment, thereby reducing sales loss.

In addition, it does not require separate prior learning for item recognition, so it can be applied directly to actual sites.

Additionally, compared to methods that use camera images and train a learning model, the system can be implemented with a higher recognition rate at a lower cost.

Figure 1 shows the area of the shelf that constitutes the unmanned payment system,
Figure 2 shows the buyer's actions at each stage of scanning and paying for items in the unmanned payment system,
Figures 3A to 3C show the process by which the buyer moves the goods from the storage area (Z1) before payment through the scanning area (Z2) to the storage area (Z3) after scanning,
Figure 4 shows the configuration of the unmanned payment system according to this specification in functional blocks,
Figure 5 shows the specific configuration of the image analysis unit of the unmanned payment system in functional blocks,
Figure 6 is a table that classifies events that occur in the process of a buyer paying for a product and explains in detail the conditions for occurrence of each event;
Figure 7 expresses the occurrence of each event described in Figure 6 as a change in space occupancy in each area of the shelf constituting the unmanned payment system,
Figures 8a to 8c show images taken of a shelf using an actual 3D depth camera, separated by each action that constitutes an event related to unmanned payment,
Figure 9 shows an operation flow chart for the unmanned payment method according to this specification,
Figure 10 shows an operation flowchart of a method for determining whether an item is scanned abnormally.

Hereinafter, an embodiment of the unmanned payment system will be described in detail based on the attached drawings.

Figure 1 shows the area of the shelf that constitutes the unmanned payment system, Figure 2 shows the buyer's actions at each stage of scanning and paying for the product in the unmanned payment system, and Figures 3A to 3C show the buyer's purchase of the product. This shows the process of moving from the pre-calculation stacking area (Z1) through the scan area (Z2) to the stacking area (Z3) after scanning.

The payment system for the buyer to pay for the purchased product by himself or herself includes a first area (Z1) where the purchased product is placed, and a second area where the identification information of the purchased product, that is, the barcode or QR code printed on the packaging of the product is scanned. A shelf consisting of (Z2) and a third area (Z3) where items with scanned identification information are placed until payment is completed, and a scanner (SCANNER) for scanning items in the second area (Z2) is included as a buyer interface. do.

Additionally, the payment system may further include a display for displaying the name and price of the product whose identification information has been scanned and a payment method for paying for the scanned product.

The buyer accesses the system (entrance) to pay for the items purchased at the store through an unmanned payment system, and stacks the items in the cart or basket for purchase in the first area (Z1) of the shelf, as shown in Figure 3a. (Loading), as shown in Figure 3b, the goods piled up in the first area (Z1) are moved one by one to the second area (Z2) and the barcode printed on the packaging of the goods is scanned with a scanner, and the scanning is performed as shown in Figure 3C. The completed items are stacked in the third area (Z3). The buyer repeats the scanning operation for each item stacked in the first area (Z1), scans all the items, and stacks them in the third area (Z3). Then, the buyer pays for the scanned items and all the items for which payment has been completed. Move it from the third area (Z3) to a basket or cart (Unloading) and then exit the system (Exit).

The buyer's fraudulent behavior that the unmanned payment system seeks to detect is moving items piled in the first area (Z1) to the third area (Z3) without scanning them in the second area (Z2). That is, the buyer does not scan the items in the first area (Z1) at all and moves them directly to the third area (Z3), or moves two or more items in the first area (Z1) at the same time and does not scan more than one item. You can cheat by moving two or more objects to the third area (Z3) without doing so.

As explained earlier, both systems that use the weight of the item and systems that capture the image of the item with a camera have various problems, so they cannot be applied to actual systems. In particular, there have been ongoing attempts to learn and use deep learning models with product images in recent years, but the product recognition rate is low and the recognition speed is low in marts with many types of products and a steady flow of new products. In order to increase the cost, expensive cameras, servers, etc. must be used, which greatly increases the cost, so it has not been released as an actual product.

General RBG cameras capture two-dimensional image data at high resolution and input it into an already learned artificial intelligence model, so a large amount of image data must be processed, putting a lot of load on the system.

On the other hand, TOF cameras or 3D stereo cameras have lower resolution than regular RGB cameras, but they can obtain distance information to each point in the image, so they can use the distance information of consecutive points to identify the face, corner, vertex, etc. of the item. Features related to the shape of can be extracted, and based on this, the volume occupied by the item can be calculated or estimated.

The unmanned payment system of this specification calculates the volume of the product using the characteristics of the 3D camera, and synchronizes the difference between the volume change of the product before scanning and the volume change of the product after scanning and the scanning operation, without scanning the product. It is possible to detect fraudulent activity by buyers taking out goods.

The unmanned payment system manages the process of a buyer moving items from the first area (Z1) and stacking them in the third area (Z3) as one event, and synchronizes the event with the scanning operation in the second area (Z2). It is possible to determine whether the buyer has committed fraud.

Figure 4 shows the configuration of the unmanned payment system according to this specification in functional blocks, and Figure 5 shows the specific configuration of the image analysis unit of the unmanned payment system in functional blocks.

The unmanned payment system 1 has an input unit 10 for photographing an item as an image including distance information, scanning the bar code of the item, and receiving payment information, and analyzing the image for the item captured by the input unit 10. A control unit 20 calculates the occupied volume of the product, detects changes in the volume of the product on the shelf, and links this with barcode scanning to determine theft of the product due to fraudulent activity. The name and price of the scanned product and the payment method are provided. An output unit 30 for outputting the prices of all items on screen or in sound, and a communication unit for receiving information about payment authentication by inquiring and receiving the names and prices of scanned items from the server and transmitting payment information to the payment server. It may be configured including (40).

The input unit 10 includes a camera 110 for photographing items placed on a shelf in 3D, an item scanning unit 120 for scanning identification information to identify items, and a payment function for paying for purchased items. It may be configured to include a payment information input unit 130 for inputting information. The input unit 10 may further include a 2D camera for filming the actions of a buyer who uses the unmanned payment system 1 to pay for purchased goods.

The camera 110 may be a 3D depth camera that provides distance information to each point constituting the image along with two-dimensional image data, and may be a 3D stereo camera or a TOF camera.

The product scanning unit 120 is a device for reading identification information or codes printed on product packaging or products, such as barcodes, QR codes, data metrics, etc., in the form of a code scanned by a buyer holding it in his hand or on a shelf. It may be implemented as installed on the floor of the second area (Z2). Alternatively, the product scanning unit 120 may use an RF signal. If the product emits an RF signal and outputs an RF signal as a reflected signal in response to the emitted RF signal, this may be detected as identification information for the product. .

The control unit 20 analyzes images taken by the camera 110 of the first to third areas (Z1, Z2, Z3) of the shelf, calculates the occupied volume occupied by the item placed on the shelf, and calculates the volume occupied by the item as it moves. The image analysis unit 210 for detecting changes and the volume change of the article analyzed by the image analysis unit 210 are linked to the article scanning unit 120 so that the article moves from the first area Z1 of the shelf to the third area without scanning. It may include a scan abnormality determination unit 220 that determines whether the product is moved directly to the area Z3 or to the buyer's basket or cart.

The output unit 30 has a display ( 310) and a speaker 320 for outputting a sound corresponding to the scanning of an article and a sound warning when an abnormality occurs in the scanning of the article.

The image analysis unit 210 uses images including distance information captured and output by the camera 110 to determine each area constituting the shelf, that is, the first to third areas Z1, Z2, and Z3, respectively. A volume calculation unit 211 that calculates the occupied volume occupied by the article, and a movement that detects movement occurring in the first to third areas (Z1, Z2, Z3) using two or more images output by the camera 110 When movement is detected by the detection unit 212 and the motion detection unit 212, the change in the occupied volume of the article in each area is calculated based on the occupied volume of the article in each area output by the volume calculation unit 211, and the It may include an event generator 213 for determining whether an event corresponds to movement of and generating an event.

As explained with reference to FIGS. 2 and 3, the buyer enters the unmanned payment system, places the products in the first area (Z1), and enters the product's barcode or QR in the second area (Z2) for each product. After scanning the code, perform a stacking operation in the third area (Z3) and repeat this until the last item in the first area (Z1), then the first area (Z1) becomes empty and the stacking operation is performed in the third area (Z3). When all purchased items are accumulated, payment is made using a credit card, etc., and once payment is completed, the items in the third area (Z3) are moved to the buyer's basket or cart.

The event generator 213 can generate several events according to the buyer's normal behavior pattern in the unmanned payment system. Events occurring in the unmanned payment system will be described with reference to FIGS. 6 and 7.

Figure 6 is a table that classifies events that occur in the process of a buyer paying for a product and explains in detail the conditions for occurrence of each event, and Figure 7 shows the occurrence of each event described in Figure 6 at each shelf constituting the unmanned payment system. It is expressed as a change in space occupancy in the area.

The buyer's unattended payment can begin with an appearance event (EVT-E) where the buyer appears in the unattended payment system. When the motion detection unit 212 of the image analysis unit 210 detects movement in the image captured by the camera 110 as a person appears near the shelf (entrance), the event generator 213 generates an appearance event (EVT-E ) can be created. That is, the event generator 213 generates an appearance event when movement is detected outside the shelf while the first area (Z1) and third area (Z3) of the shelf are empty and there is no change in the second area (Z2). (EVT-E) can be created.

When an appearance event (EVT-E) is generated, the motion detection unit 212 detects movement in the first area (Z1) of the shelf, and the volume calculation unit 211 determines the occupied volume occupied by the article in the first area (Z1). Calculate , but if the occupied volume increases over time, the event generator 213 generates a loading event (EVT- L) can be created. At this time, there should be no change in the second area (Z2) of the shelf and the third area (Z3) should be empty. The event generator 213 may maintain the loading event (EVT-L) until the occupied volume of the product stops increasing in the first area (Z1) of the shelf. When the loading event (EVT-L) ends, the volume calculated in the first area (Z1) corresponds to the volume of the product purchased by the buyer. In FIG. 7, the occupied volume of all goods stored in the first area Z1 corresponds to A.

If the occupied volume is calculated in the third area (Z3) of the shelf during the loading event (EVT-L), the scan abnormality determination unit 220 regards this as a scanning abnormality, that is, fraud, and issues a warning through the output unit 30. can do.

While the loading event (EVT-L) is in progress, the buyer sequentially moves the items from the basket or cart containing the items selected in the store to the first area (Z1) of the shelf, and the volume calculation unit 211 calculates the items that the buyer removes from the basket. 1 Each time an item is moved to area (Z1), the volume of that item can be calculated.

In addition, the volume calculation unit 211 considers the variation in the calculated volume in the first area (Z1) while the items are sequentially stacked in the first area (Z1) as the loading event (EVT-L) progresses, When the loading event (EVT-L) ends, the occupied volume of all items accumulated in the first area (Z1) can be calculated.

In addition, the volume calculation unit 211 calculates the occupied volume of all products piled up in the first area (Z1) when the loading event (EVT-L) ends, and calculates the occupied volume while moving the products during the loading event (EVT-L) The volume of each item can be used to correct the occupied volume of all items. Additionally, the volume calculation unit 211 may correct the occupied volume of the product by considering the volume of the buyer's hand moving the product when calculating the volume of the product while the buyer is moving the product.

After the loading event (EVT-L) ends, the buyer sequentially scans the items stacked in the first area (Z1) of the shelf through the item scanner 120 in the second area (Z2) and scans them again in the third area (Z2). Z3), the event generator 213 can scan each item and generate an operation of moving it to the third area Z3 as a scan event (EVT-S).

In a scan event (EVT-S), the volume of items piled up in the first area (Z1) is reduced by the volume of the item being scanned, and in the second area (Z2), the buyer operates the scanner 120 to scan the item. As the scanned item is moved to the third area (Z3), a change in volume occurs, and as new items are added to the third area (Z3), the volume of the item increases.

The event generator 213 calculates the volume and movement in the first and third areas Z1 and Z3 by the volume calculation unit 211 using continuous images output from the camera 110 by photographing the shelf. The detection unit 212 may determine a scan event (EVT-S) based on the movement detected in the second area (Z2) and generate a scan event (EVT-S) for the movement of each article.

In Figures 6 and 7, a Scan1 event (EVT-S1) is generated for the scan of the first item, and a ScanF event (EVT-SF) is generated for the scan of the last item. As a result, the occupied volume (A) of the article in the third zone (Z3) is equal to the occupied volume (A) of the article in the first zone (Z1) after the loading event (EVT-L) or is less than or equal to a predetermined reference value. There has to be a difference.

While the scan event (EVT-S1 to EVT-SF) is in progress, as shown in FIG. 7, the occupied volume of the article in the first area (Z1) decreases in the form of steps, and in the second area (Z2), the scanned volume decreases. During this process, the volume is sensed briefly, and in the third area (Z3), the occupied volume of the item increases in a calculated form.

While the scanning event (EVT-S1 to EVT-SF) is in progress, that is, the volume decreases in the first area (Z1) and increases in the third area (Z3), the barcode or QR code is displayed in the article scanning unit 120. If no output occurs because there is no input of identification information for the product, the scan abnormality determination unit 220 does not scan the product for payment and the space after scanning the product, that is, the third area (Z3) ), and if it is judged to be a scanning error or a fraudulent act by the purchaser, a warning message or warning sound may be output through the output unit 30.

In addition, the scan abnormality determination unit 220 determines the difference in volume fluctuations in the first and third regions (Z1, Z3) after each scan event, that is, the volume decrease in the first region (Z1) and the volume change in the third region (Z3). Even when the difference in volume increase in ) is greater than the reference value, it is determined that there is a scanning error or the purchaser's fraud, and a warning message or warning sound can be output through the output unit 30.

After the last scan event (EVT-SF) ends, the buyer makes a payment using a credit card or the NFC function of the smart phone in the payment information input unit 130, and when the payment is authenticated by the payment server, the unmanned payment system (1 ) outputs information indicating payment completion to the output unit 30 and can also print it on paper through a printer (not shown) or send the payment information as a message to the buyer's smartphone.

Afterwards, when the buyer performs an unloading operation to move the goods accumulated in the third area Z3 to a basket or cart, the event generator 213 generates an unloading event (EVT-U). The event generator 213 determines from the volume calculation unit 211 and the motion detection unit 212 that there is no occupied volume in the first area (Z1), there is no change in the second area (Z2), and the third area (Z3) An unloading event (EVT-U) is generated after confirming that the occupied volume of is reduced, and the unloading event (EVT-U) can be maintained until the occupied volume disappears in the third area (Z3).

Afterwards, the event generator 213 generates an exit event (EVT-X) when the motion detector 212 detects that the buyer disappears (Exit) from the image output by the camera 110, The payment process for the product by the buyer can be completed.

Figures 8a to 8c show images taken of a shelf using an actual 3D depth camera, separated by each action that constitutes an event related to unmanned payment.

In Figure 8a, the buyer places two items in the first area (Z1), and in Figure 8b, the buyer moves one of the items in the first area (Z1) to the second area (Z2) and performs a barcode scanning operation. And in Figure 8c, the buyer is moving the scanned item to the third area (Z3).

The 3D depth camera 110 provides distance information to each point that makes up the captured image. In FIGS. 8A to 8C, the camera 110 reconstructs and displays the captured image, but does not display distance-related information in the image for a fixed background, and displays distances in the form of contour lines or grids only for objects separated from the background. Information is displayed.

The image analysis unit 210 uses distance information about an object separated from the background in the image output by the 3D depth camera 110 to extract features related to the shape of the object and use this to calculate the volume of the object. You can.

Figure 9 shows an operation flow chart for the unmanned payment method according to this specification.

When the camera 110 outputs an image taken of the shelf of the unmanned payment system, the image analysis unit 210 of the control unit 20 (later, considering convenience, the image analysis unit 210 or the scan abnormality determination unit 220) The operation (described as the operation of the control unit 20) analyzes images of the first to third areas (Z1, Z2, and Z3) of the shelf (S910).

The control unit 20 calculates the occupied volume of the article in the first to third areas Z1, Z2, and Z3 using the distance information included in the image output by the camera 110 (S920).

The control unit 20 checks whether there is a change in the occupied volume in the first to third areas Z1, Z2, and Z3 (S930).

If there is a change in occupancy volume in the first to third areas (Z1, Z2, Z3) (YES in S930), the control unit 20 generates or sets an event corresponding to the change in occupancy in each area (S940). , As described with reference to FIGS. 6 and 7, an appearance event (EVT-E), a loading event (EVT-L), and a scan event ( EVT-S), an unloading event (EVT-U), and an exit event (EVT-X).

The control unit 20 determines whether there is a problem with the product scanning by linking the set or generated event with the product scanning operation in the product scanning unit 120, and when there is a problem with the product scanning, the control unit 20 sends a message through the output unit 30. You can notify (S950).

The control unit 20 checks whether the occupied volume of the first area Z1 is 0 and determines whether the product scanning is complete (S960). If it determines that the product scanning is completed (YES in S960), it guides the buyer to proceed with payment. You can do it (S970).

FIG. 10 shows an operation flowchart of a method for determining whether there is an item scanning abnormality, and FIG. 10 explains the operation for a scanning abnormality that occurs during a scan event (EVT-S).

The control unit 20 checks whether there was an operation to scan the identification information of the product through the product scanning unit 120 while the scan event (EVT-S) was in progress (S1010).

A scan event (EVT-S) is set or generated when the occupied volume of the first area (Z1) decreases and the occupied volume of the third area (Z3) increases, and the item is scanned at the time of the scan event (EVT-S). If it does not occur (NO in S1010), the control unit 20 determines that there is a product scanning error or the purchaser's fraudulent activity and outputs a warning message or warning sound through the output unit 30 (S1040).

Even when scanning of an article occurs during a scan event (EVT-S) (YES in S1010), the control unit 20 determines the difference in volume changes in the first and third areas (Z1, Z3) after the scan event, that is, the first It is checked whether the difference between the volume decrease in the first region (Z1) and the volume increase in the third region (Z3) is greater than or equal to the reference value (S1020).

When the difference in volume fluctuations in the first and third areas (Z1, Z3) is greater than the reference value (YES in S1020), the control unit 20 determines that there is a product scanning abnormality or the purchaser's misconduct, and sends a message or warning sound to warn of this. Can be output through the output unit 30 (S1040).

When the difference in volume variation in the first and third areas Z1 and Z3 is smaller than the reference value (NO in S1020), the control unit 20 determines that the item has been scanned normally and can proceed to the next event.

In this way, by applying a 3D camera to the unmanned payment system, the occupied volume of the product is calculated, and by linking the change in occupied volume and the scanning operation, it is possible to effectively detect fraudulent behavior in which buyers take out the product without scanning the product.

It can be implemented at a lower cost than a system that uses a 2D camera and applies a learning model, and it can be applied right away in the field because there is no need for prior learning for each item.

The unmanned payment system and unmanned payment method described in this specification can be described as follows.

An unmanned payment system according to an embodiment includes a product scanning unit for inputting identification information related to the product; To output an image including distance information by photographing a shelf consisting of a first area where items are placed before inputting identification information, a second area where identification information is input, and a third area where items are placed after inputting identification information. 3D camera; a communication unit for transmitting the identification information output by the product scanning unit to a server and receiving product information including the price and name of the product; and calculating the occupied volume of one or more items placed in the first to third areas of the shelf based on the image, and identifying the change in the occupied volume of the one or more items in the first to third areas and the item through the article scanning unit. It may be configured to include a control unit to determine whether there is an abnormality in scanning the product based on the input.

In one embodiment, the control unit generates a loading event when the first area is filled with an empty space and the second area contains no identification information and the third area is an empty space, and loads the item in the first area. A scan event is generated when the occupied volume decreases, a volume change occurs in the second area, and the occupied volume of the article increases in the third area, and the first area is empty, there is no volume change in the second area, and the third area is empty. The situation in which the goods in is emptied can be created as an unloading event.

In one embodiment, the control unit may determine that there is an error in scanning the item if no identification information is input in the scan event.

In one embodiment, the control unit may determine that there is an error in scanning an item when the difference between the change in occupied volume in the first area and the change in occupied volume in the third area in the scan event is greater than or equal to a reference value.

In one embodiment, the controller may calculate the occupied volume of all items purchased by the buyer based on changes in images captured in the first area while a loading event is in progress.

In one embodiment, the control unit calculates the occupied volume of each item sequentially moving to the first area while the loading event is in progress, and calculates the total product volume based on images taken in the first area after the loading event ends. The occupied volume of can be corrected using the calculated sum of the occupied volumes of each article.

In one embodiment, identification information may be detected from one of a barcode or QR code displayed on the surface of the article and an RF signal output by the article in response to an RF signal emitted from the article scanning unit.

In one embodiment, a payment information input unit for inputting payment information for paying for purchased goods; and an output unit for displaying the price and product information for the product for which the purchaser inputs identification information through the product scanning unit and outputting a warning on screen or in sound indicating an abnormality in the product scan transmitted from the control unit, and the communication department pays. Payment information input through the information input unit may be transmitted to the payment server, and payment confirmation information may be received from the payment server in response to the payment information.

An unmanned payment method according to another embodiment involves photographing a shelf consisting of a first area where items before inputting identification information are placed, a second area where identification information is input, and a third area where items are placed after inputting identification information. Analyzing the output image to include information; calculating the occupied volume of the article in the first to third areas through image analysis; and when a change occurs in the occupied volume of the article in at least one of the first to third areas, determining whether there is an abnormality in the scanning of the article based on the change in occupied volume and input of identification information for the article through the article scanning unit. It can be done including.

In one embodiment, the unmanned payment method generates a loading event when the first area is filled with items in an empty space, there is no input of identification information in the second area, and the third area is an empty space, and a loading event is generated in the first area. A scan event is generated when the occupied volume of the article decreases, a volume change occurs in the second area, and the occupied volume of the article increases in the third area, and the first area is empty, there is no volume change in the second area, and the occupied volume of the article increases in the third area. The process may further include generating an unloading event when an item in area 3 is emptied.

In one embodiment, the determining step may determine that there is an error in scanning the item if no identification information is input in the scan event.

In one embodiment, the determining step may determine that there is an abnormality in scanning the item when the difference between the change in occupied volume in the first area and the change in occupied volume in the third area in the scan event is greater than or equal to a reference value.

In one embodiment, the calculating step may calculate the occupied volume of all items purchased by the buyer based on changes in images captured in the first area while a loading event is in progress.

In one embodiment, the calculating step calculates the occupied volume of each item sequentially moving to the first area while the loading event is in progress, and calculates the occupied volume based on the image taken in the first area after the loading event ends. The occupied volume of all products can be corrected using the calculated sum of the occupied volumes of each product.

In one embodiment, the unmanned payment method includes displaying the price and product information for the product for which identification information is input through the product scanning unit during a scan event; A step of outputting this on screen or in sound when it is determined that an abnormality has occurred in scanning the product; And when the first area changes to a blank area without any abnormality occurring in scanning the product, the step of proceeding with payment may be further included.

The present invention is not limited to the described embodiments, and it is obvious to those skilled in the art that various modifications and changes can be made without departing from the spirit and scope of the present invention. Accordingly, such modifications or variations should be considered to fall within the scope of the claims of the present invention.

1: Unmanned payment system 10: Input unit
20: control unit 30: output unit
40: Communication unit 110: Camera
120: product scanning unit 130: payment information input unit
210: Image analysis unit 211: Volume calculation unit
212: motion detection unit 213: event generation unit
220: Scan abnormality determination unit 310: Display
320: speaker

Claims (15)

an article scanning unit for inputting identification information related to the article;
An image containing distance information is created by photographing a shelf consisting of a first area where the goods before the input of the identification information are placed, a second area where the identification information is entered, and a third area where the goods are placed after the input of the identification information. 3D camera for printing;
a communication unit for transmitting the identification information output by the product scanning unit to a server and receiving product information including the price and name of the product; and
Calculate the occupied volume of one or more items placed in first to third areas of the shelf based on the image, and calculate the change in occupied volume of the one or more items in the first to third areas and the product through the item scanning unit. It is configured to include a control unit for determining whether there is an abnormality in scanning the article based on input of identification information for,
The control unit generates a loading event when the first area is filled with an empty space and the identification information is not input in the second area and the third area is an empty space, and the product is stored in the first area. A scan event is generated in which the occupied volume of the article decreases, a volume change occurs in the second area, and the occupied volume of the article increases in the third area, and the first area is an empty space and the volume change occurs in the second area. An unloading event is generated in which the goods in the third area are emptied, and the scan event is based on the difference between the change in occupied volume in the first area and the change in occupied volume in the third area. An unmanned payment system characterized in that it is determined that there is an error in scanning the item when the value is greater than or equal to the value. delete According to claim 1,
The control unit determines that there is a problem in scanning the product if the identification information is not input in the scan event. delete According to claim 1,
The control unit calculates the occupied volume of all items purchased by the buyer based on changes in images captured in the first area while the loading event is in progress. According to clause 5,
The control unit calculates the occupied volume of each article sequentially moving to the first area while the loading event is in progress, and the total volume calculated based on the image taken in the first area after the loading event ends. An unmanned payment system characterized in that the occupied volume of an article is corrected using the calculated sum of the occupied volume of each article. According to claim 1,
The identification information is an unmanned payment system characterized in that it is detected from one of a barcode or QR code displayed on the surface of the product and an RF signal output by the product in response to an RF signal emitted from the product scanning unit. According to claim 1,
A payment information input unit for entering payment information for paying for purchased goods; and
It further includes an output unit for displaying the price and product information for the product for which the buyer has entered identification information through the product scanning unit and outputting a warning on screen or in sound indicating an abnormality in the product scan transmitted from the control unit,
The communication unit transmits payment information input through the payment information input unit to a payment server and receives payment confirmation information corresponding to the payment information from the payment server. In an unmanned payment system including an item identification unit, a 3D camera, and a control unit,
The 3D camera photographs a shelf consisting of a first area where the product is placed before inputting the identification information through the product identification unit, a second area where the identification information is input, and a third area where the product is placed after the input of the identification information. The control unit analyzing the output image to include distance information;
The control unit generates a loading event in a situation where the first area is filled with an empty space and the identification information is not input in the second area and the third area is an empty space, and the first area is filled with goods. A scan event is generated when the occupied volume decreases, a volume change occurs in the second area, and the occupied volume of the article increases in the third area, and the first area is an empty space and a volume change occurs in the second area. generating an unloading event in a situation in which the goods in the third area are empty;
The control unit calculating an occupied volume of an article in the first to third areas through the image analysis; and
When a change occurs in the occupied volume of an article in at least one of the first to third areas, the control unit detects an abnormality in scanning the article based on the change in the occupied volume and input of identification information for the article through the article scanning unit. It includes the step of determining whether or not
The determining step is characterized in that it is determined that there is an abnormality in scanning the article when the difference between the change in the occupied volume in the first area and the change in the occupied volume in the third area in the scan event is greater than a reference value. Unmanned payment method. delete According to clause 9,
The determining step is an unmanned payment method characterized in that it is determined that there is a problem in scanning the item if the identification information is not input in the scan event. delete According to clause 9,
The calculating step is an unmanned payment method characterized in that the occupied volume of all items purchased by the buyer is calculated based on changes in images captured in the first area while the loading event is in progress. According to claim 13,
The calculating step calculates the occupied volume of each item sequentially moving to the first area while the loading event is in progress, and calculates the occupied volume based on the image taken in the first area after the loading event ends. An unmanned payment method characterized in that the occupied volume of all products is corrected using the calculated sum of occupied volumes of each product. According to clause 9,
displaying price and product information for the product for which identification information is input through the product scanning unit during the scan event;
When it is determined that an abnormality has occurred in the scanning of the product, outputting this on a screen or in sound; and
An unmanned payment method, characterized in that it further includes the step of proceeding with payment when the first area changes to a blank area without any abnormality occurring in scanning the product.

Images