WO2017126245A1

WO2017126245A1 - Information processing device, control method, and program

Info

Publication number: WO2017126245A1
Application number: PCT/JP2016/086487
Authority: WO
Inventors: 金野　大; 岩元　浩太; 松本　信一; 善弘伊藤; 晋哉山崎; 可菜子多賀
Original assignee: 日本電気株式会社
Priority date: 2016-01-21
Filing date: 2016-12-08
Publication date: 2017-07-27

Abstract

A product registration device (20) is provided with a detection unit (22) and a registration unit (24). The detection unit (22) detects a prescribed motion from a video. In cases when the prescribed motion is detected by the detection unit (22), the registration unit (24) registers, as objects to be paid for, products included in the product video.

Description

Information processing apparatus, control method, and program

The present invention relates to an apparatus used for settlement of goods.

In a product registration device (for example, a POS (Point of Sales) register) in a store such as a supermarket, a process of registering a product as a payment target is performed. For example, such processing is performed by causing a barcode reader or the like provided in the product registration apparatus to read a barcode or the like attached to the product.

・ Technology for efficient operation of product registration devices has been developed. For example, Patent Document 1 discloses a technique of registering a plurality of products as a settlement target at once by capturing a plurality of products placed on a mounting table from a plurality of different directions with a plurality of imaging devices.

JP 2013-054673 A

In the invention of Patent Document 1, when an operator instructs to register a product placed on a mounting table as a payment target, the operator needs to operate a touch panel sensor. Therefore, the operator has to move the hand to the touch panel sensor after placing the product on the mounting table, and the work for registering the product as a payment target is troublesome.

The present invention has been made in view of the above problems. The objective of this invention is providing the technique which reduces the troublesomeness of the operation | work which registers goods as a payment object.

The information processing apparatus according to the present invention includes: 1) a detecting unit that detects a predetermined operation from a video; and 2) a product included in the video is registered as a settlement target when the predetermined unit detects the predetermined operation. And registration means.

The control method of the present invention is executed by a computer. The control method includes: 1) a detection step for detecting a predetermined operation from the video generated by the imaging device; and 2) a product included in the video when the predetermined operation is detected by the detection step. And a registration step of registering as

The program of the present invention causes a computer to execute each step of the control method of the present invention.

According to the present invention, there is provided a technique for reducing the troublesome work of registering a product as a payment target.

The above-described object and other objects, features, and advantages will be further clarified by a preferred embodiment described below and the following drawings attached thereto.

It is a block diagram which illustrates the goods registration device concerning Embodiment 1. It is a figure which illustrates the hardware constitutions of the computer which implement | achieves the goods registration apparatus of Embodiment 1. It is a figure which illustrates the installation place of an imaging device. 3 is a flowchart illustrating a flow of processing executed by the product registration device according to the first embodiment. It is a figure which illustrates the goods image | video including the operation | movement which makes a predetermined shape using a user's body. It is a figure which illustrates the goods picture including the operation which draws a predetermined locus using a user's body. It is a figure which illustrates the goods picture including the operation which takes out goods from a shopping cart. It is a figure which illustrates the goods picture including the operation which puts goods into a shopping bag. It is a figure which illustrates payment information in a table format. It is a figure which illustrates the case which makes the predetermined area | region the predetermined area in a mounting base. It is a figure which illustrates the predetermined field dynamically computed by the registration part. 10 is a flowchart illustrating a flow of processing performed by the product registration device according to the second embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, the same reference numerals are given to the same components, and the description will be omitted as appropriate. In each block diagram, unless otherwise specified, each block represents a functional unit configuration, not a hardware unit configuration.

[Embodiment 1]
FIG. 1 is a block diagram illustrating a product registration device 20 according to the first embodiment. The product registration device 20 is used for processing for registering a product as a payment target (hereinafter, product registration processing). A user (such as a clerk or a customer) of the product registration device 20 operates the product registration device 20 to perform product registration processing for each product purchased by the customer.

Therefore, the product registration device 20 includes a detection unit 22 and a registration unit 24. The detection unit 22 detects a predetermined operation from an image generated by the imaging device (hereinafter, product image). The registration unit 24 registers a product included in the product video as a settlement target when a predetermined operation is detected by the detection unit 22.

According to the product registration device 20 of the present embodiment, since a predetermined operation for causing the product registration device 20 to perform product registration processing is detected from the product video, a user (a store clerk, a customer, or the like) of the product registration device 20 By performing a predetermined operation within the imaging range of the imaging device, the product registration device 20 can perform product registration processing. Therefore, the user can cause the product registration apparatus 20 to perform product registration processing without separately operating a touch panel or the like. This makes it easier to register a product as a payment target than when a product cannot be registered as a payment target unless the touch panel or the like is operated after the product is placed on a mounting table or the like. Therefore, according to the product registration device 20 of the present embodiment, bothering the work for registering a product as a payment target is reduced and the work efficiency is increased.

Hereinafter, further details of this embodiment will be described.

<Example of hardware for realizing the product registration device 20>
Each functional component of the product registration device 20 may be realized by hardware (for example, a hard-wired electronic circuit) that realizes each functional component, or a combination of hardware and software (for example: It may be realized by a combination of an electronic circuit and a program for controlling it).

FIG. 2 is a diagram illustrating a hardware configuration of a computer 200 that implements the product registration apparatus 20 according to the first embodiment. The computer 200 may be mounted using a dedicated device designed for a cash register terminal, or may be mounted using a general-purpose device such as a PC (Personal Computer) or a portable terminal.

The computer 200 includes a bus 202, a processor 204, a memory 206, a storage 208, an input / output interface 210, and a network interface 212. The bus 202 is a data transmission path for the processor 204, the memory 206, the storage 208, the input / output interface 210, and the network interface 212 to transmit / receive data to / from each other. However, the method of connecting the processors 204 and the like is not limited to bus connection. The processor 204 is an arithmetic processing unit such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The memory 206 is a memory such as a RAM (Random Access Memory) or a ROM (Read Only Memory). The storage 208 is a storage device such as a hard disk, an SSD (Solid State Drive), or a memory card. Further, the storage 208 may be a memory such as a RAM or a “ROM”.

The input / output interface 210 is an interface for connecting the computer 200 and the input / output device. For example, the input / output interface 210 is connected with a touch panel 214, a drawer 216, a change machine 218, a receipt printer 220, and the like. The touch panel 214 is used, for example, for manually inputting the amount of goods. In addition, the touch panel 214 displays various information such as information related to a product registered as a payment target. The drawer 216 stores coins and banknotes. The change machine 218 has an input port for inputting a price and an output port for discharging the change. The receipt printer 220 issues a receipt or the like. The computer 200 may be connected to an input interface other than the touch panel 214 such as a keyboard.

The network interface 212 is an interface for connecting the computer 200 to an external device so as to be communicable. The network interface 212 may be a network interface for connecting to a wired line or a network interface for connecting to a wireless line. For example, the computer 200 that implements the merchandise registration apparatus 20 is connected to a store server that manages various devices installed in the store via a network.

The storage 208 stores program modules that realize each function of the product registration apparatus 20. The processor 204 implements each function corresponding to the program module by executing each program module. Here, when executing the above-described modules, the processor 204 may execute the modules after reading them onto the memory 206 or without reading them onto the memory 206.

The hardware configuration of the computer 200 is not limited to the configuration shown in FIG. For example, each program module may be stored in the memory 206. In this case, the computer 200 may not include the storage 208.

Further, the computer 200 does not necessarily need to be communicably connected to other devices via a network. Therefore, the computer 200 may not have the network interface 212.

If the computer 200 is not used for payment processing, the computer 200 may not include the drawer 216, the change machine 218, and the receipt printer 220.

<< Details of Imaging Device 30 >>
The imaging device 30 is an arbitrary imaging device that can capture a surrounding and generate a moving image. The imaging device 30 may be provided in a housing dedicated to the imaging device, or may be provided integrally with another device (such as a camera incorporated in a smartphone).

The installation location of the imaging device 30 may be an arbitrary location where the product and user can be imaged. For example, the imaging device 30 is fixed at a position where a store clerk looks down directly on a mounting table used when registering a product to be settled. FIG. 3 is a diagram illustrating an installation location of the imaging device 30. In FIG. 3, the imaging device 30 looks down on the mounting table 40 on which the product is placed. However, the installation position of the imaging device 30 is not limited to the position shown in FIG.

<Process flow>
FIG. 4 is a flowchart illustrating the flow of processing executed by the product registration apparatus 20 according to the first embodiment. The detection unit 22 determines whether or not a predetermined operation is included in the product video (S102). When the predetermined operation is included in the product video (S102: YES), the registration unit 24 registers the product included in the product video as a settlement target (S104).

<Product video acquisition method>
The method by which the product registration device 20 acquires the product video is arbitrary. For example, the product registration device 20 acquires a product video from the imaging device 30. In this case, the product registration device 20 is connected to the imaging device 30 so as to be communicable.

For example, when the imaging device 30 stores the product video in an external storage device, the product registration device 20 may acquire the product video from the storage device. In this case, the merchandise registration apparatus 20 is communicably connected to this storage device.

The merchandise registration apparatus 20 may acquire product videos at all times, or may acquire only for a specific period. In the latter case, for example, the product registration device 20 starts acquiring the product video at the timing of starting the product registration process, and ends the acquisition of the product video at the timing of ending the product registration process. The product registration process starts or ends when the user performs a predetermined input operation on the product registration device 20, for example.

<About the specified operation>
The detection unit 22 detects a predetermined operation from the product video (S102). Various operations can be set as the predetermined operation to be detected by the detection unit 22. Hereinafter, a predetermined operation and a method for detecting the predetermined operation will be exemplified.

<< predetermined operation 1 >>
For example, the predetermined operation is an operation of creating a predetermined shape using the user's body. FIG. 5 is a diagram illustrating a product image 32 including an operation of creating a predetermined shape using the user's body.

In the example of FIG. 5, the user first places a product to be settled on the mounting table 40 (FIG. 5A). When the product is placed on the mounting table 40, the user performs a predetermined operation 70 (FIG. 5B). In FIG.5 (b), the user is carrying out the operation | movement which makes a circle mark by hand. The detection unit 22 detects a circle created by the user's hand. Then, the registration unit 24 registers the product 50-1 and the product 50-2 included in the product video 32 as a settlement target.

Note that the shape made using the user's body is not limited to a circle. Any shape can be adopted as the shape to be detected by the detection unit 22.

Also, various existing image processing techniques can be used for the process of detecting a predetermined shape created by the user's body from the video. Therefore, a specific description of this process is omitted.

<< predetermined operation 2 >>
For example, the predetermined operation is an operation of drawing a predetermined locus using the user's body. FIG. 6 is a diagram illustrating a product image 32 including an operation of drawing a predetermined trajectory using the user's body.

In the example of FIG. 6, as in the example of FIG. 6, the product to be settled is placed on the mounting table 40 (FIG. 6A). Thereafter, the user performs a predetermined operation 70 (FIG. 6B). In FIG. 6B, the user moves the hand to draw a circular locus. The detection unit 22 detects that a circular locus is drawn by the user's hand. Then, the registration unit 24 registers the product 50-1 and the product 50-2 included in the product video 32 as a settlement target.

Note that the trajectory drawn using the user's body is not limited to a circle. An arbitrary trajectory can be adopted as the trajectory to be detected by the detection unit 22.

Also, various existing image processing techniques can be used for the process of detecting from a video that a predetermined locus has been drawn by the user's body. Therefore, a specific description of this process is omitted.

<< predetermined operation 3 >>
Further, for example, the predetermined operation is an operation in which the user takes out a product from the shopping cart. Here, the shopping cart refers to various containers in which products before being registered as a payment target are put.

FIG. 7 is a diagram exemplifying a product video 32 including an operation for taking out a product from the shopping cart. In FIG. 7A, the product 50-1 and the product 50-2 are in the shopping cart 80. In FIG. 7A, the user has a product 50-1. Next, in FIG. 7B, the user takes out the product 50-1 held in the hand from the shopping cart 80. The detection unit 22 detects a series of operations included in the product video 32 in FIGS. 7A and 7B as a predetermined operation 70. Then, the registration unit 24 registers the product 50-1 that is the target of the predetermined operation 70 as a settlement target.

The detection unit 22 analyzes the image of the product video 32 as follows, for example, and detects that the product 50 is taken out from the shopping cart 80. First, the detection unit 22 detects an area representing the shopping cart 80 from each frame of the product video 32. Furthermore, the detection part 22 detects the goods in the area | region showing the shopping cart 80 from each flame | frame in which the shopping cart 80 was detected. The detection unit 22 determines that the product has been taken out from the shopping cart 80 when the product in the region representing the shopping cart 80 moves out of the region representing the shopping cart 80.

In addition, the existing object recognition process can be used for the process of detecting the shopping cart 80 or the product 50 from the image and the process of specifying the position thereof. Therefore, a specific description of these processes is omitted.

<< predetermined operation 4 >>
Further, for example, the predetermined operation is an operation in which the user puts the product into a predetermined container. Here, the predetermined containers are various containers for storing products registered as a payment target. For example, the predetermined container is a shopping bag in which a product to be taken home by the customer is put.

FIG. 8 is a diagram illustrating a product video 32 including an operation of putting a product into a shopping bag. In FIG. 8A, the product 50-1 is held by the user, and the product 50-2 is placed on the mounting table 40. The product 50-1 has not been put in the shopping bag 100 yet. Next, in FIG. 8B, the user puts the product 50-1 held in the hand into the shopping bag 100. The detection unit 22 detects a series of operations included in the product video 32 in FIGS. 8A and 8B as a predetermined operation 70. Then, the registration unit 24 registers the product 50-1 that is the target of the predetermined operation 70 as a settlement target.

The detection unit 22 analyzes the image of the product video 32 as follows, for example, and detects that the product 50 is put in the shopping bag 100. First, the detection unit 22 detects an area representing the shopping bag 100 from each frame of the product image 32. Furthermore, the detection part 22 detects the goods outside the area | region showing the shopping bag 100 from each flame | frame in which the shopping bag 100 was detected. When the product outside the area representing the shopping bag 100 moves into the area representing the shopping bag 100, the detection unit 22 determines that the product has been put into the shopping bag 100.

In addition, the existing object recognition process can be used for the process for detecting the shopping bag 100 or the product 50 from the image and the process for specifying the position thereof. Therefore, a specific description of these processes is omitted.

In addition, the operation of putting the product into the predetermined container may be an operation of moving the product toward the predetermined container. In this case, for example, the detection unit 22 performs image analysis on the product video 32 to detect that the product has been moved toward a predetermined container (for example, the shopping bag 100). Specifically, first, the detection unit 22 detects an area representing the shopping bag 100 from each frame of the product video 32. Next, the detection unit 22 detects a product outside the area representing the shopping bag 100 from each frame in which the shopping bag 100 is detected. Furthermore, the detection unit 22 calculates the moving direction of the product based on the change in the position of the product detected from each frame. Then, when the moving direction is the direction toward the shopping bag 100, the detection unit 22 determines that the product is moved toward the shopping bag 100. A known technique can be used as a specific method for calculating the moving direction of the object included in the video.

<< predetermined operation 5 >>
In addition, for example, the predetermined operation may be an operation of lifting a product. The user of the merchandise registration apparatus 20 lifts the merchandise to put the merchandise in the predetermined container (for example, the shopping bag 100) described above. The user may also lift the product when adjusting the position of the product placed on the mounting table 40. However, in such a case, it is considered that the height at which the product is lifted is lower than when the product is put into the shopping bag 100 or the like.

Therefore, the detection unit 22 detects an operation of lifting a product by a predetermined height or more as a predetermined operation. In this case, it is preferable that the imaging device 30 captures the product from the lateral direction (a direction perpendicular to the vertical direction). When the imaging device 30 photographs the mounting table 40 from the lateral direction, the detection unit 22 first detects a product placed on the mounting table 40 from the product video 32. Next, the detection unit 22 calculates the change in the position in the height direction of the detected product using the product image 32 generated thereafter. Specifically, the detection unit 22 determines the height at which the product placed on the mounting table 40 is lifted, the position in the height direction after the product is lifted, and the position in the height direction of the mounting table 40. Calculated as the difference between If the height at which the product is lifted is equal to or higher than the predetermined height, the detection unit 22 determines that the product has been lifted (a predetermined operation has been performed).

Note that the direction in which the imaging device 30 captures the product is not limited to the lateral direction described above. For example, as shown in FIG. 3, when imaging is performed so that the imaging device 30 looks down at the product, the detection unit 22 determines whether the product is lifted based on a change in the size of the product shown in each frame of the product video 32. Is calculated. Here, the relationship between the change in the size of the product and the height at which the product is lifted (the difference between the position in the height direction of the product and the position in the height direction of the mounting table 40) It can be determined in advance based on parameters such as installation position and angle of view.

The predetermined height described above may be preset in the detection unit 22 or may be stored in a storage device accessible from the detection unit 22.

<< predetermined operation 6 >>
Further, for example, the predetermined operation may be an operation of taking out a predetermined container (shopping bag 100 or the like) for putting a product. Specifically, the detection unit 22 detects a predetermined operation by detecting that a predetermined container has been newly captured (included in the product video 32).

The predetermined operation to be detected by the detection unit 22 may be one of the above-described operations or other operations, or a plurality of operations. Information indicating which predetermined operation the detection unit 22 should detect may be preset in the detection unit 22 or may be stored in a storage device accessible from the detection unit 22.

<Registering products>
When the predetermined operation is detected by the detection unit 22 (S102: YES), the registration unit 24 registers the product included in the product video as a settlement target (S104). Here, when the target of the predetermined operation is not a specific product (the predetermined operation 1 or 2 described above), for example, the registration unit 24 registers all the products included in the product video 32 as a settlement target. On the other hand, when the target of the predetermined action is a specific product (such as the above-described predetermined actions 3 and 4), for example, the registration unit 24 registers the product that is the target of the predetermined action as a settlement target. For example, in the example of the predetermined operation 3, the registration unit 24 registers the product 50-1 taken out from the shopping cart 80 as a payment target. Further, in the example of the predetermined operation 4, the registration unit 24 registers the product 50-1 put in the shopping bag 100 as a payment target. Note that two or more products may be taken out from the shopping cart 80 or put into the shopping bag 100 in one operation.

The specific process performed by the registration unit 24 to register the product as a payment target is arbitrary. For example, the registration unit 24 registers the product as a settlement target by generating settlement information regarding the product registered as the settlement target. The checkout information is information related to the checkout process for a product registered as a checkout target. For example, the registration unit 24 stores the generated settlement information in an internal or external storage unit of the product registration device 20.

精 A single checkout can include multiple products. For example, the merchandise registration apparatus 20 may receive one or a plurality of registered items after receiving an input operation instructing the start of the registration process for the payment target until receiving an input operation instructing the end of the registration process for the payment target. Is registered as a checkout target for one checkout.

The checkout information indicating the checkout target for one checkout indicates the ID of each product registered as the checkout target. Further, the settlement information may further indicate the number of products registered as a settlement target, a transaction number, the amount of each product, and the settlement amount.

FIG. 9 is a diagram illustrating the settlement information in a table format. The settlement information shown in FIG. The settlement information 300 has two columns of a settlement information ID (Identifier) 302 and a product list 304. The settlement information ID 302 indicates an identifier of the settlement information. The product list 304 is a list of product IDs registered as settlement targets. The product ID 306 is an identifier of a product registered as a payment target.

It should be noted that since various known techniques can be used as a specific method for generating checkout information for a product to be checked in, description regarding this method is omitted.

<Identification of product ID>
There are various methods for the product registration device 20 to specify the product ID of the product included in the product video. For example, the product registration device 20 identifies the product ID of the product included in the product video by matching the product image included in the product video with the image of each product stored in the product database. Further, for example, the product registration device 20 may identify the product ID of the product by analyzing various codes (barcode, QR code (registered trademark), etc.) attached to the product included in the product video. Good.

<About Product Registration Device 20>
The merchandise registration device 20 may be a device operated by a store clerk or a device operated by a customer. In the latter case, the customer causes the merchandise registration apparatus 20 to perform merchandise registration processing.

The merchandise registration apparatus 20 may or may not have a function for performing a settlement process for a commodity registered as a settlement target. For example, the settlement process includes a process of presenting the settlement amount indicated in the settlement information to the customer, a process of accepting the input of the price by the customer, a process of counting the entered price, a process of discharging the change to the customer, and a receipt Including the process of issuing. When the merchandise registration apparatus 20 does not have a function of performing a settlement process, a device having a function of performing a settlement process is separately installed in the store.

<Modification>
In a case where the predetermined action does not target a specific product (such as the predetermined action 1 or 2), the video used for detecting the user's predetermined action need not include the product. Therefore, in this case, an imaging device for imaging a product and an imaging device for imaging a user's predetermined motion may be prepared separately. The detection unit 22 detects the user's predetermined operation from the video generated by the imaging device for imaging the user's predetermined operation. Then, the registration unit 24 sets the product included in the video at the time when the detection unit 22 detects the user's predetermined motion or the time close to it as the settlement target among the video generated by the imaging device for capturing the product. sign up. In addition, the imaging device which images a product can be installed at an arbitrary position where the product can be imaged. Similarly, an imaging device for imaging a user's predetermined motion can be installed at an arbitrary position where the user's motion can be captured.

As described above, by separately preparing an imaging device for imaging the user's predetermined operation, the user can perform the predetermined operation at a place other than the place where the product is imaged (such as on the mounting table 40). For example, in this case, the predetermined operation may be an operation performed by the user's foot under the mounting table 40. Thereby, since the variation of the operation | movement which can be handled as predetermined operation | movement increases, the convenience of the merchandise registration apparatus 20 improves.

In the embodiment described later, an example is shown in which both the user's predetermined operation and the product are detected from the product video. However, in the embodiments described later, as described above, when the predetermined operation is not for a specific product, the imaging device for capturing the product and the imaging device for capturing the user's predetermined operation are separately provided. You may prepare.

[Embodiment 2]
The product registration apparatus 20 of Embodiment 2 has the structure shown by FIG. 1, for example like the product registration apparatus 20 of Embodiment 1. FIG. Except for the points described below, the product registration device 20 of the second embodiment has the same function as the product registration device 20 of the first embodiment.

The registration unit 24 according to the second embodiment registers a product included in a predetermined area included in the product video 32 as a settlement target when a predetermined operation is detected by the detection unit 22. By setting a region where a product to be registered as a checkout target is set as a predetermined region, other products included in the product video can be prevented from being registered as a checkout target. Therefore, it becomes possible to easily register the product as a payment target. In the present embodiment, the predetermined operation is an operation that does not target a specific product (such as the predetermined operation 1 and the predetermined operation 2).

<Predetermined area 1>
For example, the predetermined area is an area predetermined in the mounting table 40. FIG. 10 is a diagram illustrating a case where a predetermined area in the mounting table 40 is a predetermined area. In the mounting table 40 of FIG. 10, a predetermined pattern or color is given to the predetermined area 110 in advance. The registration unit 24 registers the product 50-1 and the product 50-2 included in the predetermined area 110 as a settlement target when the predetermined operation is detected by the detection unit 22. On the other hand, the registration unit 24 does not register the product 50-3 that is not included in the predetermined area 110 as a settlement target.

For example, the user once places all the products in the shopping cart on the mounting table 40. Then, the user places a product to be registered as a payment target in the predetermined area 110, and places a product not registered as a payment target (for example, a product to be returned without purchase) in the other area. By doing so, the user can easily divide the products in the shopping cart into those to be registered as a payment target and those not to be registered as a payment target. Therefore, the efficiency of the work of registering the product as a payment target is improved.

In addition, the method of providing the predetermined area 110 in the mounting table 40 may be any method that allows the user to recognize the location of the predetermined area 110, and is not limited to a method in which a pattern or a color is added to the predetermined area 110 in advance. Information for specifying the location of the predetermined area 110 may be set in the detection unit 22 in advance, or may be stored in a storage unit accessible from the detection unit 22.

<Predetermined area 2>
For example, the predetermined area may be dynamically calculated by the registration unit 24. FIG. 11 is a diagram illustrating a predetermined area that is dynamically calculated by the registration unit 24. In the example of FIG. 11, the user takes out the product 50 desired to be registered as a payment target from the shopping cart 80 and places it in an area between the shopping cart 80 and the shopping bag 100. Then, the user puts a product registered as a payment target in the shopping bag 100.

Therefore, the registration unit 24 calculates “the area between the shopping cart 80 and the shopping bag 100 and on the mounting table 40” as the predetermined area 110. In the example of FIG. 11, the product 50-2 is included in the predetermined area 110. Therefore, the registration unit 24 registers the product 50-2 as a payment target when a predetermined operation is detected by the detection unit 22.

By defining the predetermined area 110 in this way, the product 50-1 in the shopping bag 100 that has already been registered as a payment target is not registered as a payment target. Therefore, it is possible to prevent the product 50-1 in the shopping bag 100 that has already been registered as a payment target from being erroneously registered as the payment target again.

Further, by defining the predetermined area 110 in this way, the product 50-3 in the shopping cart 80 is not registered as a payment target. The product 50-3 contained in the shopping cart 80 is different from the product 50-2 taken out from the shopping cart 80 and placed on the mounting table 40, and it may not be determined that it should be registered as a payment target. is there. Therefore, by preventing the product 50-3 in the shopping cart 80 from being registered as a checkout target, it is possible to prevent a product that has not yet been registered as a checkout target from being registered as a checkout target by mistake. be able to.

Note that the product video 32 may include only one of the shopping cart 80 and the shopping bag 100. When only the shopping cart 80 is included in the product video 32, the detection unit 22 sets a region left of the region including the shopping cart 80 as the predetermined region 110. When only the shopping bag 100 is included in the product image 32, the detection unit 22 sets a region on the right side of the region including the shopping bag 100 as the predetermined region 110. However, the method of determining each of the predetermined areas 110 is a method of determining when the product to be settled is moved from right to left in the product video 32. When the product to be settled is moved from left to right in the product image 32, the predetermined area 110 is on the right side of the area including the shopping cart 80 and on the left side of the area including the shopping bag 100.

The predetermined area 110 may be both the predetermined area 1 and the predetermined area 2 described above, or may be either one. In the former case, the detection unit 22 determines a region included in both a region provided with a predetermined pattern and color and a region included in a region dynamically determined by the position of the shopping cart 80 or the shopping bag 100. Region 110 is assumed.

<Process flow>
FIG. 12 is a flowchart illustrating the flow of processing performed by the product registration device 20 according to the second embodiment. The detection unit 22 determines whether or not a predetermined operation is included in the product video (S202). If the product video includes a predetermined action (S202: YES), the process of FIG. 12 proceeds to S204. On the other hand, when the predetermined operation is not included in the product video (S202: NO), the processing in FIG. 12 ends.

In S204, the detection unit 22 detects a predetermined area from the product video 32. Then, the registration unit 24 registers the products included in the detected predetermined area as a payment target (S206).

<Hardware configuration example>
The product registration apparatus 20 of Embodiment 2 is implement | achieved using the computer 200 similarly to Embodiment 1 (refer FIG. 2). In the present embodiment, each program module stored in the storage 208 described above further includes a program that realizes each function described in the present embodiment.

As described above, the embodiments of the present invention have been described with reference to the drawings. However, these are exemplifications of the present invention, and various configurations other than the above can be adopted.

Hereinafter, examples of the reference form will be added.
1. Detection means for detecting a predetermined operation from the video;
A registration unit for registering a product included in the video as a payment target when a predetermined operation is detected by the detection unit;
An information processing apparatus.
2. The registration means registers each of a plurality of products included in the video as a settlement target. The information processing apparatus described in 1.
3. The registration means detects a predetermined area from the video, and registers a product arranged in the predetermined area as a settlement target. Or 2. The information processing apparatus described in 1.
4). The predetermined operation is an operation of putting a product into a container,
The registration means registers a product to be put into the container by the predetermined operation as a settlement target. Or 2. The information processing apparatus described in 1.
5). The predetermined operation is an operation of taking out a product from a shopping cart,
The registration means registers the product taken out from the shopping cart by the predetermined operation as a settlement target. Or 2. The information processing apparatus described in 1.
6). The predetermined operation is an operation of lifting a product,
The registration means registers the product lifted by the predetermined operation as a settlement target. Or 2. The information processing apparatus described in 1.
7). The imaging device that generates the video shoots a table on which a product to be settled is placed from above. To 6. The information processing apparatus according to any one of the above.
8). A control method executed by a computer, the detection step for detecting a predetermined operation from an image,
A registration step of registering a product included in the video as a payment target when a predetermined operation is detected by the detection step;
A control method.
9. 7. In the registration step, each of a plurality of products included in the video is registered as a payment target. The control method described in 1.
10. 7. In the registration step, a predetermined area is detected from the video, and merchandise arranged in the predetermined area is registered as a payment target. Or 9. The control method described in 1.
11. The predetermined operation is an operation of putting a product into a container,
7. In the registration step, register a product to be put into the container by the predetermined operation as a settlement target. Or 9. The control method described in 1.
12 The predetermined operation is an operation of taking out a product from a shopping cart,
7. In the registration step, register the product taken out from the shopping cart by the predetermined operation as a settlement target. Or 9. The control method described in 1.
13. The predetermined operation is an operation of lifting a product,
7. In the registration step, register the product lifted by the predetermined operation as a payment target; Or 9. The control method described in 1.
14 7. The imaging device that generates the video images a table on which a product to be settled is placed from above. Thru 13. The control method as described in any one.
15. 8). To 14. A program for causing a computer to execute each step of the control method according to any one of the above.

This application claims priority based on Japanese Patent Application No. 2016-009750 filed on January 21, 2016, the entire disclosure of which is incorporated herein.

Claims

Detection means for detecting a predetermined operation from the video;
A registration unit for registering a product included in the video as a payment target when a predetermined operation is detected by the detection unit;
An information processing apparatus.
The information processing apparatus according to claim 1, wherein the registration unit registers each of a plurality of products included in the video as a settlement target.
The information processing apparatus according to claim 1 or 2, wherein the registration unit detects a predetermined area from the video and registers a product arranged in the predetermined area as a settlement target.
The predetermined operation is an operation of putting a product into a container,
The information processing apparatus according to claim 1, wherein the registration unit registers a product to be put into the container by the predetermined operation as a settlement target.
The predetermined operation is an operation of taking out a product from a shopping cart,
The information processing apparatus according to claim 1, wherein the registration unit registers a product taken out from the shopping cart by the predetermined operation as a settlement target.
The predetermined operation is an operation of lifting a product,
The information processing apparatus according to claim 1, wherein the registration unit registers the product lifted by the predetermined operation as a settlement target.
The information processing apparatus according to any one of claims 1 to 6, wherein the imaging apparatus that generates the video images a table on which a product to be settled is arranged from above.
A control method executed by a computer, the detection step for detecting a predetermined operation from an image,
A registration step of registering a product included in the video as a payment target when a predetermined operation is detected by the detection step;
A control method.
The control method according to claim 8, wherein, in the registration step, each of a plurality of products included in the video is registered as a payment target.
10. The control method according to claim 8 or 9, wherein in the registration step, a predetermined area is detected from the video, and a product arranged in the predetermined area is registered as a settlement target.
The predetermined operation is an operation of putting a product into a container,
The control method according to claim 8 or 9, wherein in the registration step, a product that is put into the container by the predetermined operation is registered as a settlement target.
The predetermined operation is an operation of taking out a product from a shopping cart,
The control method according to claim 8 or 9, wherein, in the registration step, the product taken out from the shopping cart by the predetermined operation is registered as a settlement target.
The predetermined operation is an operation of lifting a product,
The control method according to claim 8 or 9, wherein in the registration step, the product lifted by the predetermined operation is registered as a payment target.
The control method according to any one of claims 8 to 13, wherein the imaging device that generates the video images a table on which a product to be settled is arranged from above.
A program for causing a computer to execute each step of the control method according to any one of claims 8 to 14.