JP2023017034A - Registration device, registration method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable an accounting operator to grasp a progress status of image analysis.

SOLUTION: A registration device 10 of the present invention includes: an image acquisition unit 11 that acquires an image captured by imaging a mounting surface of a table on which a commodity is mounted; an analysis unit 12 that recognizes the commodity included in the image; a registration unit 14 that registers the recognized commodity as a target of accounting; and an output unit 13 that projects on the table, or displays on a display of the table formed by the display, information corresponding to a progress status of analysis of the image performed by the analysis unit 12. The output unit 13 displays first information on the table when the analysis unit 12 recognizes that an object is mounted on the table, and displays second information different from the first information on the table when the analysis unit 12 recognizes the commodity.

SELECTED DRAWING: Figure 2

COPYRIGHT: (C)2023,JPO&INPIT

Description

The present invention relates to a registration device, registration method and program.

Japanese Patent Laid-Open No. 2002-200000 discloses a commodity registration device that captures a picture of a commodity to be accounted for, recognizes the commodity in the image by image analysis, and registers the commodity.

Patent Literature 2 discloses a display control device that determines a display area of a display object displayed on a display surface according to information on a real object on the display surface. The display control device can recognize the type of tableware placed on the dining table and output a display effect and sound according to the type.

Patent Document 3 discloses a user terminal that receives an image of a product generated by a camera, displays the image, and accepts designation of a product from among the displayed products.

JP 2016-62545 A WO2015/098189 JP-A-3-143185

In the technique of photographing a product to be checked out and recognizing and registering the product in the image by image analysis, there is a problem that it is difficult for the checkout operator to grasp the progress of the image analysis. SUMMARY OF THE INVENTION An object of the present invention is to enable a checkout operator to grasp the progress of image analysis in a technique of recognizing and registering products for checkout by image analysis.

According to the invention,
an image acquisition means for acquiring an image of a placement surface of a table on which a product is placed;
analysis means for recognizing the product included in the image;
registration means for registering the recognized product as an accounting object;
output means for displaying information corresponding to the progress of analysis of the image by the analysis means on the platform;
has
When the product registered by the registration means is no longer detected in the image after registration, the output means displays predetermined information at the placement position specified in the image before registration, and displays the predetermined information at the placement position specified in the image before registration. A registration device is provided that thins out the information as .

Moreover, according to the present invention,
the computer
an image acquisition step of acquiring an image of the placement surface of the table on which the product is placed;
an analysis step of recognizing the product included in the image;
a registration step of registering the recognized product as an accounting object;
an output step of displaying information corresponding to the progress of analysis of the image in the analysis step on the table;
and run
In the output step, when the product registered in the registration step is no longer detected in the image after registration, predetermined information is displayed at the placement position specified in the image before registration, and the product is not detected for a continuous period of time. A registration method is provided that thins out the information as .

Moreover, according to the present invention,
the computer,
image acquisition means for acquiring an image of the placement surface of the table on which the product is placed;
analysis means for recognizing the product included in the image;
registration means for registering the recognized product as an accounting object;
output means for displaying information corresponding to the progress of analysis of the image by the analysis means on the platform;
function as
When the product registered by the registration means is no longer detected in the image after registration, the output means displays predetermined information at the placement position specified in the image before registration, and displays the predetermined information at the placement position specified in the image before registration. A program is provided that thins out the information as .

According to the present invention, in a technique of recognizing and registering products for accounting by image analysis, it becomes possible for an accounting worker to grasp the progress of image analysis.

The above-mentioned objects, as well as other objects, features and advantages, will be further clarified by the preferred embodiments described below and the accompanying drawings below.

It is a figure which shows an example of the hardware constitutions of the registration apparatus of this embodiment. It is a figure which shows an example of the functional block diagram of the registration apparatus of this embodiment. It is a figure which shows typically an example of the camera and stand which are used by the accounting process using the registration apparatus of this embodiment. It is a flowchart which shows an example of the flow of a process of the registration apparatus of this embodiment. It is a figure which shows typically an example of the information which the registration apparatus of this embodiment displays on a stand. It is a figure which shows typically an example of the information which the registration apparatus of this embodiment displays on a stand. It is a figure which shows typically an example of the information which the registration apparatus of this embodiment displays on a stand.

<First Embodiment>
First, an outline of the registration device of this embodiment will be described. The registration device of this embodiment is used for checkout processing at a store that sells products. The registration device may be a so-called POS (point of sales) register. The registration device may be a device intended to be operated by a store clerk, or may be a device intended to be operated by a customer.

In transaction processing using a registration device, a table on which products to be processed are placed and a camera for photographing the placement surface of the table are prepared. In checkout processing, checkout operators (clerks, customers, etc.) place products to be checked out on the placement surface of a stand. When the registration device acquires the image generated by the camera, it analyzes the image and recognizes the products included in the image. Then, the registration device registers the recognized product as a product for accounting.

Note that the registration device has a function that allows the checkout operator to grasp the progress of the image analysis. Specifically, the registration device displays information corresponding to the progress of the image analysis on the table according to the progress of the image analysis. A plurality of pieces of information corresponding to the progress status are different from each other.

According to the registration device of this embodiment, the checkout operator can grasp the progress of the image analysis based on the information displayed on the table.

Next, the configuration of the registration device of this embodiment will be described in detail. First, an example of the hardware configuration of the registration device will be described. The functions provided by the registration device are the CPU (Central Processing Unit) of any computer, memory, programs loaded into the memory, storage units such as hard disks storing the programs (programs stored in advance from the stage of shipping the device) In addition, it can store programs downloaded from a storage medium such as a CD (Compact Disc) or a server on the Internet), and is realized by any combination of hardware and software centering on a network connection interface. It should be understood by those skilled in the art that there are various modifications to the implementation method and apparatus.

FIG. 1 is a block diagram illustrating the hardware configuration of a registration device. As shown in FIG. 1, the registration device has a processor 1A, a memory 2A, an input/output interface 3A, a peripheral circuit 4A and a bus 5A. The peripheral circuit 4A includes various modules. Note that the peripheral circuit 4A may not be provided.

The bus 5A is a data transmission path for mutually transmitting and receiving data between the processor 1A, memory 2A, peripheral circuit 4A and input/output interface 3A. The processor 1A is, for example, an arithmetic processing device such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The memory 2A is, for example, a RAM (Random Access Memory) or a ROM (Read Only Memory). The input/output interface 3A is an interface for acquiring information from input devices (eg, keyboard, mouse, microphone, etc.), external devices, external servers, external sensors, etc., and output devices (eg, displays, speakers, printers, mailers, etc.). etc.), an interface for outputting information to an external device, an external server, and the like. The processor 1A can issue commands to each module and perform calculations based on the calculation results thereof.

FIG. 2 shows an example of a functional block diagram of the registration device 10. As shown in FIG. As illustrated, the registration device 10 has an image acquisition section 11 , an analysis section 12 , an output section 13 and a registration section 14 .

The image acquisition unit 11 acquires an image of the placement surface of the table on which the product is placed. As described above, in transaction processing using the registration device 10, a table for placing products to be processed and a camera for photographing the placement surface of the table are prepared. The image acquisition unit 11 acquires an image generated by the camera. A camera may generate a moving image (a plurality of images), and the image acquisition unit 11 may acquire the moving image from the camera.

FIG. 3 schematically shows an example of a platform and a camera. In the figure, a base 1, a member 2, a support 3 and a camera 4 are shown. The member 2 is located on the top surface of the platform 1 and forms part of the platform 1 . The exposed surface of the member 2 serves as a placement surface on which the product is placed. Note that the member 2 may be a display, or may be another member. A camera 4 is attached to the post 3 and photographs the exposed surface of the member 2 from above. An image generated by the camera 4 is transmitted to a registration device 10 (not shown) through wired and/or wireless communication.

Returning to FIG. 2, the analysis unit 12 analyzes the image acquired by the image acquisition unit 11 and recognizes the product included in the image. The analysis unit 12 can recognize the product using a pattern matching technique or the like.

For example, the analysis unit 12 detects an object included in the image using techniques such as image binarization and outline extraction (object detection). After that, the analysis unit 12 compares the features of appearance appearing in the image of the detected object (features of the area where the object exists in the image) with the features of appearance of each of a plurality of pre-registered products. , to identify which product the object is (product recognition). For example, the analysis unit 12 may identify a product with the highest similarity in appearance to the object, or a product with the highest similarity in appearance to the object and having the similarity equal to or higher than a reference value. Appearance features of the product include, but are not limited to, color, surface unevenness, shape, and the like.

The output unit 13 displays information corresponding to the progress of image analysis by the analysis unit 12 on a table (for example, the mounting surface of the table). A plurality of pieces of information corresponding to the progress status are different from each other. For example, the output unit 13 displays a first mark M (e.g., a circle, etc.) on the table as shown in FIG. 5 upon completion of object detection, and displays a first mark M (eg, circle, etc.) on the platform as shown in FIG. 6 upon completion of product recognition. A second mark M (eg, square, etc.) may be displayed on the platform. Here, an example is shown in which information is displayed on the table according to the completion of object detection and product recognition. good too. Also, the information to be displayed is not limited to the mark, and may be other information such as letters, numbers, pictures, and characters. Furthermore, the display mode (eg, color, shade) of the mark may be changed according to the progress of the image analysis. For example, when the analysis unit 12 recognizes that an object is placed on the table, the output unit 13 displays a mark of a first color on the table, and when the analysis unit 12 recognizes a product, the output unit 13 displays a mark of a second color. A colored mark may be displayed on the table.

When a display is provided on a part of the table, the output unit 13 can display information corresponding to the progress of the image analysis on the display. For example, as described with reference to FIG. 3, when the placement surface on which the product is placed is configured by the display (member 2), the output unit 13 displays the progress of image analysis on the display (member 2). can display information corresponding to Alternatively, the output unit 13 may use a projection device to project information onto the table.

In addition, the output unit 13 can display information corresponding to the progress of image analysis at a position corresponding to the placement position of the object. For example, as shown in FIGS. 5 and 6, information (mark M) may be displayed surrounding the placement position of the object, or information (mark M) may be displayed near the placement position of the object as shown in FIG. A mark M) may be displayed. For example, the output unit 13 holds in advance a conversion rule for converting "coordinates of an image" into "coordinates of a display area of a display constituting a mounting surface of a table". Then, the output unit 13 converts the position (coordinates) of the object in the image into the coordinates of the display area of the display based on the conversion rule, and, for example, surrounds the converted position or near the position and converts the information. display.

Similarly, the output unit 13 may store in advance a conversion rule for converting "coordinates of the image" into "coordinates of the projection area of the projection device". Then, the output unit 13 converts the position (coordinates) of the object in the image into the coordinates of the projection area of the projection device based on the conversion rule, and for example, converts the converted position so as to surround or be close to the position. information may be displayed.

The coordinates of an image are represented by a coordinate system in which an arbitrary position in the image is set as the origin and arbitrary directions are defined as the x-axis direction and the y-axis direction. The coordinates of the display area of the display are represented by a coordinate system in which an arbitrary position within the display area is set as the origin and arbitrary directions are defined as the x-axis direction and the y-axis direction. The coordinates of the projection area of the projection device are represented by a coordinate system in which an arbitrary position within the projection area is the origin and arbitrary directions are defined as the x-axis direction and the y-axis direction.

The registration unit 14 registers the product recognized by the analysis unit 12 as an accounting object. For example, the registration unit 14 acquires the product information of the product recognized by the analysis unit 12 from the product master, and registers at least part of the acquired product information in the registration device 10 . The product information includes price, product name, product code, and the like.

According to the registration device 10 of the present embodiment described above, it is possible to recognize the product placed on the table by image analysis and register the recognized product as an object of accounting processing. According to such a registration device 10, product registration work is made more efficient.

In addition, the registration device 10 can display information corresponding to the progress of the image analysis on the table according to the progress of the image analysis. It should be noted that the plurality of pieces of information corresponding to the progress status are different from each other. For example, the registration device 10 displays a first mark (eg, circle, etc.) on the platform upon completion of object detection, and displays a second mark (eg, square, etc.) on the platform upon completion of product recognition. may be displayed. According to the registration device 10 as described above, the checkout operator can grasp the progress of the image analysis based on the information displayed on the table.

<Second embodiment>
The registration device 10 of the present embodiment can display information on the table at a timing different from that of the first embodiment. A detailed description will be given below.

An example of the hardware configuration of the registration device 10 is the same as in the first embodiment.

An example of a functional block diagram of the registration device 10 is shown in FIG. 2, as in the first embodiment. The configurations of the image acquisition unit 11 and the registration unit 14 are the same as those of the first embodiment.

As in the first embodiment, the analysis unit 12 analyzes the image acquired by the image acquisition unit 11 and recognizes the product included in the image. The analysis unit 12 can execute "detection of an object", "recognition that an object has been placed on the table", and "identification of which product the object placed on the table is" in this order. can.

Detection of an object can be realized using all techniques such as image binarization and outline extraction.

Recognition that an object has been placed on a table can be realized, for example, as follows. First, the analysis unit 12 is caused to hold in advance information indicating the display area of the mounting surface of the table in the image. The display area may be indicated by the coordinates of a coordinate system in which the origin is an arbitrary point in the image and the x-axis and y-axis are arbitrary directions (e.g., if the display area is a polygon, the coordinates of the vertices ). A part of the image may be the display area of the mounting surface of the table, or the entire image may be the display area of the mounting surface of the table. It is assumed that the installation position of the table, the installation position of the camera, the orientation of the camera, etc. are fixed, and the relative positional relationship between the table and the camera is constant. When the analysis unit 12 detects that the movement of the object present in the display area of the mounting surface of the table has stopped for a predetermined time or longer, the analysis unit 12 may recognize that the object has been placed on the table.

Identification of which product an object placed on the table is (product recognition) is based on the features of the appearance appearing in the image of the detected object (features of the area in which the object exists in the image) and the pre-registered This is realized by matching the features of the appearance of each of a plurality of commodities. For example, the analysis unit 12 may identify a product with the highest similarity in appearance to the object, or a product with the highest similarity in appearance to the object and having the similarity equal to or higher than a reference value. Appearance features of the product include, but are not limited to, color, surface unevenness, shape, and the like.

The output unit 13 displays information indicating that the analysis unit 12 has recognized that the object has been placed on the table and that the product that has been placed on the table has been specified. For example, when the analysis unit 12 recognizes that an object has been placed on the table, the output unit 13 displays the first information corresponding to the situation on the table. Further, when the analysis unit 12 identifies which product is the object placed on the table, the output unit 13 displays second information corresponding to the situation on the table. The first information and the second information are information different from each other.

Note that, when an object is detected by the analysis unit 12, the output unit 13 may display information corresponding to the situation (information different from the first and second information) on the table.

Next, an example of the flow of processing of the registration device 10 of this embodiment will be described using the flowchart of FIG.

When the process is started, the image acquiring section 11 starts acquiring images from the camera. Then, the analysis unit 12 starts analyzing the image.

When the analysis unit 12 recognizes that the object has been placed on the table (Yes in S10), the output unit 13 displays the first information corresponding to the situation on the table (S11). The output unit 13 displays the first information at a position corresponding to the placement position of the object. Note that the first information is not displayed on the table while the analysis unit 12 does not recognize that the object is placed on the table (No in S10).

After that, when the analysis unit 12 identifies which product the object is (Yes in S12), the output unit 13 displays the second information corresponding to the situation (S13). The output unit 13 displays the second information at a position corresponding to the placement position of the object. Further, in S13, the registration unit 14 registers the specified product as an accounting object. If the analysis unit 12 does not specify which product the object placed on the table is (No in S12), the second information is not displayed on the table.

Note that the output unit 13 may continue to display the first information until the second information is displayed. When the analysis unit 12 identifies which product the object is (Yes in S12), the display of the first information is terminated accordingly, and the second information is displayed instead. good. After that, the output unit 13 may continue to display the second information until the object leaves the table or until there is an input for payment processing. As another example, the output unit 13 may display the first information and the second information for a predetermined period of time, and stop displaying the first information and the second information after the predetermined period of time has elapsed.

After that, when the analysis unit 12 recognizes that the next object has been placed on the table (Yes in S14), the output unit 13 displays the first information corresponding to the situation on the table (S15). . The output unit 13 displays the first information at a position corresponding to the placement position of the object. After that, when the analysis unit 12 identifies which product the object is (Yes in S16), the output unit 13 displays the second information corresponding to the situation on the table (S17). The output unit 13 displays the second information at a position corresponding to the placement position of the object. In S17, the registration unit 14 further registers the identified product as an accounting object. Note that when a plurality of objects are placed on the table, a plurality of pieces of first information may be displayed on the table, a plurality of pieces of second information may be displayed on the table, or a plurality of pieces of the first information may be displayed on the table. Both the information of and the second information may be displayed.

If there is no new recognition/specification in S14 and S16 by the analysis unit 12 (No in S14, No in S16) and there is an input for settlement processing (Yes in S18), the registration device 10 executes the settlement processing ( S19). For example, the registration device 10 may accept an input of cash as payment for the total amount of payment calculated based on the products registered so far, and may output change or a receipt as necessary. Further, the registration device 10 may accept credit card input, communicate with a credit company system, and perform settlement processing. Further, the registration device 10 may transmit information for settlement processing (information indicating registered products, total payment amount, etc.) to another settlement device.

According to the registration device 10 of this embodiment described above, it is possible to achieve the same effects as those of the first embodiment. Further, according to the registration device 10 of the present embodiment, it is possible to display different information on the table according to the recognition that the object is placed on the table and the product recognition.

According to the registration device 10 of the present embodiment, the checkout operator can recognize that an object has been placed on the table based on the information displayed on the table. Also, the accounting worker can grasp that the product has been recognized. Since the accounting worker can grasp the progress of the image analysis, he/she can proceed with the registration work without anxiety.

<Third Embodiment>
The registration device 10 of this embodiment differs from those of the first and second embodiments in the information displayed on the table when the commodity is recognized. A detailed description will be given below.

An example of the hardware configuration of the registration device 10 is the same as in the first and second embodiments.

An example of a functional block diagram of the registration device 10 is shown in FIG. 2, as in the first and second embodiments. The configurations of the image acquisition unit 11 and the registration unit 14 are the same as those of the first and second embodiments.

The analysis unit 12 identifies the product category of the recognized product. Other configurations of the analysis unit 12 are the same as in the first and second embodiments. For example, the product category information indicating the product category for each product is stored in advance in the analysis unit 12 . Then, when the product is recognized by the image analysis, the analysis unit 12 refers to the product category information and specifies the product category of the recognized product.

The output unit 13 changes the second information to be displayed on the stand when the analysis unit 12 identifies which product the object placed on the stand is, according to the product category of the specified product. . For example, the output unit 13 may display a star mark on the stand when "food" is specified, and display a triangular mark on the stand when "daily miscellaneous goods" is specified. Other configurations of the output unit 13 are the same as in the first and second embodiments.

According to the registration device 10 of this embodiment described above, it is possible to achieve the same effects as those of the first and second embodiments. Further, according to the registration device 10 of the present embodiment, the checkout operator can grasp not only that the product has been recognized but also the recognition content (product category) based on the information displayed on the table. Based on this, the accounting worker can confirm whether the recognized contents are correct.

<Fourth Embodiment>
The registration device 10 of this embodiment differs from those of the first and second embodiments in the information displayed on the table when the commodity is recognized. A detailed description will be given below.

An example of the hardware configuration of the registration device 10 is the same as in the first and second embodiments.

An example of a functional block diagram of the registration device 10 is shown in FIG. 2, as in the first and second embodiments. The configurations of the image acquisition unit 11 and the registration unit 14 are the same as those of the first and second embodiments.

The analysis unit 12 outputs the reliability of the recognition result of product recognition. The reliability may be a similarity calculated by pattern matching (similarity between a recognized product and an object included in an image). Other configurations of the analysis unit 12 are the same as in the first and second embodiments.

The output unit 13 changes the information displayed on the stand when the analyzing unit 12 recognizes which product the object placed on the stand is, according to the reliability of the recognition result. For example, reliability may be divided into multiple layers such as "high", "medium", and "low", and information may be prepared for each layer. For example, the output unit 13 displays a double circle mark on the platform when the reliability is "high", displays a circle mark on the platform when the reliability is "medium", and displays a circle mark on the platform when the reliability is "low". In some cases, a cross mark may be displayed on the table. Other configurations of the output unit 13 are the same as in the first and second embodiments.

According to the registration device 10 of this embodiment described above, it is possible to achieve the same effects as those of the first and second embodiments. Further, according to the registration device 10 of the present embodiment, the checkout operator can grasp not only that the product has been recognized, but also the reliability of the product recognition based on the information displayed on the table. If the reliability is low, the accounting worker can confirm whether the recognition content is correct. According to this embodiment, the checkout operator can check the recognition details only for products with low reliability, instead of checking whether the recognition details for all products are correct. As a result, work can be saved.

<Fifth Embodiment>
The registration device 10 of the present embodiment can display information on the table at timings different from those of the first to fourth embodiments. A detailed description will be given below.

An example of the hardware configuration of the registration device 10 is the same as in the first to fourth embodiments.

An example of a functional block diagram of the registration device 10 is shown in FIG. 2, as in the first to fourth embodiments. The configurations of the image acquisition unit 11 and the registration unit 14 are the same as those of the first to fourth embodiments.

The analysis unit 12 continuously analyzes a plurality of frames (images) included in the moving image. After recognizing in the first frame the first product that was not recognized in the frame immediately before the first frame, the analysis unit 12 determines that the recognition result for the frame after the first frame satisfies a predetermined condition. When satisfied, it establishes a recognition result that the first item is included in the image. The predetermined condition may be to recognize the first product in M consecutive frames after recognizing the first product in the first frame (M is an integer equal to or greater than 1). Other configurations of the analysis unit 12 are the same as those of the first to fourth embodiments.

When the analysis unit 12 recognizes the first product in the first frame, the output unit 13 displays the third information on the table. Then, when the analysis unit 12 determines that the predetermined condition is satisfied, the output unit 13 displays the second information on the platform. The third information is different from information corresponding to other situations of image analysis (eg, first information, second information). Other configurations of the output unit 13 are the same as those of the first to fourth embodiments.

After confirming the recognition result that the first product is included in the image, the registration unit 14 registers the first product as an accounting object.

According to the registration device 10 of this embodiment described above, it is possible to achieve the same effects as those of the first to fourth embodiments. Further, according to the registration device 10 of the present embodiment, it is possible to determine the recognition result of the product included in the image based on not only the analysis result for one frame but also the analysis result for a plurality of frames. As a result, the reliability of analysis results can be enhanced.

Further, according to the registration device 10 of the present embodiment, the checkout operator proceeds with the process of finalizing the recognition results of the products included in the image based on the analysis results for a plurality of frames based on the information displayed on the table. You can grasp the situation. As a result, the accounting worker can grasp the state of image analysis in more detail.

A variant applicable to all embodiments will now be described. The registration device 10 may have a switching unit that turns ON/OFF the display of information on the table by the output unit 13 . The switching unit can switch ON/OFF according to user input. When the display of information on the table by the output unit 13 is ON, the output unit 13 displays information corresponding to the progress of the analysis of the image by the analysis unit 12 on the table. On the other hand, when the display of information on the table by the output unit 13 is turned off, the output unit 13 does not display information corresponding to the progress of image analysis by the analysis unit 12 on the table. A customer or store who finds the display of information on the table to be troublesome can avoid displaying the information on the table by operating the registration device 10 to turn it off. Thus, according to the modification, a user-friendly configuration can be realized.

Note that the switching unit for turning ON/OFF the display of information on the table by the output unit 13 may have a function for turning ON/OFF the display of information about a part of the product group. For example, it may have a function of displaying information on the table when a product (perishable food, medicine, etc.) belonging to a specific product category is identified. Alternatively, the display of information on the stand may be restricted for products belonging to a specific product category. Alternatively, it may be a function of displaying information on the stand for products with a predetermined amount or more, or a function not displaying information on the table with respect to goods with a predetermined amount or less. The switching unit may be a physical switch, or the button of the switching unit may be displayed on a table for displaying information.

Other modifications applicable to all embodiments will be described. When the analysis unit 12 recognizes that the object is placed on the table, but does not specify which product the object is (when the product is not recognized), the output unit 13 outputs information to that effect. It can be displayed on the table. This information is different from information corresponding to other situations of image analysis (eg, first information, second information, third information). The case where the commodity is not recognized may be, for example, the case where there is no commodity with the similarity above the reference value. According to the modification, the checkout operator can grasp the situation where the object is detected but the product is not recognized based on the information displayed on the table. As a result, the situation of image analysis can be grasped in detail.

Other modifications applicable to all embodiments will be described. A product recognized by the analysis unit 12 (in the case of the fifth embodiment, the recognition result is confirmed) and registered by the registration unit 14 (hereinafter referred to as “registered product”) is displayed in a frame image after registration (eg, product frame images subsequent to the recognized frame image, and frame images subsequent to the frame image for which the recognition result was confirmed), the output unit 13 returns the original placement position of the registered product (the last detected frame image). Information to that effect is displayed at the placement position specified by the frame image). The output unit 13 may change the form of the information displayed at the original placement position according to the undetected continuous time (the number of undetected continuous frames). For example, the output unit 13 may display predetermined information (eg, predetermined mark) at the original placement position, and gradually lighten the density of the information as the continuous time increases. In addition, the output unit 13 gradually reduces the density of the information (eg, the predetermined mark) displayed at the original placement position while the registered product is being detected as the continuous time increases. good too.

Occurrences, noise, and other factors may cause temporarily undetected items to be placed. If detection and non-detection alternate in successive frames, the information display changes discontinuously, making it difficult for the checkout operator to grasp the recognition result status. According to the discussion variant, this can be prevented.

Examples of reference forms are added below.
1. an image acquisition means for acquiring an image of a placement surface of a table on which a product is placed;
analysis means for recognizing the product included in the image;
registration means for registering the recognized product as an accounting object;
output means for displaying information corresponding to the progress of analysis of the image by the analysis means on the platform;
registration device.
2. 1. In the registration device according to 1,
The output means is a registration device for displaying the first information on the table when the analyzing means recognizes that the object is placed on the table.
3. In the registration device according to 1 or 2,
The output means is a registration device for displaying the second information on the stand when the commodity is recognized by the analysis means.
4. 3. In the registration device according to 3,
The output means is a registration device that changes the second information according to the product category of the recognized product.
5. 3. In the registration device according to 3,
The output means is a registration device that varies the second information according to the reliability of the product recognition result.
6. 6. In the registration device according to any one of 3 to 5,
After recognizing, in the first frame, the first product that was not recognized in the frame immediately preceding the first frame, the analysis means is configured to perform recognition of a recognition result for a frame after the first frame under a predetermined condition. is satisfied, confirming the recognition result that the first product is included in the image;
The output means displays the third information on the table when the first product is recognized in the first frame by the analysis means, and is determined by the analysis means to satisfy the predetermined condition. and a registration device for displaying the second information on the platform.
7. In the registration device according to any one of 1 to 6,
The output means displays a mark of a first color on the table when the analysis means recognizes that the object is placed on the table, and displays a second mark when the analysis means recognizes the commodity. A registration device that displays on the platform a mark of the color of
8. In the registration device according to any one of 1 to 7,
The output means is a registration device for displaying information indicating that the product cannot be recognized on the table when the image includes an object that cannot be recognized as the product by the analysis means.
9. 9. In the registration device according to any one of 1 to 8,
When the product registered by the registration means is no longer detected in images after registration, the output means displays predetermined information at the placement position specified in the last detected image, a registration device that thins out the information as .
10. the computer
an image acquisition step of acquiring an image of the placement surface of the table on which the product is placed;
an analysis step of recognizing the product included in the image;
a registration step of registering the recognized product as an accounting object;
an output step of displaying information corresponding to the progress of analysis of the image in the analysis step on the table;
Registration method to perform.
11. the computer,
image acquisition means for acquiring an image of the placement surface of the table on which the product is placed;
analysis means for recognizing the product included in the image;
registration means for registering the recognized product as an accounting object;
output means for displaying information corresponding to the progress of analysis of the image by the analysis means on the platform;
A program that acts as

Claims (9)

an image acquisition means for acquiring an image of a placement surface of a table on which a product is placed;
analysis means for recognizing the product included in the image;
registration means for registering the recognized product as an accounting object;
output means for displaying information corresponding to the progress of analysis of the image by the analysis means on the platform;
has
When the product registered by the registration means is no longer detected in the image after registration, the output means displays predetermined information at the placement position specified in the image before registration, and displays the predetermined information at the placement position specified in the image before registration. a registration device that thins out the information as . The registration device according to claim 1,
The output means is a registration device for displaying the first information on the table when the analysis means recognizes that the object is placed on the table. The registration device according to claim 1 or 2,
The output means is a registration device for displaying the second information on the stand when the commodity is recognized by the analysis means. The registration device according to claim 3,
The output means is a registration device that changes the second information according to the product category of the recognized product. The registration device according to claim 3,
The output means is a registration device that varies the second information according to the reliability of the product recognition result. The registration device according to any one of claims 1 to 5,
The output means displays a mark of a first color on the table when the analysis means recognizes that the object is placed on the table, and displays a second color mark when the analysis means recognizes the product. A registration device that displays on the platform a mark of the color of The registration device according to any one of claims 1 to 6,
The output means is a registration device that, when the image includes an object that cannot be recognized as the product by the analysis means, displays information indicating that the product cannot be recognized on the table. the computer
an image acquisition step of acquiring an image of the placement surface of the table on which the product is placed;
an analysis step of recognizing the product included in the image;
a registration step of registering the recognized product as an accounting object;
an output step of displaying information corresponding to the progress of analysis of the image in the analysis step on the table;
and run
In the output step, when the product registered in the registration step is no longer detected in the image after registration, predetermined information is displayed at the placement position specified in the image before registration, and the product is not detected for a continuous period of time. A registration method in which the information is thinned as . the computer,
image acquisition means for acquiring an image of the placement surface of the table on which the product is placed;
analysis means for recognizing the product included in the image;
registration means for registering the recognized product as an accounting object;
output means for displaying information corresponding to the progress of analysis of the image by the analysis means on the platform;
function as
When the product registered by the registration means is no longer detected in the image after registration, the output means displays predetermined information at the placement position specified in the image before registration, and displays the predetermined information at the placement position specified in the image before registration. A program that thins out the information as .

Images