JP7435716B2 - Registration device, registration method and program - Google Patents

Description

The present invention relates to a registration device, a registration method, and a program.

Patent Document 1 discloses a product registration device that photographs a product to be purchased, uses image analysis to recognize and register the product in the image.

Patent Document 2 discloses a display control device that determines a display area of a display object displayed on a display surface according to information about a real object on the display surface. The display control device can recognize the type of tableware placed on the dining table and output display effects and sounds according to the type.

Patent Document 3 discloses a user terminal that receives an image of a product generated by a camera, displays the image, and accepts designation of a product from among the displayed products.

JP2016-62545A International Publication No. 2015/098189 Japanese Patent Application Publication No. 3-143185

In the technology of photographing a product to be checked out and using image analysis to recognize and register the product in the image, there is a problem in that it is difficult for the checkout operator to grasp the progress of the image analysis. An object of the present invention is to enable a checkout operator to grasp the progress of image analysis in a technology that uses image analysis to recognize and register products to be checked out.

According to the invention,
an image acquisition means for acquiring an image of the placement surface of the table on which the product is placed;
analysis means for recognizing the product included in the image;
a registration means for registering the recognized product as an accounting object;
output means for displaying information corresponding to the progress of the analysis of the image by the analysis means on the stand;
has
When the product registered by the registration means is no longer detected in the image after registration, the output means displays predetermined information at the placement position specified in the image before registration, and displays a continuous period of time during which the product is not detected. A registration device is provided that thins out the information as the information increases.

Further, according to the present invention,
The computer is
an image acquisition step of acquiring an image of the placement surface of the table on which the product is placed;
an analysis step of recognizing the product included in the image;
a registration step of registering the recognized product as an accounting object;
an output step of displaying information corresponding to the progress of the analysis of the image in the analysis step on the stand;
Run
In the output step, if the product registered in the registration step is no longer detected in the image after registration, predetermined information is displayed at the placement position specified in the image before registration, and the continuous time when the product is not detected is displayed. A registration method is provided in which the information becomes thinner as the information becomes larger.

Further, according to the present invention,
computer,
image acquisition means for acquiring an image of the placement surface of the table on which the product is placed;
analysis means for recognizing the product included in the image;
registration means for registering the recognized product as an accounting object;
output means for displaying information corresponding to the progress of the analysis of the image by the analysis means on the stand;
function as
When the product registered by the registration means is no longer detected in the image after registration, the output means displays predetermined information at the placement position specified in the image before registration, and displays a continuous period of time during which the product is not detected. A program is provided that thins out the information as it becomes larger.

According to the present invention, in a technology that uses image analysis to recognize and register products to be accounted for, it becomes possible for an accounting operator to grasp the progress status of image analysis.

The above-mentioned objects, and other objects, features, and advantages will become more apparent from the following preferred embodiments and accompanying drawings.

FIG. 2 is a diagram showing an example of the hardware configuration of a registration device according to the present embodiment. FIG. 2 is a diagram showing an example of a functional block diagram of the registration device according to the present embodiment. FIG. 2 is a diagram schematically showing an example of a camera and a stand used in accounting processing using the registration device of the present embodiment. It is a flowchart which shows an example of the flow of processing of the registration device of this embodiment. FIG. 2 is a diagram schematically showing an example of information displayed on a stand by the registration device of the present embodiment. FIG. 2 is a diagram schematically showing an example of information displayed on a stand by the registration device of the present embodiment. FIG. 2 is a diagram schematically showing an example of information displayed on a stand by the registration device of the present embodiment.

<First embodiment>
First, an overview of the registration device of this embodiment will be explained. The registration device of this embodiment is used for accounting processing at a store that sells products. The registration device may be a so-called POS (point of sales) register. The registration device may be a device intended to be operated by a store clerk, or may be a device intended to be operated by a customer.

In accounting processing using a registration device, a table for placing products to be processed for transaction and a camera for photographing the placement surface of the table are prepared. In checkout processing, a checkout worker (a clerk, a customer, etc.) places the products to be checked out on the placement surface of a stand. Upon acquiring the image generated by the camera, the registration device analyzes the image and recognizes the product included in the image. Then, the registration device registers the recognized product as a product to be accounted for.

Note that the registration device has a function that allows the accounting operator to grasp the progress status of the image analysis. Specifically, the registration device displays information corresponding to the progress status of the image analysis on the stand depending on the progress status of the image analysis. A plurality of pieces of information corresponding to the progress status are different from each other.

According to the registration device of this embodiment, the checkout operator can grasp the progress of image analysis based on the information displayed on the table.

Next, the configuration of the registration device of this embodiment will be explained in detail. First, an example of the hardware configuration of the registration device will be described. The functions provided by the registered device include the CPU (Central Processing Unit) of any computer, memory, programs loaded into the memory, and storage units such as hard disks that store the programs (programs stored in advance from the stage of shipping the device). In addition, it can also store programs downloaded from storage media such as CDs (Compact Discs) and servers on the Internet, etc.), and is realized by any combination of hardware and software, centering on network connection interfaces. It will be understood by those skilled in the art that there are various modifications to the implementation method and device.

FIG. 1 is a block diagram illustrating the hardware configuration of a registration device. As shown in FIG. 1, the registration device includes a processor 1A, a memory 2A, an input/output interface 3A, a peripheral circuit 4A, and a bus 5A. The peripheral circuit 4A includes various modules. Note that the peripheral circuit 4A may not be provided.

The bus 5A is a data transmission path through which the processor 1A, memory 2A, peripheral circuit 4A, and input/output interface 3A exchange data with each other. The processor 1A is, for example, an arithmetic processing device such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The memory 2A is, for example, a RAM (Random Access Memory) or a ROM (Read Only Memory). The input/output interface 3A is an interface for acquiring information from input devices (e.g., keyboard, mouse, microphone, etc.), external devices, external servers, external sensors, etc., and output devices (e.g., displays, speakers, printers, mailers, etc.). etc.), an interface for outputting information to external devices, external servers, etc. The processor 1A can issue commands to each module and perform calculations based on the results of those calculations.

FIG. 2 shows an example of a functional block diagram of the registration device 10. As illustrated, the registration device 10 includes an image acquisition section 11, an analysis section 12, an output section 13, and a registration section 14.

The image acquisition unit 11 acquires an image of the placement surface of the table on which the product is placed. As described above, in the checkout process using the registration device 10, a stand for placing the product to be processed for checkout and a camera for photographing the placement surface of the stand are prepared. The image acquisition unit 11 acquires an image generated by the camera. The camera may generate a moving image (a plurality of images), and the image acquisition unit 11 may acquire the moving image from the camera.

FIG. 3 schematically shows an example of a stand and a camera. In the figure, a stand 1, a member 2, a support 3, and a camera 4 are shown. The member 2 is located on the upper surface of the table 1 and forms a part of the table 1. The exposed surface of the member 2 becomes a placement surface on which the product is placed. Note that the member 2 may be a display or another member. A camera 4 is attached to the support 3 and photographs the exposed surface of the member 2 from above. Images generated by the camera 4 are transmitted to a registration device 10 (not shown) through wired and/or wireless communication.

Returning to FIG. 2, the analysis unit 12 analyzes the image acquired by the image acquisition unit 11 and recognizes the product included in the image. The analysis unit 12 can recognize products using pattern matching technology or the like.

For example, the analysis unit 12 detects an object included in the image using techniques such as image binarization and contour extraction (object detection). Thereafter, the analysis unit 12 compares the appearance characteristics appearing in the image of the detected object (the characteristics of the area where the object is present in the image) with the appearance characteristics of each of the plurality of products registered in advance. , identify which product the object is (product recognition). For example, the analysis unit 12 may identify a product that has the highest degree of similarity in appearance to the object, or a product that has the highest degree of similarity in appearance to the object and for which the degree of similarity is equal to or higher than a reference value. Characteristics of the appearance of the product include, but are not limited to, color, surface unevenness, shape, and the like.

The output unit 13 displays information corresponding to the progress of the image analysis by the analysis unit 12 on the table (eg, on the mounting surface of the table). A plurality of pieces of information corresponding to the progress status are different from each other. For example, the output unit 13 displays a first mark M (e.g., a circle, etc.) on the stand as shown in FIG. A second mark M (eg, square, etc.) may be displayed on the stand. In addition, here we have shown an example in which information is displayed on the stand in response to the completion of object detection and product recognition, but it is also possible to display information on the stand in response to the image analysis reaching other conditions. Good too. Further, the information to be displayed is not limited to marks, but may be other information such as letters, numbers, pictures, characters, etc. Furthermore, the display mode (eg, color, shade) of the mark may be changed depending on the progress of image analysis. For example, when the analysis unit 12 recognizes that an object has been placed on the table, the output unit 13 displays a first color mark on the table, and when the analysis unit 12 recognizes the product, displays a second color mark on the table. Colored marks may be displayed on the stand.

When a display is provided on a part of the stand, the output unit 13 can display information corresponding to the progress status of the image analysis on the display. For example, as explained using FIG. 3, when the placement surface on which the product is placed is composed of a display (member 2), the output unit 13 displays the progress status of image analysis on the display (member 2). It is possible to display information corresponding to Alternatively, the output unit 13 may use a projection device to project information onto the stand.

Further, the output unit 13 can display information corresponding to the progress status of the image analysis at a position corresponding to the placement position of the object. For example, as shown in FIGS. 5 and 6, information (mark M) may be displayed surrounding the object placement position, or information (mark M) may be displayed near the object placement position as shown in FIG. Mark M) may also be displayed. For example, the output unit 13 stores in advance a conversion rule for converting "coordinates of an image" into "coordinates of a display area of a display that constitutes a mounting surface of a stand." Then, the output unit 13 converts the position (coordinates) of the object in the image into the coordinates of the display area of the display based on the conversion rule, and, for example, outputs information surrounding or close to the converted position. Display.

Similarly, the output unit 13 may store in advance a conversion rule for converting "coordinates of an image" into "coordinates of a projection area of a projection device." Then, the output unit 13 converts the position (coordinates) of the object in the image into the coordinates of the projection area of the projection device based on the conversion rule, and for example, surrounds the converted position or closes the position. Information may also be displayed.

The coordinates of an image are expressed in a coordinate system in which an arbitrary position within the image is the origin and arbitrary directions are defined as the x-axis direction and the y-axis direction. The coordinates of the display area of the display are expressed in a coordinate system in which an arbitrary position within the display area is the origin and arbitrary directions are defined as the x-axis direction and the y-axis direction. The coordinates of the projection area of the projection device are expressed in a coordinate system in which an arbitrary position within the projection area is the origin and arbitrary directions are defined as the x-axis direction and the y-axis direction.

The registration unit 14 registers the products recognized by the analysis unit 12 as accounting objects. For example, the registration unit 14 acquires product information of the product recognized by the analysis unit 12 from the product master, and registers at least a part of the acquired product information in the registration device 10. Product information includes price, product name, product code, etc.

According to the registration device 10 of the present embodiment described above, it is possible to recognize a product placed on a stand by image analysis, and to register the recognized product as a subject of accounting processing. According to such a registration device 10, product registration work becomes more efficient.

Further, the registration device 10 can display information corresponding to the progress status on the stand depending on the progress status of the image analysis. Note that the plural pieces of information corresponding to the progress status are different from each other. For example, the registration device 10 displays a first mark (e.g., circle, etc.) on the stand in response to completion of object detection, and displays a second mark (e.g., square, etc.) on the stand in response to completion of product recognition. May be displayed. According to such a registration device 10, the checkout operator can grasp the progress status of image analysis based on the information displayed on the table.

<Second embodiment>
The registration device 10 of this embodiment can display information on the stand at a timing different from that of the first embodiment. This will be explained in detail below.

An example of the hardware configuration of the registration device 10 is the same as that in the first embodiment.

An example of a functional block diagram of the registration device 10 is shown in FIG. 2, as in the first embodiment. The configurations of the image acquisition unit 11 and the registration unit 14 are similar to those in the first embodiment.

Similar to the first embodiment, the analysis unit 12 analyzes the image acquired by the image acquisition unit 11 and recognizes the product included in the image. The analysis unit 12 can perform "detection of an object," "recognition that an object has been placed on a stand," and "identification of which product the object placed on the stand is," in this order. can.

Object detection can be achieved using any technique such as image binarization and contour extraction.

Recognition that an object has been placed on the stand can be realized, for example, as follows. First, the analysis unit 12 is made to hold in advance information indicating the display area of the mounting surface of the table in the image. The display area may be indicated by the coordinates of a coordinate system with an arbitrary point in the image as the origin and arbitrary directions as the x and y axes (e.g., if the display area is a polygon, the coordinates of the vertices ). Part of the image may be the display area of the mounting surface of the stand, or the entire image may be the display area of the mounting surface of the stand. It is assumed that the installation position of the stand, the installation position of the camera, the orientation of the camera, etc. are fixed, and the relative positional relationship between the stand and the camera is constant. When the analysis unit 12 detects that the movement of the object present in the display area of the mounting surface of the stand has stopped for a predetermined period of time or more, it may recognize that the object has been placed on the stand.

Identification of which product the object placed on the stand is (product recognition) is based on the appearance characteristics that appear in the image of the detected object (characteristics of the area where the object is present in the image) and the pre-registered product. This is achieved by comparing the appearance characteristics of each of the multiple products available. For example, the analysis unit 12 may identify a product that has the highest degree of similarity in appearance to the object, or a product that has the highest degree of similarity in appearance to the object and for which the degree of similarity is equal to or higher than a reference value. Characteristics of the appearance of the product include, but are not limited to, color, surface unevenness, shape, and the like.

The output unit 13 displays information indicating that the analysis unit 12 has recognized that the object has been placed on the stand, and has identified which product the object placed on the stand is. For example, when the analysis unit 12 recognizes that the object is placed on the stand, the output unit 13 displays first information corresponding to the situation on the stand. Furthermore, when the analysis unit 12 identifies which product the object placed on the stand is, the output unit 13 displays second information corresponding to the situation on the stand. The first information and the second information are mutually different information.

Note that, when an object is detected by the analysis unit 12, the output unit 13 may display information corresponding to the situation (information different from the first and second information) on the stand.

Next, an example of the process flow of the registration device 10 of this embodiment will be described using the flowchart of FIG. 4.

When the process is started, the image acquisition unit 11 starts acquiring images from the camera. The analysis unit 12 then starts analyzing the image.

When the analysis unit 12 recognizes that the object is placed on the stand (Yes in S10), the output unit 13 displays first information corresponding to the situation on the stand (S11). The output unit 13 displays the first information at a position corresponding to the placement position of the object. Note that while the analysis unit 12 does not recognize that the object is placed on the stand (No in S10), the first information is not displayed on the stand.

Thereafter, when the analysis unit 12 specifies which product the object is (Yes in S12), the output unit 13 displays second information corresponding to the situation (S13). The output unit 13 displays the second information at a position corresponding to the placement position of the object. Further, in S13, the registration unit 14 registers the specified product as a subject of accounting. If the analysis unit 12 does not specify which product the object placed on the stand is (No in S12), the second information is not displayed on the stand.

Note that the output unit 13 may continue to display the first information until it displays the second information. When the analysis unit 12 specifies which product the object is (Yes in S12), the display of the first information is ended and the second information is displayed instead. good. Thereafter, the output unit 13 may continue to display the second information until the object is removed from the table or until payment processing is input. As another example, the output unit 13 may display the first information and the second information for a predetermined period of time, and may end the display of the first information and the second information according to the elapse of the predetermined period of time.

Thereafter, when the analysis unit 12 recognizes that the next object has been placed on the stand (S14: Yes), the output unit 13 displays the first information corresponding to the situation on the stand (S15). . The output unit 13 displays the first information at a position corresponding to the placement position of the object. Thereafter, when the analysis unit 12 specifies which product the object is (Yes in S16), the output unit 13 displays second information corresponding to the situation on the stand (S17). The output unit 13 displays the second information at a position corresponding to the placement position of the object. In S17, the registration unit 14 further registers the specified product as an object of accounting. Note that when a plurality of objects are placed on the stand, a plurality of pieces of first information may be displayed on the stand, a plurality of pieces of second information may be displayed on the stand, or a plurality of pieces of second information may be displayed on the stand. Both the information and the second information may be displayed.

If there is no new recognition/identification by the analysis unit 12 in S14 and S16 (No in S14, No in S16) and there is an input for payment processing (Yes in S18), the registration device 10 executes the payment processing ( S19). For example, the registration device 10 may accept input of cash as payment for the total payment amount calculated based on the products registered so far, and may output change or a receipt as necessary. The registration device 10 may also accept credit card input, communicate with a credit company system, and perform payment processing. Further, the registration device 10 may transmit information for payment processing (information indicating registered products, total payment amount, etc.) to another payment device.

According to the registration device 10 of this embodiment described above, the same effects as in the first embodiment can be achieved. Further, according to the registration device 10 of this embodiment, different information can be displayed on the stand depending on the recognition that an object is placed on the stand and the product recognition.

According to the registration device 10 of this embodiment, the checkout operator can understand that the placement of an object on the table has been recognized based on the information displayed on the table. Further, the accounting worker can understand that the product has been recognized. Since the accounting worker can grasp the progress of image analysis, he or she can proceed with the registration work without anxiety.

<Third embodiment>
The registration device 10 of this embodiment differs from the first and second embodiments in the information displayed on the stand when the product is recognized. This will be explained in detail below.

An example of the hardware configuration of the registration device 10 is the same as in the first and second embodiments.

An example of a functional block diagram of the registration device 10 is shown in FIG. 2 similarly to the first and second embodiments. The configurations of the image acquisition unit 11 and the registration unit 14 are similar to those in the first and second embodiments.

The analysis unit 12 identifies the product category of the recognized product. The other configurations of the analysis unit 12 are the same as those in the first and second embodiments. For example, product category information indicating the product category for each product is stored in the analysis unit 12 in advance. When the analysis unit 12 recognizes the product through image analysis, it refers to the product category information and identifies the product category of the recognized product.

The output unit 13 changes second information displayed on the stand when the analysis unit 12 identifies which product the object placed on the stand is, depending on the product category of the identified product. . For example, the output unit 13 may display a star mark on the stand when "food" is specified, and display a triangle mark on the stand when "daily miscellaneous goods" is specified. The other configurations of the output section 13 are the same as those in the first and second embodiments.

According to the registration device 10 of this embodiment described above, the same effects as those of the first and second embodiments can be achieved. Further, according to the registration device 10 of the present embodiment, the checkout operator can grasp not only that the product has been recognized but also the details of the recognition (product category) based on the information displayed on the counter. Based on this information, the accounting worker can check whether the recognized information is correct.

<Fourth embodiment>
The registration device 10 of this embodiment differs from the first and second embodiments in the information displayed on the stand when the product is recognized. This will be explained in detail below.

An example of the hardware configuration of the registration device 10 is the same as in the first and second embodiments.

An example of a functional block diagram of the registration device 10 is shown in FIG. 2 similarly to the first and second embodiments. The configurations of the image acquisition unit 11 and the registration unit 14 are similar to those in the first and second embodiments.

The analysis unit 12 outputs the reliability of the recognition result of product recognition. The reliability may be a similarity calculated by pattern matching (a similarity between a recognized product and an object included in an image). The other configurations of the analysis unit 12 are the same as those in the first and second embodiments.

The output unit 13 changes the information displayed on the stand when the analyzing unit 12 recognizes which product the object placed on the stand is, depending on the reliability of the recognition result. For example, reliability may be divided into multiple levels such as "high," "medium," and "low," and information may be prepared for each level. For example, the output unit 13 displays a double circle mark on the stand when the reliability is "high", displays a circle mark on the stand when the reliability is "medium", and displays a double circle mark on the stand when the reliability is "low". In such cases, a cross mark may be displayed on the stand. The other configurations of the output section 13 are the same as those in the first and second embodiments.

According to the registration device 10 of this embodiment described above, the same effects as those of the first and second embodiments can be achieved. Further, according to the registration device 10 of the present embodiment, the checkout operator is able to grasp not only that the product has been recognized but also the reliability of the product recognition based on the information displayed on the counter. If the reliability level is low, the accounting worker can check whether the recognized contents are correct. According to this embodiment, the checkout operator can check the recognition details only for products with low reliability, instead of checking whether the recognition details are correct for all products. As a result, the work effort can be saved.

<Fifth embodiment>
The registration device 10 of this embodiment can display information on the stand at a timing different from that of the first to fourth embodiments. This will be explained in detail below.

An example of the hardware configuration of the registration device 10 is the same as in the first to fourth embodiments.

An example of a functional block diagram of the registration device 10 is shown in FIG. 2, as in the first to fourth embodiments. The configurations of the image acquisition unit 11 and the registration unit 14 are the same as those in the first to fourth embodiments.

The analysis unit 12 continuously analyzes multiple frames (images) included in a moving image. Then, after recognizing the first product that was not recognized in the frame immediately before the first frame in the first frame, the analysis unit 12 determines that the recognition result for the frame after the first frame satisfies a predetermined condition. If the conditions are met, the recognition result that the first product is included in the image is determined. The predetermined condition may be that after the first product is recognized in the first frame, the first product is recognized for M consecutive frames (M is an integer of 1 or more). The other configurations of the analysis unit 12 are the same as those in the first to fourth embodiments.

When the analysis unit 12 recognizes the first product in the first frame, the output unit 13 displays third information on the stand. Then, when the analysis unit 12 determines that the predetermined condition is satisfied, the output unit 13 displays the second information on the stand. The third information is different from information corresponding to other situations of image analysis (eg, first information, second information). The other configurations of the output unit 13 are the same as those in the first to fourth embodiments.

After confirming the recognition result that the first product is included in the image, the registration unit 14 registers the first product as a transaction object.

According to the registration device 10 of this embodiment described above, the same effects as those of the first to fourth embodiments can be achieved. Further, according to the registration device 10 of the present embodiment, it is possible to determine the recognition result of a product included in an image based not only on the analysis result for one frame but also on the analysis result for a plurality of frames. As a result, the reliability of the analysis results can be increased.

Further, according to the registration device 10 of the present embodiment, the checkout operator can proceed with the process of determining the recognition result of the product included in the image based on the analysis results for a plurality of frames based on the information displayed on the table. Be able to grasp the situation. As a result, the accounting worker can understand the status of image analysis in more detail.

Here, a modification that is applicable to all embodiments will be described. The registration device 10 may include a switching unit that turns ON/OFF the display of information on the stand by the output unit 13. The switching unit can be switched ON/OFF according to user input. When the display of information on the table by the output unit 13 is ON, the output unit 13 displays information corresponding to the progress status of the image analysis by the analysis unit 12 on the table according to the progress status of the image analysis. On the other hand, when the display of information on the stand by the output unit 13 is OFF, the output unit 13 does not display information corresponding to the progress status of the image analysis by the analysis unit 12 on the stand. Customers or stores who find it bothersome to display information on the table can avoid displaying information on the table by operating the registration device 10 and setting it to OFF. In this way, according to the modified example, a user-friendly configuration can be realized.

Note that the switching section that turns ON/OFF the display of information on the stand by the output section 13 may have a function of turning ON/OFF the display of information regarding some product groups. For example, it may have a function of displaying information on the stand when a product belonging to a specific product category (fresh food, medicine, etc.) is identified. Alternatively, display of information on the stand may be restricted for products belonging to a specific product category. Alternatively, it may be a function of displaying information on the stand for products worth more than a predetermined amount, or a function of not displaying information on the stand for products worth less than a predetermined amount. The switching unit may be a physical switch, or a button for the switching unit may be displayed on a stand displaying information.

Other modifications applicable to all embodiments will be described. If the analysis unit 12 recognizes that an object has been placed on the stand, but does not identify which product the object is (if the product is not recognized), the output unit 13 outputs information indicating this. It may be displayed on the stand. This information is different from information corresponding to other situations of image analysis (eg, first information, second information, third information). The case where the product is not recognized may be, for example, a case where there is no product with the above-mentioned degree of similarity equal to or higher than the reference value. According to this modification, the checkout operator can grasp the situation where an object is detected but the product is not recognized based on the information displayed on the table. As a result, the status of image analysis can be understood in detail.

Other modifications applicable to all embodiments will be described. A product recognized by the analysis unit 12 (in the case of the fifth embodiment, the recognition result is finalized) and registered by the registration unit 14 (hereinafter referred to as "registered product") is a frame image after registration (e.g. product If it is no longer detected in the frame images after the recognized frame image or the frame images after the frame image for which the recognition result has been confirmed, the output unit 13 outputs the original placement position of the registered product (the last detected frame image). Information indicating this is displayed at the placement position specified in the frame image. The output unit 13 may change the form of the information displayed at the original placement position depending on the continuous time without detection (the number of continuous frames without detection). For example, the output unit 13 may display predetermined information (eg, a predetermined mark) at the original placement position, and gradually make the density of the information lighter as the continuous time increases. In addition, the output unit 13 gradually reduces the shading of the information (e.g., a predetermined mark) displayed at the original placement position while the registered product is being detected as the continuous time increases. Good too.

A placed product may not be detected temporarily due to obscurity or noise. When detection and non-detection alternate in consecutive frames, the information display changes discontinuously, making it difficult for the accounting operator to grasp the status of the recognition results. According to the discussion variant, this can be prevented.

Below, examples of reference forms will be added.
1. an image acquisition means for acquiring an image of the placement surface of the table on which the product is placed;
analysis means for recognizing the product included in the image;
a registration means for registering the recognized product as an accounting object;
output means for displaying information corresponding to the progress of the analysis of the image by the analysis means on the stand;
A registration device having a
2. In the registration device according to 1,
The output means is a registration device that displays the first information on the stand when the analysis means recognizes that the object is placed on the stand.
3. In the registration device according to 1 or 2,
The output means is a registration device that displays the second information on the stand when the product is recognized by the analysis means.
4. In the registration device described in 3.
The output means is a registration device that changes the second information depending on the recognized product category of the product.
5. In the registration device described in 3.
The output means is a registration device that changes the second information depending on the reliability of the recognition result of the product.
6. In the registration device according to any one of 3 to 5,
After recognizing the first product that was not recognized in the frame immediately before the first frame in the first frame, the analysis means determines that the recognition result for the frame after the first frame meets a predetermined condition. If the first product is included in the image, confirming the recognition result that the first product is included in the image;
The output means displays a third piece of information on the stand when the first product is recognized in the first frame by the analysis means, and when it is determined by the analysis means that the predetermined condition is satisfied. and a registration device that displays the second information on the stand.
7. In the registration device according to any one of 1 to 6,
The output means displays a mark of a first color on the stand when the analysis means recognizes that the object is placed on the stand, and displays a second mark when the analysis means recognizes the product. A registration device that displays a mark of a color on the stand.
8. In the registration device according to any one of 1 to 7,
The output means is a registration device for displaying information indicating that the product cannot be recognized on the stand when the image includes an object that cannot be recognized by the analysis means.
9. In the registration device according to any one of 1 to 8,
When the product registered by the registration device is no longer detected in images after registration, the output device displays predetermined information at the placement position specified in the last detected image, and displays a continuous period of time during which the product is not detected. A registration device that thins out the information as it becomes larger.
10. The computer is
an image acquisition step of acquiring an image of the placement surface of the table on which the product is placed;
an analysis step of recognizing the product included in the image;
a registration step of registering the recognized product as an accounting object;
an output step of displaying information corresponding to the progress of the analysis of the image in the analysis step on the stand;
How to register to perform.
11. computer,
image acquisition means for acquiring an image of the placement surface of the table on which the product is placed;
analysis means for recognizing the product included in the image;
registration means for registering the recognized product as an accounting object;
output means for displaying information corresponding to the progress of the analysis of the image by the analysis means on the stand;
A program that functions as

Claims (9)

an image acquisition means for acquiring an image of the placement surface of the table on which the product is placed;
analysis means for recognizing the product included in the image;
a registration means for registering the recognized product as an accounting object;
Output means for displaying information corresponding to the progress of the analysis of the image by the analysis means on the stand using a projection device or using the display if the stand is equipped with a display;
has
When the product registered by the registration means is no longer detected in the image after registration, the output means displays predetermined information at the placement position specified in the image before registration, and displays a continuous period of time during which the product is not detected. A registration device that thins out the information as it becomes larger. The registration device according to claim 1,
The output means is a registration device that displays first information on the stand as information corresponding to the progress status when the analysis means recognizes that the object has been placed on the stand. The registration device according to claim 1 or 2,
The output means is a registration device that displays second information on the stand as information corresponding to the progress status when the product is recognized by the analysis means. In the registration device according to claim 3 depending on claim 2 ,
The output means displays a mark of a first color on the stand as the first information when the analysis means recognizes that the object is placed on the stand, and the analysis means displays the mark of the product on the stand. A registration device that, when recognized, displays a mark of a second color on the stand as the second information. In the registration device according to claim 3 or 4 ,
The output means is a registration device that changes the second information depending on the recognized product category of the product. In the registration device according to claim 3 or 4 ,
The output means is a registration device that changes the second information depending on the reliability of the recognition result of the product. The registration device according to any one of claims 1 to 6,
The output means is a registration device for displaying information indicating that the product cannot be recognized on the stand when the image includes an object that cannot be recognized by the analysis means. The computer is
an image acquisition step of acquiring an image of the placement surface of the table on which the product is placed;
an analysis step of recognizing the product included in the image;
a registration step of registering the recognized product as an accounting object;
an output step of displaying information corresponding to the progress of the analysis of the image in the analysis step on the stand using a projection device or using the display if the stand is equipped with a display;
Run
In the output step, if the product registered in the registration step is no longer detected in the image after registration, predetermined information is displayed at the placement position specified in the image before registration, and the continuous time when the product is not detected is displayed. A registration method in which the information becomes thinner as the value increases. computer,
image acquisition means for acquiring an image of the placement surface of the table on which the product is placed;
analysis means for recognizing the product included in the image;
registration means for registering the recognized product as an accounting object;
Output means for displaying information corresponding to the progress of the analysis of the image by the analysis means on the stand using a projection device or using the display if the stand is equipped with a display;
function as
When the product registered by the registration means is no longer detected in the image after registration, the output means displays predetermined information at the placement position specified in the image before registration, and displays a continuous period of time during which the product is not detected. A program that thins out the information as it becomes larger.

Images